Skip to main content

Privacera Documentation

Integrate ADLS with Privacera Platform using the Data Access Server

You can integrate Azure Data Lake Storage (ADLS) with Privacera Platform using the Privacera Data Access Server.

Prerequisites

Ensure that the following prerequisites are met:

  • You have access to an Azure Storage account along with required credentials. For more information on how to set up an Azure storage account, see Create a storage account.

  • You have the values for the Application (client) ID and Client secrets Azure properties.

Procedure
  1. Go to the privacera-manager folder in your virtual machine. Open the config folder, copy the sample vars.dataserver.azure.yml file to the custom-vars/ folder, and edit it.

    cd ~/privacera/privacera-manager
    cp config/sample-vars/vars.dataserver.azure.yml config/custom-vars/
    vi custom-vars/vars.dataserver.azure.yml
    
  2. Edit the Azure-related information. For property details and description, see Azure ADLS configuration properties.

    1. If you want to use Azure CLI, use the following properties:

      ENABLE_AZURE_CLI: "true"
      AZURE_GEN2_SHARED_KEY_AUTH: "true"
      AZURE_ACCOUNT_NAME: "<PLEASE_CHANGE>"
      AZURE_SHARED_KEY: "<PLEASE_CHANGE>"
      
    2. If you want to access multiple Azure storage accounts with shared key authentication, use the following properties:

      AZURE_GEN2_SHARED_KEY_AUTH: "true"
      AZURE_ACCT_SHARED_KEY_PAIRS: "<PLEASE_CHANGE>"
      

      Note

      Configuring AZURE_GEN2_SHARED_KEY_AUTH property allows you to access the resources in the Azure accounts only through the File Explorer in Privacera Portal.

    3. If you want to access multiple Azure storage accounts with OAuth application based authentication, use the following properties:

      AZURE_GEN2_SHARED_KEY_AUTH: "false"
      DATASERVER_AZURE_APP_CLIENT_CONFIG_LIST: 
      - index: 0   
        tenantId: "<PLEASE_CHANGE>"   
        subscriptionId: "<PLEASE_CHANGE>"   
        resourceGroup: "<PLEASE_CHANGE>"   
        clientId: "<PLEASE_CHANGE>"   
        clientSecret: "<PLEASE_CHANGE>"   
        storageAccName: "<PLEASE_CHANGE>"

      Note

      Configuring the AZURE_GEN2_SHARED_KEY_AUTH property allows you to access the resources in the Azure accounts only through the File Explorer in Privacera Portal.

      Note

      You can also add custom properties that are not included by default. For more information, see Custom Data Access Server properties.

  3. Run the following command.

    cd ~/privacera/privacera-manager
    ./privacera-manager.sh update
    

Azure ADLS configuration properties

Property Name

Description

Example

ENABLE_AZURE_CLI

Uncomment to use Azure CLI.

The AZURE_ACCT_SHARED_KEY_PAIRS property wouldn't work with this property. So, you have set the AZURE_ACCOUNT_NAME and AZURE_SHARED_KEY properties.

true

AZURE_GEN2_SHARED_KEY_AUTH

For AZURE_GEN2_SHARED_KEY_AUTH property, use shared key authentication. Set it to true.

To use multiple Azure storage accounts with shared key authentication, then set this property to true, along with AZURE_ACCT_SHARED_KEY_PAIRS.

To use multiple Azure storage accounts with OAuth authentication, then set this property to false, along with DATASERVER_AZURE_APP_CLIENT_CONFIG_LIST.

true

AZURE_ACCOUNT_NAME

Azure ADLS storage account name

company-qa-dept

AZURE_SHARED_KEY

Azure ADLS storage account shared access key

=0Ty4br:2BIasz>rXm{cqtP8hA;7|TgZZZuTHJTg40z8E5z4UJ':roeJy=d7*/W"

AZURE_ACCT_SHARED_KEY_PAIRS

Comma-separated multiple storage account names and its shared keys.

The format must be:

${storage_account_name_1}:${secret_key_1},${storage_account_name_2}:${secret_key_2}

accA:sharedKeyA, accB:sharedKeyB

DATASERVER_AZURE_APP_CLIENT_CONFIG_LIST:
- index: 0   
  tenantId: "<PLEASE_CHANGE>"   
  subscriptionId: "<PLEASE_CHANGE>"   
  resourceGroup: "<PLEASE_CHANGE>"   
  clientId: "<PLEASE_CHANGE>"   
  clientSecret: "<PLEASE_CHANGE>"   
  storageAccName: "<PLEASE_CHANGE>" 
                                    

Configure multiple OAuth Azure applications and the storage accounts mapped with the configured client id and subscription.

The clientSecret property must be in BASE64 format in the YAML file.

DATASERVER_AZURE_APP_CLIENT_CONFIG_LIST: 
- index: 0   
  tenantId: "su88xx-xxxx-xxxx-xxxxfe"   
  subscriptionId: "khkjxx-xxxx-xxxx-xxxjk"   
  resourceGroup: "priv-group-1"   
  clientId: "8c08xxxx-xxxx-xxxx-xxxx-6w0c95v0xxxx"   
  clientSecret: "WncwSaMxxxxxxxxxxxxxxxxxx0FfVAo="   
  storageAccName: "storageAccA,storageAccB" 
- index: 1   
  tenantId: "yf8xx-xxxx-xxxx-xxxxgu"   
  subscriptionId: "jixx-xxxx-xxxx-xxxmh"   
  resourceGroup: "priv-group-2"   
  clientId: "5d37xxxx-xxxx-xxxx-xxxx-7z0cu7e0xxxx"   
  clientSecret: "ZncwSaMplxxxxxxxxxxxxxxxdVN0FfVAo="   
  storageAccName: "storageAccC"

Azure ADLS configuration property values

Property

Configuration Values

tenantId

Go to Azure portal > Azure Active Directory > Properties >Tenant ID

subscriptionId

Go to Azure portal > Click Subscriptions on the left nav > Select whichever subscription is needed > Click on Overview > Copy the Subscription ID.

resourceGroup

Go to Azure portal > Storage accounts > Select the storage account you want to configure > Click on Overview > Resource Group

clientId

Go to Azure portal > Azure Active Directory > Properties > Client ID

clientSecret

Go to Azure portal > Azure Active DirectoryCertificates &amp; secrets > New client secret > add  > Secrete value (It should be base64 encoded when adding into vars.dataserver.azure.yml)

storageAccName

Comma separated storage accounts.

ADLS validation

All accesses or attempted accesses (Allowed and Denied) for Azure ADLS resources will now be recorded to the audit stream. This Audit stream can be reviewed on the Audit page. The default access for a data repository is 'Denied'.

To verify Privacera Data Management control, follow these steps:

  1. Log in to the Privacera Portal as a portal administrator,

  2. From the navigation menu, select Data Inventory > Data Explorer.

  3. View the targeted ADLS files or folders. The data will be hidden and a Denied status will be registered in the Audit page.

  4. From the navigation menu, select Access Management > Resource Policies.

  5. Open System 'ADLS' and 'application' (data repository) 'privacera_adls'.

  6. Create or modify an access policy to allow access to some or all of your ADLS storage.

  7. Return to Data Inventory > Data Explorer.

  8. View the data as allowed by your new policy or policy change. Repeat step 1.

    You can now view files or folders in the account. An Allowed status will be registered in the Audit page.

To check the log in the Audit page in Privacera Portal, select Access Management > Audit from the navigation menu.