Skip to content

Setup Access Management for Databricks Unity Catalog

This section outlines the steps to set up the Databricks Unity Catalog connector in Privacera. Ensure that all prerequisites are met before proceeding.

Best Practice for enabling the connector

This connector will manage the access control policies for the Databricks Unity Catalog. If there are existing policies in Unity Catalog, then it they will be overwritten by the policies from Privacera. It is strongly recommended that to test managing few resources initially before enabling the connector for all resources. You could create a test catalog in Databricks Unity Catalog and manage the access control policies for that catalog, and after you are comfortable you can enable the connector for all catalogs.

Create Instance of Databricks Unity Catalog

Note

The Ops Server Setup is required for the Self-Managed (Portal Configuration) setup.

  1. Navigate to Settings → Applications in the Self-Managed Portal.

  2. Select Databricks Unity Catalog connector under Available Connections.

  3. Enter the application Name and Description, then click Next.

    • The name can be any meaningful identifier, such as uc-connector-1.
    • The description can provide additional context, e.g. Unity Catalog Connector for dev account xyz.

  4. Click on the application name or icon. Enable the Access Management toggle button.

  1. SSH to the instance where Privacera Manager is installed.

  2. Run the following command to navigate to the /config directory.

    Bash
    1
    cd ~/privacera/privacera-manager/config

  3. Create a new directory for the Databricks Unity Catalog connector configuration.

    Note

    In the example below, instance1 is the name of the connector instance. You can change this name to uniquely identify your installed connector configuration. The connector instance name should consist of only hyphens and alphanumeric characters.

    Bash
    1
    mkdir -p custom-vars/connectors/databricks-unity-catalog/instance1

  4. Copy the sample connector configuration file to your custom directory:

    Bash
    1
    cp -n sample-vars/vars.connector.databricks.unity.catalog.yml custom-vars/connectors/databricks-unity-catalog/instance1/

  5. Run the following command to open the .yml file to be edited:

    Bash
    1
    vi custom-vars/connectors/databricks-unity-catalog/instance1/vars.connector.databricks.unity.catalog.yml

  1. In PrivaceraCloud, navigate to Settings -> Applications.

  2. On the Applications screen, select Databricks Unity Catalog application under Available connections.

  3. Enter the application Name and Description, then click Save. Name could be any name of your choice. E.g. Databricks Unity Catalog Connector for account 123456.

  4. Open the Databricks Unity Catalog application.

  5. Enable the Access Management option with toggle button.

Connection Details

  1. Specify the Databricks URL. For example, https://dev-environment.cloud.databricks.com.
  2. Provide the Databricks personal access token used to connect to the Databricks API.
  3. Account Admin Privileges for Personal Access Token: Set this to true if the personal access token has account admin privileges.
  4. Specify the JDBC URL for Databricks. This value is required for Unity Catalog’s native masking, row filtering, and access auditing features.
Default DBX API Token Privileges

The DBX API token user is granted the following default privileges on the Catalog:

  • Use Catalog
  • Create Schema
  • Use Schema
  • Create Table
  • Select
  • Modify
  • Create Function

Replace the below example values with your actual values.

Enter these fields under the BASIC tab:

Note

  • To view the property details, click the icon next to the property name.
  • Fields with asterisks (*) are mandatory.
  • Requires restart indicates that the connector pod will restart after updating the property.
  • Cleans RocksDB: When this property is updated, the connector pod initiates a cleanup process to remove cached data and other internal resources from RocksDB. The duration of this process may vary depending on the volume of data and resources being managed.
  • Databricks Workspace Base URL: https://dev-environment.cloud.databricks.com
  • Databricks personal access token: <databricks-access-token>
  • Databricks JDBC url: <jdbc-url>

OAuth Authentication

If you are using OAuth authentication for Databricks, then for the token, you need to follow the instructions from the OAuth Token Configuration section in the Advanced Configuration.

YAML
1
2
3
4
CONNECTOR_DATABRICKS_UNITY_CATALOG_DATABRICKS_BASE_URL: "https://<workspace-url>.cloud.databricks.com"
CONNECTOR_DATABRICKS_UNITY_CATALOG_JDBC_URL: "<jdbc-url>"
CONNECTOR_DATABRICKS_UNITY_CATALOG_API_ACCESS_TOKEN: "<databricks-access-token>"
CONNECTOR_DATABRICKS_UNITY_CATALOG_IS_API_ACCESS_TOKEN_ACCOUNT_ADMIN: "true"

Enter these fields under the BASIC tab:

  • Databricks Workspace Base URL: https://dev-environment.cloud.databricks.com
  • Databricks personal access token: <databricks-access-token>
  • Enable if the personal access token has account admin privileges: true
  • Databricks JDBC url: <jdbc-url>

Managed Catalogs

This property is used to set comma-separated catalog names for which access control policies will be applied. Wildcards are supported.

Example: test_catalog1,test_catalog2,sales_*.

Warning

  • Values are case-sensitive.
  • Replace the below example value with your actual value.

It is recommended to first test the connector with a test catalog before enabling it for all catalogs.

Under the BASIC tab, enter the values for:

  • Catalogs to set access control policies: test_catalog
YAML
1
CONNECTOR_DATABRICKS_UNITY_CATALOG_MANAGE_CATALOG_LIST: "test_catalog"

Under the BASIC tab, enter the values for:

  • Catalogs to set access control policies: test_catalog

Apply the Configuration

  1. Once all the required fields are filled, click Save.

  2. Download the configuration as a YML file by clicking the icon in the Action column.

  3. Copy the downloaded zip file to the following location on Privacera Manager instance:

    Bash
    1
    ~/privacera/privacera-manager/config/custom-vars/

  4. SSH to the Privacera Manager instance and navigate to the location where zip file is copied and unzip the file:

    Bash
    1
2
    cd ~/privacera/privacera-manager/config/custom-vars/
unzip <file_name>.zip

    Note

    • Backup existing configurations if needed.
    • Ensure unzip is installed on the system.
    • The replacement will only replace files with the same name; other files within the folder remain unchanged.

  5. After all the changes are done you can start the connector by running the following instructions:

    Step 1 - Setup which generates the helm charts. This step usually takes few minutes.

    Bash
    1
2
    cd ~/privacera/privacera-manager
./privacera-manager.sh setup
    Step 2 - Apply the Privacera Manager helm charts.
    Bash
    1
2
    cd ~/privacera/privacera-manager
./pm_with_helm.sh upgrade
    Step 3 - Post-installation step which generates Plugin tar ball, updates Route 53 DNS and so on.

    Bash
    1
2
    cd ~/privacera/privacera-manager
./privacera-manager.sh post-install

  6. Once the connector is started, you can manage and monitor it using the Self-Managed portal.

After all the changes are done you can start the connector by running the following instructions:

Step 1 - Setup which generates the helm charts. This step usually takes few minutes.

Bash
1
2
cd ~/privacera/privacera-manager
./privacera-manager.sh setup
Step 2 - Apply the Privacera Manager helm charts.
Bash
1
2
cd ~/privacera/privacera-manager
./pm_with_helm.sh upgrade
Step 3 - Post-installation step which generates Plugin tar ball, updates Route 53 DNS and so on.

Bash
1
2
cd ~/privacera/privacera-manager
./privacera-manager.sh post-install

  1. Once all the required fields are filled, click Save.

  2. The configured Databricks Unity Catalog connector appears under Connected Applications.

  3. Once saved and enabled, the Databricks Unity Catalog connector will start. Then you can hover on the VIEW LOGS button to check the status, either Running or Stopped.

  4. Perform the following steps to restart the Databricks Unity Catalog connector application:

    1. Go to Settings → Applications → select the Databricks Unity Catalog connector application.

    2. Edit the application by disabling the Access Management option with toggle button and then Save it.

    3. Open the same application again and then enable the Access Management option with toggle button and then Save it.

Comments