Skip to content

Manage Resources List

You can configure the Databricks Unity Catalog connector to manage access control policies for specific schemas, tables/views, and functions. You can specify lists to include and exclude resources. The connector manages access control policies for resources in the include list and ignores resources in the exclude list. If a resource is in the exclude list, the connector does not manage it, even if it is also in the include list.

Use the following properties to specify comma-separated schemas, tables/views, and functions whose access control should be managed by PolicySync. To manage all resources, do not specify these properties. You can use wildcard characters (*) to match multiple schemas, tables, or functions.

Example:

  • Schemas: test_catalog1.schema1,test_catalog2*.sales*
  • Tables/Views: test_catalog1.schema1.table1,test_catalog2*.sales*.view*
  • Functions: test_catalog1.schema1.function1,test_catalog2*.sales*.func*

Prerequisites

For managing external locations and storage credentials, the service user must have the MANAGE privilege on the external location and storage credential. Here is an example of how to grant the MANAGE privilege:

If you have provided the METASTORE ADMIN privilege to the service user, then you can skip these privilege.

For external locations:

SQL
1
GRANT MANAGE ON EXTERNAL LOCATION `LOCATION_NAME` TO `PRIVACEA_SERVICE_USER`;

For storage credentials:

SQL
1
GRANT MANAGE ON STORAGE CREDENTIAL `STORAGE_CREDENTIAL_NAME` TO `PRIVACEA_SERVICE_USER`;

Setup

Warning

  • Values are case-sensitive.
  • Provide fully qualified names for schemas, tables/views, and functions. E.g. catalog1.schema1.*
  • Replace the example values with your actual resource names.

  1. SSH to the instance where Privacera Manager is installed.

  2. Run the following command to open the .yml file to be edited.

    If you have multiple connectors, then replace instance1 with the appropriate connector instance name.

    Bash
    1
    vi ~/privacera/privacera-manager/config/custom-vars/connectors/databricks-unity-catalog/instance1/vars.connector.databricks.unity.catalog.yml

  3. Set the following properties to enable the connector to manage the permissions for schemas, tables/views, and other resources in the Databricks Unity Catalog:

    YAML
    1
2
3
4
5
6
    CONNECTOR_DATABRICKS_UNITY_CATALOG_MANAGE_CATALOG_LIST: "test_catalog"
CONNECTOR_DATABRICKS_UNITY_CATALOG_MANAGE_SCHEMA_LIST: "test_catalog1.schema1"
CONNECTOR_DATABRICKS_UNITY_CATALOG_MANAGE_TABLE_LIST: "test_catalog1.schema1.table1"
CONNECTOR_DATABRICKS_UNITY_CATALOG_MANAGE_FUNCTION_LIST: "test_catalog1.schema1.function1"
CONNECTOR_DATABRICKS_UNITY_CATALOG_MANAGE_EXTERNAL_LOCATION_LIST: "sales_location*"
CONNECTOR_DATABRICKS_UNITY_CATALOG_MANAGE_STORAGE_CREDENTIAL_LIST: "sales_location*"

  4. For excluding resources, set the following properties:

    YAML
    1
2
3
4
5
6
    CONNECTOR_DATABRICKS_UNITY_CATALOG_IGNORE_CATALOG_LIST: "test_catalog1"
CONNECTOR_DATABRICKS_UNITY_CATALOG_IGNORE_SCHEMA_LIST: "test_catalog1.schema1"
CONNECTOR_DATABRICKS_UNITY_CATALOG_IGNORE_TABLE_LIST: "test_catalog1.schema1.table1"
CONNECTOR_DATABRICKS_UNITY_CATALOG_IGNORE_FUNCTION_LIST: "test_catalog1.schema1.function1"
CONNECTOR_DATABRICKS_UNITY_CATALOG_IGNORE_EXTERNAL_LOCATION_LIST: "external_location1"
CONNECTOR_DATABRICKS_UNITY_CATALOG_IGNORE_STORAGE_CREDENTIAL_LIST: "storage_credential1"

  5. Once the properties are configured, run the following commands to update your Privacera Manager platform instance:

    Step 1 - Setup which generates the helm charts. This step usually takes few minutes.

    Bash
    1
2
    cd ~/privacera/privacera-manager
./privacera-manager.sh setup
    Step 2 - Apply the Privacera Manager helm charts.
    Bash
    1
2
    cd ~/privacera/privacera-manager
./pm_with_helm.sh upgrade
    Step 3 - Post-installation step which generates Plugin tar ball, updates Route 53 DNS and so on.

    Bash
    1
2
    cd ~/privacera/privacera-manager
./privacera-manager.sh post-install

  1. In PrivaceraCloud portal, navigate to Settings -> Applications.

  2. On the Connected Applications screen, select Databricks Unity Catalog.

  3. Click the pen icon or the Account Name to modify the settings.

  4. On the Edit Application screen, go to Access Management -> ADVANCED tab.

  5. For including resources, enter the following values in the respective fields:

    • Schemas to set access control policies: test_catalog1.schema1
    • Tables to set access control policies: test_catalog1.schema1.table1
    • User defined functions to set access control policies: test_catalog1.schema1.function1
    • External locations to set access control policies: Enable this option to manage permissions for external locations.

  6. For excluding resources, enter the following values in the respective fields:

    • Catalogs to ignore while setting access control policies: test_catalog1
    • Schemas to ignore while setting access control policies: test_catalog1.schema1
    • Tables to ignore while setting access control policies: test_catalog1.schema1.table1
    • User defined functions to ignore while setting access control policies: test_catalog1.schema1.function1
    • External locations to ignore while setting access control policies: external_location1
    • Storage credentials to ignore while setting access control policies: storage_credential1

  7. Click SAVE to apply the changes.

Managing Unity Catalog Metastore with Multiple Privacera Unity Catalog Connectors

In Unity Catalog, multiple workspaces within the same Databricks account can reference the same set of catalogs. To prevent conflicts in access management policies, follow these best practices:

  • Use a single connector to manage all catalogs within the Databricks account.
  • If deploying one connector per workspace, ensure that each catalog is managed by only one workspace to avoid overlapping policies.
  • The system catalog should be managed by only one connector, and all other connectors should explicitly exclude it.

To maintain consistent access control, each resource should be managed by only one connector or workspace. Any additional connectors must be configured to ignore the resource to prevent conflicting policies.

Example: System Catalog Management in Unity Catalog

If multiple Databricks workspaces share the System Catalog, only one Unity Catalog (UC) connector should manage its permissions. All other UC connectors must exclude the System Catalog to avoid conflicting policies, which could lead to intermittent access revokes and inconsistent permissions.

The following steps demonstrate how to configure the UC connector to manage the System Catalog.

The following steps apply to other connectors and resources, but the specific configurations may vary depending on the resource being managed.

  1. SSH to the instance where Privacera Manager is installed.

  2. Run the following command to open the .yml file to be edited.

    Update <instance_name> with the appropriate connector instance name.

    Bash
    1
    vi ~/privacera/privacera-manager/config/custom-vars/connectors/databricks-unity-catalog/<instance_name>/vars.connector.databricks.unity.catalog.yml

  3. On the UC connector designated to manage a resource, set the following property:

    YAML
    1
    CONNECTOR_DATABRICKS_UNITY_CATALOG_MANAGE_CATALOG_LIST: "system"

  4. On all other UC connectors, configure them to ignore the resource:

    YAML
    1
    CONNECTOR_DATABRICKS_UNITY_CATALOG_IGNORE_CATALOG_LIST: "system"

  5. Once the properties are configured, run the following commands to update your Privacera Manager platform instance:

    Step 1 - Setup which generates the helm charts. This step usually takes few minutes.

    Bash
    1
2
    cd ~/privacera/privacera-manager
./privacera-manager.sh setup
    Step 2 - Apply the Privacera Manager helm charts.
    Bash
    1
2
    cd ~/privacera/privacera-manager
./pm_with_helm.sh upgrade
    Step 3 - Post-installation step which generates Plugin tar ball, updates Route 53 DNS and so on.

    Bash
    1
2
    cd ~/privacera/privacera-manager
./privacera-manager.sh post-install

  1. In PrivaceraCloud portal, navigate to Settings -> Applications.

  2. On the Connected Applications screen, select Databricks Unity Catalog.

  3. For the UC connector managing the resource (e.g., system catalog):

    1. Click the pen icon or the Account Name to modify the settings.

    2. On the Edit Application screen, go to Access Management -> BASIC tab.

    3. Update Catalogs to set access control policies: system.

    4. Click SAVE to apply the changes.

  4. For all other UC connectors, configure them to ignore the resource:

    1. Click the pen icon or the Account Name to modify the settings.

    2. On the Edit Application screen, go to Access Management -> ADVANCED tab.

    3. Update Catalogs to ignore while setting access control policies: system.

    4. Click SAVE to apply the changes.

  5. Restart the Databricks Unity Catalog connector application:

    1. Disable the Access Management toggle and save.
    2. Re-enable the Access Management toggle, then save it.

Comments