Skip to content

Selective Resource Type Loading

Runtime tab

The Runtime tab describes the Omni Connector on a Runtime Plane, which is a separate connector from PolicySync. Sections without a Runtime tab apply to PolicySync only.

This configuration allows you to selectively enable or disable loading of specific resource types when your access token doesn't have sufficient permissions to access certain resources in Databricks Unity Catalog. This prevents UNAUTHORIZED_ACCESS or PERMISSION_DENIED errors during resource and permission loading.

Configuration Parameters

Property Default Description
CONNECTOR_DATABRICKS_UNITY_CATALOG_LOAD_METASTORE_ENABLED true Enable/disable loading of metastore resources
CONNECTOR_DATABRICKS_UNITY_CATALOG_LOAD_CONNECTION_ENABLED true Enable/disable loading of connection resources
CONNECTOR_DATABRICKS_UNITY_CATALOG_LOAD_SERVICE_CREDENTIAL_ENABLED true Enable/disable loading of service credential resources
CONNECTOR_DATABRICKS_UNITY_CATALOG_LOAD_STORAGE_CREDENTIAL_ENABLED true Enable/disable loading of storage credential resources
CONNECTOR_DATABRICKS_UNITY_CATALOG_LOAD_EXTERNAL_LOCATION_ENABLED true Enable/disable loading of external location resources
CONNECTOR_DATABRICKS_UNITY_CATALOG_LOAD_VOLUME_ENABLED false Enable/disable loading of volume resources

Note

If metastore loading is enabled, then volumes will be loaded by default.

Setup

  1. Navigate to Settings → Applications in the Self-Managed Portal.

  2. From the list of Connected Applications, select Databricks Unity Catalog.

  3. Click on the application name or the icon to edit. Then, go to the Access Management tab.

  4. Under the ADVANCED tab, add the following properties under Add New Custom Properties to disable specific resource types:

    Bash
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
    # Disable metastore loading if token doesn't have metastore admin permissions
ranger.policysync.connector.0.load.metastore.enable=false

# Disable connection loading
ranger.policysync.connector.0.load.connection.enable=false

# Disable service credential loading
ranger.policysync.connector.0.load.service_credential.enable=false

# Disable storage credential loading
ranger.policysync.connector.0.load.storage_credential.enable=false

# Disable external location loading
ranger.policysync.connector.0.load.external_location.enable=false

# Disable volume loading
ranger.policysync.connector.0.load.volume.enable=false

  5. Click SAVE to apply the changes.

  1. SSH to the instance where Privacera Manager is installed.

  2. Run the following command to open the .yml file to be edited.

    If you have multiple connectors, then replace instance1 with the appropriate connector instance name.

    Bash
    1
    vi ~/privacera/privacera-manager/config/custom-vars/connectors/databricks-unity-catalog/instance1/vars.connector.databricks.unity.catalog.yml

  3. Set the following properties to disable loading of specific resource types:

    YAML
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
    # Disable metastore loading if token doesn't have metastore admin permissions
CONNECTOR_DATABRICKS_UNITY_CATALOG_LOAD_METASTORE_ENABLED: "false"

# Disable connection loading
CONNECTOR_DATABRICKS_UNITY_CATALOG_LOAD_CONNECTION_ENABLED: "false"

# Disable service credential loading
CONNECTOR_DATABRICKS_UNITY_CATALOG_LOAD_SERVICE_CREDENTIAL_ENABLED: "false"

# Disable storage credential loading
CONNECTOR_DATABRICKS_UNITY_CATALOG_LOAD_STORAGE_CREDENTIAL_ENABLED: "false"

# Disable external location loading
CONNECTOR_DATABRICKS_UNITY_CATALOG_LOAD_EXTERNAL_LOCATION_ENABLED: "false"

# Disable volume loading
CONNECTOR_DATABRICKS_UNITY_CATALOG_LOAD_VOLUME_ENABLED: "false"

  4. Once the properties are configured, run the following commands to update your Privacera Manager platform instance:

    Step 1 - Setup which generates the helm charts. This step usually takes few minutes.

    Bash
    1
2
    cd ~/privacera/privacera-manager
./privacera-manager.sh setup
    Step 2 - Apply the Privacera Manager helm charts.
    Bash
    1
2
    cd ~/privacera/privacera-manager
./pm_with_helm.sh upgrade
    Step 3 - (Optional) Post-installation step which generates Plugin tar ball, updates Route 53 DNS and so on. This step is not required if you are updating only connector properties.

    Bash
    1
2
    cd ~/privacera/privacera-manager
./privacera-manager.sh post-install

  1. In PrivaceraCloud portal, navigate to Settings -> Applications.

  2. On the Connected Applications screen, select Databricks Unity Catalog.

  3. Click the pen icon or the Account Name to modify the settings.

  4. On the Edit Application screen, go to Access Management -> ADVANCED tab.

  5. Under Add New Custom Properties, add the following properties to disable specific resource types:

    Bash
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
    # Disable metastore loading if token doesn't have metastore admin permissions
ranger.policysync.connector.0.load.metastore.enable=false

# Disable connection loading
ranger.policysync.connector.0.load.connection.enable=false

# Disable service credential loading
ranger.policysync.connector.0.load.service_credential.enable=false

# Disable storage credential loading
ranger.policysync.connector.0.load.storage_credential.enable=false

# Disable external location loading
ranger.policysync.connector.0.load.external_location.enable=false

# Disable volume loading
ranger.policysync.connector.0.load.volume.enable=false

  6. Click SAVE to apply the changes.

In the Omni Connector, the resource types that are loaded are controlled by Resource types governed. Only the listed types, and the types needed to reach them, are scanned and have their permissions read. Types that you leave out are not loaded, so they cannot cause UNAUTHORIZED_ACCESS or PERMISSION_DENIED errors.

  1. In the portal, navigate to Settings → Runtime Plane and open your Runtime Plane.

  2. Open the Connectors tab and open your Databricks Unity Catalog connector.

  3. Go to ACCESS CONTROL CONFIGS and expand Managed Resources.

  4. Under Resource types governed, add only the types that your credential can access. Leave it empty to use the defaults: SCHEMA, TABLE and VIEW. For example, to also load volumes and functions:

    Text Only
    1
    SCHEMA, TABLE, VIEW, VOLUME, FUNCTION

  5. Select Save.

Resource type Loaded by default
SCHEMA, TABLE, VIEW Yes
CATALOG, MATERIALIZED_VIEW, VOLUME, FUNCTION, MODEL (listed, not scanned yet) No
EXTERNAL_LOCATION, STORAGE_CREDENTIAL, CONNECTION, SERVICE_CREDENTIAL, EXTERNAL_METADATA (listed, not scanned yet), CLEAN_ROOM (listed, not scanned yet) No

For more details, see Managed resources.