Skip to content

Skip Foreign Catalogs in Scheduled Scans

This document explains how to configure Databricks Unity Catalog connector to skip foreign catalogs during scheduled resource scans.

Overview

Enable this feature to exclude foreign catalogs from scheduled scans in theDatabricks Unity Catalog connector. When enabled, foreign catalog resources are only scanned when on-demand events for those resources are explicitly sent to the Ops Server.

Configuration property

  • Property Name:
    CONNECTOR_DATABRICKS_UNITY_CATALOG_SCHEDULED_LOAD_FOREIGN_RESOURCES

    Value Behavior
    false(default) Foreign catalog resources are included in scheduled syncs
    true Foreign catalog resources are skipped in scheduled syncs

Warning

When CONNECTOR_DATABRICKS_UNITY_CATALOG_SCHEDULED_LOAD_FOREIGN_RESOURCES is set to true, you must manually send events for foreign catalog resources to the Ops Server to load those resources.

Setup

  1. Navigate to Settings → Applications in the Self-Managed Portal.

  2. From the list of Connected Applications, select Databricks Unity Catalog.

  3. Click on the application name or the icon to edit. Then, go to the Access Management tab.

  4. Under Add New Custom Properties add the following properties to include specific permissions to be managed by the connector:

    Bash
    1
    ranger.policysync.connector.0.scheduled.load.foreign.resources=true

  5. Click SAVE to apply the changes.

  1. SSH to the instance where Privacera Manager is installed.

  2. Run the following command to open the .yml file to be edited.

    If you have multiple connectors, then replace instance1 with the appropriate connector instance name.

    Bash
    vi ~/privacera/privacera-manager/config/custom-vars/connectors/databricks-unity-catalog/instance1/vars.connector.databricks.unity.catalog.yml

  3. Uncomment (if commented) and set the following properties to include specific permissions to be managed by the connector:

    YAML
    1
    CONNECTOR_DATABRICKS_UNITY_CATALOG_SCHEDULED_LOAD_FOREIGN_RESOURCES: "true"

  4. Once the properties are configured, run the following commands to update your Privacera Manager platform instance:

    Step 1 - Setup which generates the helm charts. This step usually takes few minutes.

    Bash
    1
2
    cd ~/privacera/privacera-manager
./privacera-manager.sh setup
    Step 2 - Apply the Privacera Manager helm charts.
    Bash
    1
2
    cd ~/privacera/privacera-manager
./pm_with_helm.sh upgrade
    Step 3 - Post-installation step which generates Plugin tar ball, updates Route 53 DNS and so on.

    Bash
    1
2
    cd ~/privacera/privacera-manager
./privacera-manager.sh post-install

  1. In PrivaceraCloud portal, navigate to Settings → Applications.

  2. On the Connected Applications screen, select Databricks Unity Catalog.

  3. Click the icon or the Account Name to modify the settings.

  4. On the Edit Application screen, go to Access Management → ADVANCED tab.

  5. Under Add New Custom Properties add the following properties to include specific permissions to be managed by the connector:

    Bash
    1
    ranger.policysync.connector.0.scheduled.load.foreign.resources=true

  6. Click SAVE to apply the changes.