Skip to content

Switching from API to JDBC

The Unity Catalog Connector supports the use of JDBC (Java Database Connectivity) for all data transactions. To enable JDBC, configure the following properties in the connector:

Setup

To enable JDBC, update the following properties in Privacera Manager:

  1. SSH to the instance where Privacera Manager is installed.

  2. Run the following command to open the .yml file to be edited.

    If you have multiple connectors, then replace instance1 with the appropriate connector instance name.

    Bash
    1
    vi ~/privacera/privacera-manager/config/custom-vars/connectors/databricks-unity-catalog/instance1/vars.connector.databricks.unity.catalog.yml

  3. Set the following properties to enable the connector to use JDBC for all data transactions:

    YAML
    1
2
    CONNECTOR_DATABRICKS_UNITY_CATALOG_USE_DATABASE_CONNECTION_API: "false"
CONNECTOR_DATABRICKS_UNITY_CATALOG_JDBC_URL: "<JDBC_URL>"

  4. Once the properties are configured, run the following commands to update your Privacera Manager platform instance:

    Step 1 - Setup which generates the helm charts. This step usually takes few minutes.

    Bash
    1
2
    cd ~/privacera/privacera-manager
./privacera-manager.sh setup
    Step 2 - Apply the Privacera Manager helm charts.
    Bash
    1
2
    cd ~/privacera/privacera-manager
./pm_with_helm.sh upgrade
    Step 3 - Post-installation step which generates Plugin tar ball, updates Route 53 DNS and so on.

    Bash
    1
2
    cd ~/privacera/privacera-manager
./privacera-manager.sh post-install

  1. In PrivaceraCloud portal, navigate to Settings -> Applications.

  2. On the Connected Applications screen, select Databricks Unity Catalog.

  3. Click the pen icon or the Account Name to modify the settings.

  4. On the Edit Application screen, go to Access Management.

  5. Under BASIC tab, enter the JDBC URL in the Databricks JDBC url field.

  6. Under ADVANCED tab, add the following property under Add New Custom Properties:

    Bash
    1
    ranger.policysync.connector.0.use.database.connection.api=false

  7. Click SAVE to apply the changes.

Benefits

  • Increased Efficiency and Performance: The move to JDBC from API-based connections marks a significant improvement in the performance of data transactions, ensuring faster and more reliable operations.
  • No Call Limits: Unlike API calls, which are subject to throttling and rate limits, JDBC connections do not have a limit on the number of calls per second, enabling uninterrupted data transactions.
  • Reduced Throttling and Failure Issues: By shifting to JDBC, the issues caused by Databricks throttling API calls, including per-second limits (link) that can cause failures during permission application and data loading, are mitigated. This leads to smoother data interactions.

Downside

  • Increased Cost:
    • Transitioning to JDBC may incur additional costs associated with maintaining and managing a data warehouse. Users should be aware of these potential costs and plan accordingly.
    • For detailed guidance on managing data warehouse costs, please refer to Databricks Pricing.

Comments