Skip to content

OAuth Authentication

The default authentication mechanism in the Databricks SQL connector is token-based, but it also provides the option to enable OAuth. To generate OAuth credentials for the Databricks SQL, follow the instructions in the Databricks documentation.

Prerequisites

Given below is the instructions to generate OAuth credentials for the Databricks SQL. The instructions might change, so it is recommended to refer to the latest Databricks documentation.

It is important to note that the service principal should have the necessary permissions to manage the policies in the Databricks SQL.

  1. Generate OAuth token: Refer to the Databricks documentation on how to use a service principal to authenticate with Databricks.
  2. Note down the Secret and Client ID during the OAuth Secret generation.

  3. To ensure the generated service principal has the necessary permissions, follow these steps:

    1. Add the service principal to the Admin Group in Databricks.
    2. Login into Databricks SQL Workspace URL
    3. Go to settings "Identity and access" → Management and permissions → Groups → Click on Manage.
    4. Locate the Admin Group and add your service principal.
    5. Grant the service principal admin privileges to manage resources.
    6. Ensure the service principal has access to all required databases and tables.

  4. Ensure that the service principal you create is added to the Admin Group Role in Databricks and has admin access.

  5. It must have access to all required databases and tables.

Setup

Replace <server-hostname>, <http-path>, <CLIENT_ID> and <CLIENT_SECRET> with the actual values.

  1. Navigate to Settings → Applications in the Self-Managed Portal.

  2. From the list of Connected Applications, select Databricks SQL.

  3. Click on the application name or the icon to edit. Then, go to the Access Management tab.

  4. Under the BASIC tab

    • Update the value for JDBC Authentication Mechanism as oauth.

    • Update the value for Databricks JDBC url with the following format:

      Bash
      jdbc:databricks://<server-hostname>:443/default;transportMode=http;ssl=1;AuthMech=11;Auth_Flow=1;httpPath=<http-path>;

    • Make sure the value for Databricks personal access token is empty.

    • Add the values for OAuth Client ID and OAuth Client Secret.

  5. Click SAVE to apply the changes.

To enable OAuth authentication, update the following properties in Privacera Manager:

  1. SSH to the instance where Privacera Manager is installed.

  2. Run the following command to open the .yml file to be edited.

    If you have multiple connectors, then replace instance1 with the appropriate connector instance name.

    Bash
    1
    vi ~/privacera/privacera-manager/config/custom-vars/connectors/databricks-sql-analytics/instance1/vars.connector.databricks.sql.analytics.yml

  3. Set the following properties to enable OAuth authentication for the Databricks SQL Connector:

    YAML
    1
2
3
4
    CONNECTOR_DATABRICKS_SQL_ANALYTICS_JDBC_URL: "jdbc:databricks://<server-hostname>:443/default;transportMode=http;ssl=1;AuthMech=11;Auth_Flow=1;httpPath=<http-path>;"
CONNECTOR_DATABRICKS_SQL_ANALYTICS_AUTH_MECHANISM: "oauth"
CONNECTOR_DATABRICKS_SQL_ANALYTICS_OAUTH_CLIENT_ID: "<CLIENT_ID>"
CONNECTOR_DATABRICKS_SQL_ANALYTICS_OAUTH_CLIENT_SECRET: "<CLIENT_SECRET>"

  4. Once the properties are configured, run the following commands to update your Privacera Manager platform instance:

    Step 1 - Setup which generates the helm charts. This step usually takes few minutes.

    Bash
    1
2
    cd ~/privacera/privacera-manager
./privacera-manager.sh setup
    Step 2 - Apply the Privacera Manager helm charts.
    Bash
    1
2
    cd ~/privacera/privacera-manager
./pm_with_helm.sh upgrade
    Step 3 - (Optional) Post-installation step which generates Plugin tar ball, updates Route 53 DNS and so on. This step is not required if you are updating only connector properties.

    Bash
    1
2
    cd ~/privacera/privacera-manager
./privacera-manager.sh post-install

  1. In PrivaceraCloud portal, navigate to Settings -> Applications.

  2. On the Connected Applications screen, select Databricks SQL.

  3. Click the pen icon or the Account Name to modify the settings.

  4. On the Edit Application screen, go to Access Management.

  5. Under the BASIC tab

    • Update the value for JDBC Authentication Mechanism as oauth.

    • Update the value for Databricks JDBC url with the following format:

      Bash
      jdbc:databricks://<server-hostname>:443/default;transportMode=http;ssl=1;AuthMech=11;Auth_Flow=1;httpPath=<http-path>;

    • Make sure the value for Databricks personal access token is empty.

    • Add the values for OAuth Client ID and OAuth Client Secret.

  6. Click SAVE to apply the changes.