Skip to content

Preventing DBX SCIM API throttling

The Databricks SQL connector uses the Databricks SCIM API to manage users, groups, and roles within Databricks SQL. The Databricks SCIM API enforces rate limits that, if exceeded, can cause the connector to be throttled. To prevent throttling, you can configure the connector to automatically retry DBX SCIM API requests.

This page also covers REST API HTTP connection and socket timeouts, which help prevent stuck audit-loader runs caused by slow or unresponsive Databricks API calls.

Configuration Properties

SCIM API Retry

The following properties configure SCIM API retry behavior and related throttling settings:

Property Description Default Value
Maximum Retry Attempts Number of retry attempts for a failed API request due to rate limiting. 31
Retry Interval Wait time between retry attempts (in seconds). 20
Group Update Delay Delay interval between consecutive patch API calls to update group users (in milliseconds). 6000
Group ID Cache Timeout Cache timeout for group IDs in minutes. 15
Min Retry Delay Minimum retry delay in seconds for API limit exceeded scenarios. 1
Max Retry Delay Maximum retry delay in seconds for API limit exceeded scenarios. 10

REST API HTTP Connection Timeouts

The following properties configure HTTP timeouts for the Databricks REST API client. They apply to both token-based and OAuth authentication.

Property Description Default Value
API Connection Timeout Maximum time in milliseconds to establish a connection to the Databricks REST API. 60000 ms (60 s)
API Socket Timeout Maximum time in milliseconds to wait for data on an established socket after connection is made. 300000 ms (5 min)

Setup

Warning

Replace the example values shown below with values appropriate for your environment.

  1. SSH to the instance where Privacera Manager is installed.

  2. Run the following command to open the .yml file to be edited:

    If you have multiple connectors, then replace instance1 with the appropriate connector instance name.

    Bash
    1
    vi ~/privacera/privacera-manager/config/custom-vars/connectors/databricks-sql-analytics/instance1/vars.connector.databricks.sql.analytics.yml

  3. Add or update the following properties to enable the connector to handle retrying of DBX SCIM API requests:

    YAML
    1
2
3
4
5
6
    CONNECTOR_DATABRICKS_SQL_ANALYTICS_API_LIMIT_EXCEEDED_MAX_RETRY_ATTEMPTS: "35"
CONNECTOR_DATABRICKS_SQL_ANALYTICS_TIME_INTERVAL_BETWEEN_APPLY_UPDATE_RETRY_ATTEMPT_IN_SEC: "30"
CONNECTOR_DATABRICKS_SQL_ANALYTICS_API_RETRY_DELAY_MIN_SECONDS: "5"
CONNECTOR_DATABRICKS_SQL_ANALYTICS_API_RETRY_DELAY_MAX_SECONDS: "15"
CONNECTOR_DATABRICKS_SQL_ANALYTICS_GROUP_ID_CACHE_TIMEOUT_MINUTES: "20"
CONNECTOR_DATABRICKS_SQL_ANALYTICS_API_CONSECUTIVE_GROUP_USER_UPDATE_DELAY_INTERVAL_MS: "6000"

To configure REST API HTTP connection timeouts, also add:

YAML
1
2
CONNECTOR_DATABRICKS_SQL_ANALYTICS_API_CONNECTION_TIMEOUT_MS: "<PLEASE_CHANGE>"
CONNECTOR_DATABRICKS_SQL_ANALYTICS_API_SOCKET_TIMEOUT_MS: "<PLEASE_CHANGE>"

  1. Once the properties are configured, run the following commands to update your Privacera Manager platform instance:

    Step 1 - Setup which generates the helm charts. This step usually takes few minutes.

    Bash
    1
2
    cd ~/privacera/privacera-manager
./privacera-manager.sh setup
    Step 2 - Apply the Privacera Manager helm charts.
    Bash
    1
2
    cd ~/privacera/privacera-manager
./pm_with_helm.sh upgrade
    Step 3 - (Optional) Post-installation step which generates Plugin tar ball, updates Route 53 DNS and so on. This step is not required if you are updating only connector properties.

    Bash
    1
2
    cd ~/privacera/privacera-manager
./privacera-manager.sh post-install

  1. In PrivaceraCloud portal, navigate to Settings → Applications.

  2. On the Connected Applications screen, select Databricks SQL.

  3. Click on the icon or the Account Name to modify the settings.

  4. On the Edit Application screen, go to Access Management → ADVANCED tab.

  5. Under Add New Custom Properties, add the following properties:

    Bash
    1
2
3
4
5
6
    ranger.policysync.connector.0.dbx.api.limit.exceeded.max.retry.attempts=35
ranger.policysync.connector.0.time.interval.between.apply.update.retry.attempt.in.seconds=30
ranger.policysync.connector.0.dbx.retry.delay.min.seconds=5
ranger.policysync.connector.0.dbx.retry.delay.max.seconds=15
ranger.policysync.connector.0.dbx.group.id.cache.timeout.minutes=20
ranger.policysync.connector.0.dbx.api.consecutive.group.user.update.delay.interval.ms=6000

To configure REST API HTTP connection timeouts, also add:

Properties
1
2
ranger.policysync.connector.0.databricks.api.connection.timeout.ms=<PLEASE_CHANGE>
ranger.policysync.connector.0.databricks.api.socket.timeout.ms=<PLEASE_CHANGE>

  1. Click SAVE to apply the changes.