Preventing DBX SCIM API throttling¶
The Databricks SQL connector uses the Databricks SCIM API to manage users, groups, and roles within Databricks SQL. The Databricks SCIM API enforces rate limits that, if exceeded, can cause the connector to be throttled. To prevent throttling, you can configure the connector to automatically retry DBX SCIM API requests.
Configuration Properties¶
The following properties configure SCIM API retry behavior and related throttling settings:
Property | Description | Default Value |
---|---|---|
Maximum Retry Attempts | Number of retry attempts for a failed API request due to rate limiting. | 31 |
Retry Interval | Wait time between retry attempts (in seconds). | 20 |
Group Update Delay | Delay interval between consecutive patch API calls to update group users (in milliseconds). | 6000 |
Group ID Cache Timeout | Cache timeout for group IDs in minutes. | 15 |
Min Retry Delay | Minimum retry delay in seconds for API limit exceeded scenarios. | 1 |
Max Retry Delay | Maximum retry delay in seconds for API limit exceeded scenarios. | 10 |
Setup¶
Warning
Replace the example values shown below with values appropriate for your environment.
-
SSH to the instance where Privacera Manager is installed.
-
Run the following command to open the
.yml
file to be edited:If you have multiple connectors, then replace
instance1
with the appropriate connector instance name.Bash -
Add or update the following properties to enable the connector to handle retrying of DBX SCIM API requests:
-
Once the properties are configured, run the following commands to update your Privacera Manager platform instance:
Step 1 - Setup which generates the helm charts. This step usually takes few minutes.
Step 2 - Apply the Privacera Manager helm charts. Step 3 - Post-installation step which generates Plugin tar ball, updates Route 53 DNS and so on.
-
In PrivaceraCloud portal, navigate to Settings → Applications.
-
On the Connected Applications screen, select Databricks SQL.
-
Click on the icon or the Account Name to modify the settings.
-
On the Edit Application screen, go to Access Management → ADVANCED tab.
-
Under Add New Custom Properties, add the following properties:
-
Click SAVE to apply the changes.
- Prev topic: Advanced Configuration