Databricks Unity Catalog Resource Loader Configuration¶
This section outlines the configuration settings for resource loading behavior and threading when using the Databricks Unity Catalog Connector. These settings help improve performance by enabling parallel loading of Unity Catalog entities such as schemas, tables, columns, functions, and volumes.
Connector Type
These configurations are applicable only when the connector is configured using JDBC for Databricks Unity Catalog.
Configuration Parameters¶
You can configure the following:
- Optional toggle to enable column loading
- Thread counts for each supported resource type
If these parameters are not specified, default values may apply, depending on the system and connector version.
Setup¶
Warning
- All configuration values must be entered as strings.
- Thread-related parameters should be tuned based on the volume of metadata and available system resources.
-
SSH into the instance where Privacera Manager is installed.
-
Open the Unity Catalog connector configuration file:
Note
Replace
instance1
with your actual connector instance name.Bash -
Configure the resource loader parameters as needed:
Usage Guide
- CONNECTOR_DATABRICKS_UNITY_CATALOG_LOAD_COLUMNS: Enables loading of column metadata. Defaults to
"true"
if not set. - Thread properties: Controls the number of threads used to load each resource type. Increase values for faster loading at the cost of higher CPU/memory usage.
- These properties only take effect if the connector is set up using JDBC.
- CONNECTOR_DATABRICKS_UNITY_CATALOG_LOAD_COLUMNS: Enables loading of column metadata. Defaults to
-
After updating the configuration, apply the changes by running:
Step 1 - Setup which generates the helm charts. This step usually takes few minutes.
Step 2 - Apply the Privacera Manager helm charts. Step 3 - Post-installation step which generates Plugin tar ball, updates Route 53 DNS and so on.
- Prev topic: Advanced Configuration