Databricks SQL Fields

Parameter Field Name Allowable Values Default Description
Service* Supplied by the system DATABRICKSQL
Service name* String DatabricksSQLAnalytics Your freeform name for this service
Service enabled* true | false true Enable the feature
Service type* databricks_sql_analytics The only allowable service type
Service jdbc url* This property is used to set JDBC URL used for connecting to Databricks SQL endpoint. JDBC URL should follow below convention
jdbc:spark://<WORKSPACE_URL>:443/<DATABASE>;transportMode=http;ssl=1;AuthMech=3;httpPath=/sql/1.0/endpoints/1234567890

Example :- jdbc:spark://example.cloud.databricks.com:443/default;transportMode=http;ssl=1;AuthMech=3;httpPath=/sql/1.0/endpoints/1234567890
Service jdbc username* No default This property is used to set JDBC Username. This is the email used to log in to Databricks. Foe example, username@privacera.com)
Service jdbc password* No default This is a personal access token used to access the Databricks SQL endpoint. For example,dapi71f84xxxxxxxxxxx9e17a10ba8d53
Service database name* This property is used to set JDBC database to make initial connection to Databricks SQL endpoint.

You can set the value as default.
Ownership role* This value is the same as Service jdbc username.

The owner for all new resources created in Databricks. This ensures admins know exactly the owner of all new resources.

It sets the ownership for all the resources managed by PolicySync. The specified user will become owner for all managed resources and will have full control on those resources. Changing owners of database, tables and views are supported.

Note: If owner role is kept as blank, then ownership will not change and users who creates tables/views or any other object will be the owner of those objects and PolicySync won't be able to do access control on that object.
Service base url* This property is used to make a call to SQL analytics API for users/groups/audits. Base URL of Databricks, for example https://db1.cloud.databricks.com/
System config* privacera-databricks_sql_analytics-system-config.json You can also set the value to privacera-databricks_sql_analytics-hive-system-config.json, when you choose the Hive service.
Load resources load_like This property controls which method to be used to load resources from Databricks SQL Endpoint. Keep the default unchanged, Databricks only supports the value load_like as a method for loading resources.
Sync interval in seconds Number 60 This property is used to set the interval in seconds for resource sync process. Resource sync is the process where we load the resources from the Databricks SQL endpoint to check whether any new resource has been created or there is any change in existing resource. This process happens in defined interval time.
Sync service user in seconds Number 420 This property is used to set the interval in seconds for principal sync process. Principal sync is the process where we load the user/group/roles from the Databricks SQL endpoint to check which all users/groups/roles are created and membership between them. We validate these with what is defined in PCloud. This process happens in defined interval time.
Sync service policy in seconds Number 540 This property is used to set the interval in seconds for existing policies sync process. Existing policies sync is the process where we load the permissions or policies which are already synced to Databricks SQL endpoint, and then validate it with Ranger policies. If differences are found, then required grants/revokes get triggered. This process happens in defined interval time.
Audit interval in seconds Number 30 This property is used to set the interval in seconds for the access audits process. Access audits process is the process where access audits are retrieved from Databricks and then pushed to Solr. By storing it in Solr, they can displayed in PCloud Access Audit page. This process happens in defined interval time.
Manage database list See description. none Comma-separated list of names of databases you want to manage.
Format: databasename,databasename,databasename,...
If specified as none, no databases are managed.
Manage table list This property is used to set comma-separated table/view FQDN on which access control should be managed by PolicySync. If you want to manage all tables/views, then keep the value blank. This supports wildcards. The ignore table list has precedence over manage table list.

For example, testdb1.table1,testdb2.view2,sales_db*.sales*.*
Ignore database list This property is used to set comma-separated database names on which you don't want access control to be managed by PolicySync. If you don't want to ignore any database, then keep the value blank. This supports wildcards. This has precedence over manage database list.

EFor example, testdb1,testdb2,sales_db*
Ignore table list This property is used to set comma-separated table/view FQDN on which you don't want access control to be managed by PolicySync. If you don't want to ignore any tables/views, then keep the value blank. This supports wildcards. This has precedence over manage table list.

For example, testdb1.schema1.table1,testdb2.schema2.view2,sales_db*.sales*.*
Name replace from regex [~`$&+:;=?@#|'<>.^*()_%\\\\[\\\\]!\\\\-\\\\/\\\\\\\\{}] While creating the user/group/role name in Databricks, it the regular expression as input and finds the matching characters in a user name and replaces them with the characters specified in Replace to string property.

If kept blank, no find and replace operation is performed.
Replace to string _ The value specified in this property is used to replace the characters found by the regex specified in Name replace from regex property.

If kept blank, no find and replace operation is performed.
User name replace from regex [~`$&+:;=?@#|'<>.^*()_%\\\\[\\\\]!\\\\-\\\\/\\\\\\\\{}] While creating the user name in Databricks, it takes the regular expression as input and finds the matching characters in a user name and replaces them with the characters specified in User name replace to string property.

If kept blank, no find and replace operation is performed.
User name replace to string _ The value specified in this property is used to replace the characters found by the regex specified in User Name replace from regex property.

If kept blank, no find and replace operation is performed.
Group name replace from regex [~`$&+:;=?@#|'<>.^*()_%\\\\[\\\\]!\\\\-\\\\/\\\\\\\\{}] While creating the group name in Databricks, it takes the regular expression as input and finds the matching characters in a group name and replaces them with the characters specified in Group name replace to string property.

If kept blank, no find and replace operation is performed.
Group name replace to string _ The value specified in this property is used to replace the characters found by the regex specified Group name replace from regex property.

If kept blank, no find and replace operation is performed.
Role name replace from regex [~`$&+:;=?@#|'<>.^*()_%\\\\[\\\\]!\\\\-\\\\/\\\\\\\\{}] While creating the role name in Databricks, it takes the regular expression as input and finds the matching characters in a role name and replaces them with the characters specified in Role name replace to string property.

If kept blank, no find and replace operation is performed.
Role name replace to string _ The value specified in this property is used to replace the characters found by the regex specified Role name replace from regex property.

If kept blank, no find and replace operation is performed.
Persist user name case sensitivity true | false false After loading the users using the Ranger API, they are converted into lowercase, but in some cases, you might prefer keeping the case consistent as they are in Ranger.

When setting this value to true, it will maintain the case sensitivity of names as they are in Ranger.
Group name persist case sensitivity true | false false After loading the groups using the Ranger API, they are converted into lowercase, but in some cases, you might prefer keeping the case consistent as they are in Ranger.

When setting this value to true, it will maintain the case sensitivity of names as they are in Ranger.
Role name persist case sensitivity true | false false After loading the roles using the Ranger API, they are converted into lowercase, but in some cases, you might prefer keeping the case consistent as they are in Ranger.

When setting this value to true, it will maintain the case sensitivity of names as they are in Ranger.
Create service user true | false true This property controls whether we should create the user in Databricks SQL endpoint for users retrieved from Ranger.
Manage service user* true | false false This property controls whether we should create the user in Databricks SQL endpoint for users retrieved from Ranger. It is recommended to set the value to true, if you want to grant/revoke permissions for the user. This will also send an email to users configured in Manage user list, make sure you limit the users if required.
Manage service group* true | false false This property controls whether we should create the group in Databricks SQL endpoint for groups retrieved from Ranger. It is recommended to set the value to true, if you want to grant/revoke permissions for the group.
Manage service role* true | false false This property controls whether we should create the role in Databricks SQL endpoint for roles retrieved from Ranger. It is recommended to set the value to true, if you want to grant/revoke permissions for the role.
Manage user list See description. No default This property is used to set comma-separated user names on which access control will be managed by PolicySync. If you want to manage all users, then keep the value empty. This supports wildcards. The ignore users list has precedence over manage users list.

Eg. user1,user2,dev_user*
Manage group list See description. No default This property is used to set comma-separated groups on which access control will be managed by PolicySync. If you want to manage all groups, then keep the value empty. This supports wildcards. The ignore groups list has precedence over manage groups list.

Eg. group1,group2,dev_group*
Manage role list See description. No default This property is used to set comma-separated roles on which access control will be managed by PolicySync. If you want to manage all roles, then keep the value empty. This supports wildcards. The ignore roles list has precedence over manage roles list.

Eg. role1,role2,dev_role*
Ignore user list User names No default This property is used to set comma-separated user names on which access control you don't want access control to be managed by PolicySync. If you don't want to ignore any users, then keep the value blank. This supports wildcards. This has precedence over manage users list.

Eg. user1,user2,dev_user*
Ignore group list Group names No default This property is used to set comma-separated group names on which you don't want access control to be managed by PolicySync. If you don't want to ignore any groups, then keep the value blank. This supports wildcards. This has precedence over manage groups list.

Eg. group1,group2,dev_group*
Ignore role list Role names No default This property is used to set comma-separated role names on which you don't want access control to be managed by PolicySync. If you don't want to ignore any roles, then keep the value blank. This supports wildcards. This has precedence over manage roles list.

Eg. role1,role2,dev_role*
Group role prefix priv_group_ This property is used to set a prefix for role that will be created in Databricks SQL endpoint for groups retrieved from Ranger. For example, if you have group named dev in Ranger and you have defined prefix as test_group_, then the role in Databricks SQL endpoint will be test_group_dev.

Note: This property does not exist for Ranger users because user's email is used as the username to log in.
Role role prefix priv_role_ This property is used to set a prefix for role that will be created in Databricks SQL endpoint for roles retrieved from Ranger. For example, if you have role named finance in Ranger and you have defined prefix as test_role_, then the role in Databricks SQL endpoint will be test_role_finance.

Note: This property does not exist for Ranger users because user's email is used as the username to log in.
Use native public group true | false true Set this property to true, if you want PolicySync to use the "public" group in Databricks for access grants whenever a policy is created with public group.
Manage user filterby group true | false false Set this property to true, if you want to manage only the users belonging to the groups defined in Manage groups list property.
Manage user filterby role true | false false Set this property to true, if you want to manage only the users belonging to the roles defined in Manage roles list property.
Enable view based masking true | false true Set this property to true, if you want to enable secure view-based masking in Databricks PolicySync.

Note: Databricks does not support native masking, so it is recommended to use view-based masking.
Enable view based row filter true | false true Set this property to true, if you want to enable secure view-based row filter in Databricks PolicySync.

Note: Databricks does not support native row filters, so it is recommended to use view-based row filters.
Secure view create for all true | false true Set this property to true, if you want to create secure view for all tables and all views created by users. It creates secure view for the resources although there is a masking or row filter policy in Ranger.
Default Mask number value 0 This property is used to specify the default masking value for numeric columns.
Default Mask text value '<MASKED>' This property is used to specify the default masking value for text/string columns.
Secure View Name Prefix String No default By default view-based row filters and masking related secure views have the same name as the table. If you want to change the secure view name prefix, set this field to the required string. After the prefix is specified, the view name is in this format: {prefix}{table_name}.
Secure View Name Postfix String No default By default view-based row filter- and masking-related secure views have the same name as the table. If you want to change the secure view name postfix, set this field to the required string. After the postfix is specified, the view name is in this format: {table_name}{postfix}.
Secure View Database Name Prefix String No default By default view-based row filters and masking related secure views have the same database name as the table database name. If you want to change the secure view database name prefix, set this field to the required string. After the prefix is specified, the view database name is in this format: {prefix}{view_database_name}.
Secure View Database Name Postfix String _secure By default view-based row filters and masking related secure views have the same database name as the table database name. If you want to change the secure view database name prefix, set this field to the required string. After the postfix is specified, the view database name is in this format: {view_database_name}{postfix}.
Secure view name suffix to be removed You can remove unwanted suffix attached at the end of a table/view name. For example, if the table name is some_name_table, you can remove the suffix, _table. Your secure name will be {prefix}some_name{postfix}. Enter a suffix string or a comma-separated list of suffix strings.
Secure view database name suffix to be removed You can remove unwanted suffix attached at the end of a schema name. For example, if the schema is some_name_schema, you can remove the suffix, _schema. Your secure schema name will be {schema_prefix}some_schema{schema_postfix}. Enter a suffix string or a comma-separated list of suffix strings.
Perform grant updates* true | false false This property controls whether the queries (grant/revoke/create/update/delete) for user/group/role should be run on Databricks SQL endpoint. It is recommended to set the value to true.
Enable data admin true | false true This property is used to enable the data admin feature. With it, you can create all the policies on table/native view and by default respective grants will be made on secure view of table or native view. The secure view will have row filter and masking capability. If you need permissions on a table, then you can select the permission you want and data admin in the policy. The permissions will be granted on both, the table/native view and its secure view.
Use email as service name true | false true This property is used to map the username as email address.
Enable access audit* true | false true Enables Access Audit coming from Databricks SQL. This fetches all the access permission granted by the database to its entities.
Audit excluded users This property is used to exclude the users while pushing the audits logs to Ranger access audits. It is recommended to set this as JDBC user name as there will be audits from PolicySync application.
Audit init starttime offset minutes 30 When the audits are enabled for the first time, the value will decide the number of minutes taken to pull the access audits from the database.

Last update: January 13, 2022