Skip to main content

Privacera Documentation

Scala Plugin properties on Privacera Platform

Property

Description

Example

DATABRICKS_VERSION

Set this property to select which version of Spark Config you want to be used.

From release 5.0 onwards, two versions of Spark Config can be used: V1 or V2. It uses V1 by default, and V2 is for preview purposes only.

If your Databricks version is >= 7.6, use V2 for the Spark Config. For lower versions, use V1.

V2

DATABRICKS_SCALA_ENABLE

Set the property to enable/disable Databricks Scala. This is found under Databricks Signed URL Configuration For Scala Clusters section.

DATASERVER_DATABRICKS_ALLOWED_URLS

Add a URL or comma-separated URLs.

Privacera Dataserver serves only those URLs mentioned in this property.

https://xxx-7xxxfaxx-xxxx.cloud.databricks.com

DATASERVER_AWS_STS_ROLE

Add the instance profile ARN of the AWS role, which can access Delta Files in Databricks.

arn:aws:iam::111111111111:role/assume-role

DATABRICKS_MANAGE_INIT_SCRIPT

Set the init script.

If enabled, Privacera Manager will upload Init script ('ranger_enable.sh') to the identified Databricks Host.

If disabled, Privacera Manager will take no action regarding the Init script for the Databricks File System.

DATABRICKS_HOST_URL

Enter the URL where the Databricks environment is hosted.

For AZURE Databricks,

DATABRICKS_HOST_URL: "https://xdx-66506xxxxxxxx.2.azuredatabricks.net/?o=665066931xxxxxxx"

For AWS Databricks

DATABRICKS_HOST_URL: "https://xxx-7xxxfaxx-xxxx.cloud.databricks.com"

DATABRICKS_TOKEN

Enter the token.

To generate the token:

  1. Log in to your Databricks account.

  2. Click the user profile icon in the upper right corner of your Databricks workspace.

  3. Click User Settings.

  4. Click Generate New Token.

  5. (Optional) Enter a description (comment) and expiration period.

  6. Click Generate.

  7. Copy the generated token.

xapid40xxxf65xxxxxxe1470eayyyyycdc06

DATABRICKS_SCALA_CLUSTER_POLICY_SPARK_CONF

Configure Databricks Cluster policy.

Add the following JSON in the text area:

[{"Note":"First spark conf","key":"spark.hadoop.first.spark.test","value":"test1"},{"Note":"Second spark conf","key":"spark.hadoop.first.spark.test","value":"test2"}]