Skip to content

Enable Distributor Plugin Support for Databricks OLAC

The Distributor is a utility that is used to distribute JWT and Privacera Tokens from Spark Driver to the Executors. It is only supported for Databricks OLAC Deployment using JWT.

Prerequisites

Make sure JWT Auth is configured. Refer JWT Auth Configuration.

There are two ways to enable distributor plugin in DBX:

  • Spark Config
  • Cluster Policy

Spark Config

  1. Log in to the Databricks Web UI.
  2. Click on the Compute icon on the sidebar.
  3. To enable the Distributor Plugin Support, the property needs to be included in the spark configuration of the cluster.
  4. Once on the cluster configuration page, navigate to the Advanced Options section.
  5. Click on the section named Spark. In the text box named Spark Config include the below property:
    Bash
    1
    spark.plugins com.privacera.spark.plugin.PrivaceraSparkPlugin
  6. Start the cluster once above mentioned changes are done.

Cluster Policy

  1. Log in to Databricks Web UI.
  2. Click on the Compute icon on the sidebar.
  3. Click on the Policies tab.
  4. Edit the Policy that is attached to the cluster and, include the following JSON configuration:
    JSON
    1
2
3
4
      "spark_conf.spark.plugins": {
    "type": "fixed",
    "value": "com.privacera.spark.plugin.PrivaceraSparkPlugin"
  }
  5. Save the Cluster Policy once above changes are done.
  6. Start the cluster with the policy attached to start using the Distributor Plugin.