Skip to content

Using S3 Bucket for Init Scripts in Databricks Clusters with FGAC

This is applicable only Self Managed and Data Plane deployment of Privacera

This section describes how to upload the privacera_custom_conf.zip and initialization script files, specifically ranger_enable.sh and ranger_enable_scala.sh, to the S3 location in the Databricks environment.

Pre-requisites

  1. The EC2 instance where Privacera Manager is installed should have the permission to copy init scripts to the S3 bucket.

Setup

  1. SSH into the instance where Privacera Manager is installed.
  2. Go to the config directory.
    Bash
    1
    cd ~/privacera/privacera-manager/config
  3. Run the following command to open the vars.databricks.plugin.yml file to be edited.
    Bash
    1
    vi custom-vars/vars.databricks.plugin.yml
  4. To upload ranger_enable.sh, ranger_enable_scala.sh and privacera_custom_conf.zip files to the S3 location, update the following properties in the vars.databricks.plugin.yml file:
    Bash
    1
2
3
4
5
6
7
8
9
    # To upload files to Databricks Workspace. Default: `true`.
DATABRICKS_INIT_SCRIPT_WORKSPACE_FLAG_ENABLE: "false"

# To upload the `init scripts` and `privacera_cust_conf.zip` to the S3 location.
DATABRICKS_CUST_CONF_PATH: "s3://<bucket-name>/<path>"

# To upload the plugin installation logs to the S3 location. By default, the logs will be uploaded to the path 
configured in `DATABRICKS_CUST_CONF_PATH`.
PRIVACERA_CLUSTER_LOGS_DIR: "s3://<bucket-name>/<path>"
  5. Once the properties are configured, run the Privacera Manager to update the configurations and upload to S3
    Bash
    1
2
3
    cd ~/privacera/privacera-manager
./privacera-manager.sh setup
./pm_with_helm.sh upgrade

Run the following command to run the post install steps:

Bash
1
2
cd ~/privacera/privacera-manager
./privacera-manager.sh post-install

Comments