Skip to main content

Privacera Documentation

Migrate from Databricks Spark to Apache Spark

Privacera has deprecated the use of the native Databricks Spark. You should migrate from Databricks Spark to Apache Spark, which is built into Privacera Platform.


Be sure you have administrator access to your Databricks console.


  1. In your Privacera Platform system's shell, remove the properties file vars.discovery.databricks.yml from custom-vars:

    cd ~/privacera
    rm -f custom-vars/vars.discovery.databricks.yml
  2. In the Privacera Platform user interface, go to Settings > Data Source Registration > ADLS application.

  3. Edit the application properties to change the value of Use Databricks Spark from true to false.

  4. Apply the new configuration. In your Privacera Platform system's shell, run:

    cd ~/privacera
    ./privacera-manager update
  5. After the Privacera Discovery Kubernetes pod is up and running, stop and delete the Discovery job in Databricks:

    1. Login to your Databricks console.

    2. Click the WorkFlow tab.

    3. Search for your job.

    4. Stop the job.

    5. Delete the job.