Steps to Skip the Access Check for the Databricks Cluster Libraries Path

When attaching a library to a Databricks cluster, the plugin performs an access check on the library's path. If the required permissions are missing, the library installation fails.

To skip this access check, follow the steps below:

  1. Go to the Databricks cluster where you want to attach the libraries.
  2. Click the Edit button in cluster.
  3. Under the Advanced options navigate to the Spark tab.

  4. In the Spark config section, add the following Spark property to skip the access check for the Databricks library path:

    Bash
    1
    spark.hadoop.privacera.fgac.file.ignore.path dbfs:/local_disk*

    • If this property is already configured with other paths, append dbfs:/local_disk* to the existing list using a comma separator. For example:
      Bash
      1
      spark.hadoop.privacera.fgac.file.ignore.path s3://existing-bucket-1,s3://existing-bucket-2,dbfs:/local_disk*

  5. Click the Confirm button to save the changes.

  6. Navigate to the Libraries tab to verify if libraries are already attached to the cluster.
  7. Click Install New to add a new library.
  8. Choose the appropriate Library Source and select the desired library.
  9. Click Install to install the library.
  10. Click Restart to restart the cluster
  11. With the Spark property in place, the plugin will skip access checks on the specified cluster library path.
  12. Once the cluster restarts, check the Libraries tab. The library status should display as Installed.

Comments