Skip to content

Enable Iceberg in Apache Spark OLAC

To enable Iceberg support in Apache Spark OLAC, set the OSS_ICEBERG_ENABLE variable to true before running the build_privacera_plugin.sh script.

  1. Navigate to the privacera-oss-plugin plugin directory:

    Bash
    1
    cd ~/privacera-oss-plugin

  2. Open the penv.sh file and update the following properties:

    Bash
    1
2
3
4
5
6
7
    vi penv.sh

export OSS_ICEBERG_ENABLE=true
# Set Iceberg warehouse path
export ICEBERG_WAREHOUSE_PATH="s3a://your-bucket/iceberg-warehouse"
# Set Iceberg catalog type (optional, default: hadoop)
#export ICEBERG_CATALOG_TYPE="hadoop"

  3. After updating the configuration, follow the setup steps starting from the Generate Privacera Deployment File section in the Setup guide to rebuild the image, push it to the registry, and redeploy the Spark environment.

  4. For information on how to use and validate Iceberg in Apache Spark, see the Using Iceberg with Apache Spark OLAC guide.