Skip to content

Enable Privacera Boto3 Signer Support for Apache Spark OLAC

This page covers building the Spark plugin Docker image with the Privacera Boto3 Signer.

Prerequisites

  • The Privacera Boto3 Signer supports only JWT authentication. JWT authentication must be enabled and configured in Privacera Dataserver. Refer Configuring JWT for Authentication.

To enable Privacera Boto3 Signer support in Apache Spark OLAC, set the OSS_BOTO3_SIGNER_ENABLE variable to true before running the build_privacera_plugin.sh script.

  1. Navigate to the privacera-oss-plugin plugin directory:

    Bash
    1
    cd ~/privacera-oss-plugin

  2. Open the penv.sh file and update the following properties:

    Bash
    1
2
3
4
5
    vi penv.sh

export OSS_BOTO3_SIGNER_ENABLE="true"
# Privacera release version used to resolve the Boto3 Signer package (e.g. 9.2.30.1)
export PRIVACERA_RELEASE_VERSION="<PRIVACERA_RELEASE_VERSION>"

  3. Create the privacera_boto3_signer_setup.sh script file in the scripts folder and copy the following content to it. This script downloads the Privacera Boto3 Signer package for the configured release (PRIVACERA_RELEASE_VERSION) and stages the wheel so it gets installed into the Spark plugin Docker image. The build_privacera_plugin.sh script invokes it automatically when OSS_BOTO3_SIGNER_ENABLE is true. 

    cd ~/privacera-oss-plugin/scripts
vi privacera_boto3_signer_setup.sh
chmod +x privacera_boto3_signer_setup.sh

    privacera_boto3_signer_setup.sh
    privacera_boto3_signer_setup.sh
      1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
    #!/bin/bash
# Privacera Boto3 Signer setup for the Privacera Spark plugin image.
# Invoked by build_privacera_plugin.sh when OSS_BOTO3_SIGNER_ENABLE="true" in penv.sh.
# Downloads the Privacera Boto3 Signer package (Self-Managed / PLATFORM only) and
# stages the wheel into the plugin tarball tree so the Dockerfile installs it
# into the Spark image.
set -e

readonly PRIVACERA_RELEASES_BASE_URL="https://privacera-releases.s3.us-east-1.amazonaws.com"
readonly PKG_NAME="privacera_boto3_signer.tar.gz"

SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
BASE_DIR="$( dirname "${SCRIPT_DIR}" )"

# Download working directory
BOTO3_DOWNLOAD_DIR="${BASE_DIR}/privacera-boto3"
# Staged under the tarball root so the wheel lands at /opt/privacera/boto3_signer in the image
BOTO3_STAGE_DIR="${BASE_DIR}/spark_home/opt/privacera/boto3_signer"

DEPLOYMENT_TYPE=$(echo "${PRIVACERA_DEPLOYMENT_TYPE:-PLATFORM}" | tr '[:lower:]' '[:upper:]')

main() {
    # Only Self-Managed (PLATFORM) is supported in this flow for now
    if [ "${DEPLOYMENT_TYPE}" != "PLATFORM" ]; then
        echo "Error: Unsupported deployment type '${DEPLOYMENT_TYPE}' for the Boto3 Signer build flow." >&2
        echo "       Supported value: PLATFORM (Self-Managed)." >&2
        exit 1
    fi

    if [ -z "${PRIVACERA_RELEASE_VERSION}" ] || [[ "${PRIVACERA_RELEASE_VERSION}" == \<* ]]; then
        echo "Error: PRIVACERA_RELEASE_VERSION is not set in penv.sh (e.g. 9.2.30.1)." >&2
        exit 1
    fi

    # start clean: wheels are version-named, so a wheel left over from a previous
    # release would otherwise survive and could be staged instead of the new one
    rm -rf "${BOTO3_DOWNLOAD_DIR}"
    mkdir -p "${BOTO3_DOWNLOAD_DIR}"
    cd "${BOTO3_DOWNLOAD_DIR}"

    echo "Fetching DataServer version for release ${PRIVACERA_RELEASE_VERSION}..."
    DATASERVER_SHA=$(get_privacera_dataserver_version "${PRIVACERA_RELEASE_VERSION}")

    if [ -z "${DATASERVER_SHA}" ]; then
        echo "Error: Failed to retrieve DATASERVER_VERSION for release ${PRIVACERA_RELEASE_VERSION}" >&2
        exit 1
    fi

    PRIVACERA_BOTO3_DOWNLOAD_URL="${PRIVACERA_RELEASES_BASE_URL}/privacera-dataserver/${DATASERVER_SHA}/${PKG_NAME}"

    echo "=========================================="
    echo "Downloading Privacera Boto3 Signer package"
    echo "Deployment Type : ${DEPLOYMENT_TYPE}"
    echo "Release Version : ${PRIVACERA_RELEASE_VERSION}"
    echo "Download URL    : ${PRIVACERA_BOTO3_DOWNLOAD_URL}"
    echo "=========================================="

    if command -v curl >/dev/null 2>&1; then
        curl -fL "${PRIVACERA_BOTO3_DOWNLOAD_URL}" -o "${PKG_NAME}"
    elif command -v wget >/dev/null 2>&1; then
        wget "${PRIVACERA_BOTO3_DOWNLOAD_URL}" -O "${PKG_NAME}"
    else
        echo "Error: curl or wget is required to download the package." >&2
        exit 1
    fi

    echo "Extracting package..."
    tar xfz "${PKG_NAME}"

    WHEEL_FILE=$(ls privacera_boto3_signer-*-py3-none-any.whl 2>/dev/null | head -1)
    if [ -z "${WHEEL_FILE}" ]; then
        echo "Error: No privacera_boto3_signer wheel found after extracting ${PKG_NAME}" >&2
        exit 1
    fi

    echo "Staging ${WHEEL_FILE} into the plugin image build tree..."
    rm -rf "${BOTO3_STAGE_DIR}"
    mkdir -p "${BOTO3_STAGE_DIR}"
    cp "${WHEEL_FILE}" "${BOTO3_STAGE_DIR}/"

    echo ""
    echo "Privacera Boto3 Signer wheel staged successfully at ${BOTO3_STAGE_DIR}"
}

get_privacera_dataserver_version() {
    local release_version=$1
    local url="${PRIVACERA_RELEASES_BASE_URL}/manifests/${release_version}/release-manifest.yaml"
    local content

    if command -v curl >/dev/null 2>&1; then
        content=$(curl -fs "$url")
    elif command -v wget >/dev/null 2>&1; then
        content=$(wget -q -O - "$url")
    else
        echo "Error: curl or wget is required to fetch the manifest." >&2
        return 1
    fi

    local dataserver_version
    dataserver_version=$(echo "$content" | grep 'DATASERVER_VERSION:' | awk '{print $2}')

    if [ -z "$dataserver_version" ]; then
        echo "Error: DATASERVER_VERSION not found in manifest." >&2
        return 1
    fi

    echo "$dataserver_version"
}

main "$@"

  4. After updating the configuration, follow the setup steps starting from the Generate Privacera Deployment File section in the Setup guide to rebuild the Spark plugin image and push it to the registry.

  5. Verify the signer is installed in the built image. The import must succeed:

    Bash
    1
    docker run --rm <image-name> python3 -c "import privacera_boto3_signer"