Encryption - Databricks Unity Catalog¶

This guide provides instructions for setting up encryption support for Databricks Unity Catalog using Privacera Encryption Gateway (PEG).

Overview¶

With Privacera Encryption Gateway (PEG), you can encrypt sensitive data in Databricks Unity Catalog to enhance data security and meet regulatory compliance requirements.

Deployment options¶

Privacera offers three deployment types. The following sections describe each option and how protect, unprotect, and mask operations flow between your environment and Privacera.

Privacera Cloud¶

All encryption components run in Privacera's cloud. Privacera manages PEG, Scheme Server, metadata DB, and key management (AWS Secrets Manager). You run the client application (e.g., Databricks), which connects to Privacera for protect/unprotect operations.

sequenceDiagram
    box Privacera Cloud Environment
        participant APP as Client Application (DBX)
        participant PEG as PEG Server
        participant SS as Scheme Server
        participant DB as Metadata DB
        participant KMS as Key Management Service (AWS Secrets Manager)
    end

    APP->>PEG: Protect / Unprotect (JWT)
    PEG->>SS: Get Scheme

    SS->>DB: Fetch Scheme Metadata
    DB-->>SS: Scheme + Encrypted Data Key
    Note right of SS: Data key encrypted with Master Key

    SS->>KMS: Get Master Key
    KMS-->>SS: Master Key
    SS->>SS: Decrypt Data Key (in-memory only)

    SS-->>PEG: Scheme Metadata (no plaintext key)

    PEG->>PEG: Encrypt / Decrypt (runtime only)
    PEG-->>APP: Encrypted / Decrypted Data

D2P (Privacera Cloud Data-plane)¶

D2P is Privacera Cloud Data-plane. PEG runs in your environment; Scheme Server, metadata DB, and key management run in Privacera's SaaS and cloud. Privacera manages Scheme Server, metadata DB, and key management (AWS Secrets Manager). You manage the client application and PEG in your own environment.

sequenceDiagram
    box Customer Environment
        participant APP as Client Application (DBX)
        participant PEG as PEG Server
    end

    box Privacera SaaS Environment
        participant SS as Scheme Server
        participant DB as Metadata DB
    end

    box Privacera Cloud (Key Management)
        participant KMS as Key Management Service (AWS Secrets Manager)
    end

    APP->>PEG: Protect / Unprotect (JWT)
    PEG->>SS: Get Scheme

    SS->>DB: Fetch Scheme Metadata
    DB-->>SS: Scheme + Encrypted Data Key
    Note right of SS: Data key encrypted with Master Key

    SS->>KMS: Get Master Key
    KMS-->>SS: Master Key
    SS->>SS: Decrypt Data Key (in-memory only)

    SS-->>PEG: Scheme Metadata (no plaintext key)

    PEG->>PEG: Encrypt / Decrypt (local execution)
    PEG-->>APP: Encrypted / Decrypted Data

Self-Managed¶

All components run in your environment with your own key management. Privacera provides the PEG and Scheme Server software. You manage PEG, Scheme Server, metadata DB, and your own Key Management Service (customer KMS).

sequenceDiagram
    box Customer Environment
        participant APP as Client Application (DBX)
        participant PEG as PEG Server
        participant SS as Scheme Server
        participant DB as Metadata DB
        participant KMS as Key Management Service (Customer KMS)
    end

    APP->>PEG: Protect / Unprotect (JWT)
    PEG->>SS: Get Scheme

    SS->>DB: Fetch Scheme Metadata
    DB-->>SS: Scheme + Encrypted Data Key
    Note right of SS: Data key encrypted with Master Key

    SS->>KMS: Get Master Key
    KMS-->>SS: Master Key
    SS->>SS: Decrypt Data Key (in-memory only)

    SS-->>PEG: Scheme Metadata (no plaintext key)

    PEG->>PEG: Encrypt / Decrypt (runtime only)
    PEG-->>APP: Encrypted / Decrypted Data

Cluster Type Comparison¶