Skip to content

Handling Invalid Input for FPE Algorithm

The FPE (Format Preserving Encryption) algorithm encrypts data while maintaining its original format. The data being protected must match the format type specified when creating the schemes. If the format type is incorrect, PEG will throw an error, which can cause ETL processes to fail, including: - Databricks jobs - Streamsets pipelines reading data through Kafka or MongoDB - FEU & Discovery large files processing

To handle this issue, PEG supports ignoring errors caused by invalid data and returning either an empty value or the original input value after encryption and decryption. By default, the configuration returns an empty value. Note that if the return value is set to empty for encryption/decryption, the data will be non-recoverable. You can modify this behavior by updating the configuration variables in the vars.peg.yml file.

Name Default Description
PEG_V2_NATIVE_FPE_INVALID_VALUE_IGNORE true When set to true, ignores errors for invalid data and returns the configured value (EMPTY/ORIGINAL).
PEG_V2_NATIVE_FPE_INVALID_VALUE_REPLACE EMPTY Configures return value: ORIGINAL returns the input value, EMPTY returns an empty string (non-recoverable).

Setup

To configure the handling of invalid input for the FPE algorithm:

  1. Navigate to the Privacera Manager directory:

    Bash
    1
    cd ~/privacera/privacera-manager

  2. Edit the vars.peg.yml file:

    Bash
    1
    vi config/custom-properties/vars.peg.yml

  3. Add or update the following properties:

    YAML
    1
2
    PEG_V2_NATIVE_FPE_INVALID_VALUE_REPLACE: "EMPTY"
PEG_V2_NATIVE_FPE_INVALID_VALUE_IGNORE:  "true"

Restart Privacera Services

Bash
1
2
3
cd ~/privacera/privacera-manager
./privacera-manager.sh setup
./pm_with_helm.sh upgrade

Comments