Skip to content

AWS Lake Formation API Retry Configuration

This section describes the configurable retry behavior for AWS Lake Formation API calls. These settings help the connector handle transient or intermittent failures gracefully by retrying failed API calls.

Configuration Parameters

The connector supports automatic retries for failed API calls. You can configure:

  • The maximum number of retry attempts before giving up.
  • The interval between each retry, in seconds.

Retry Behavior

When an API call fails due to a transient issue (e.g., timeout, throttling, or network error), the connector will retry the same API call based on the following logic:

  • The initial wait time is the value specified in CONNECTOR_LAKEFORMATION_API_RETRY_INTERVAL.
  • If the API call fails again, the connector will wait an additional multiple of the retry interval before the next attempt:
  • For example, if the retry interval is 30 seconds:
    • After the 1st failure, it waits 30 seconds.
    • After the 2nd failure, it waits 60 seconds.
    • After the 3rd failure, it waits 90 seconds.
    • This continues until the number of retries reaches the value specified in CONNECTOR_LAKEFORMATION_API_MAX_RETRY_LIMIT.

Once the retry limit is reached, the connector stops retrying and records the failure in the log.

Setup

Warning

  • Retry interval is in seconds.
  • Use reasonable retry limits to prevent long delays or excessive API usage.
  • Increasing retries may improve resilience but can also increase sync times during persistent issues.

  1. SSH into the instance where Privacera Manager is installed.

  2. Open the Lake Formation connector configuration file for editing:

    Note

    If you have multiple connectors, replace instance1 with the appropriate connector instance name.

    Bash
    1
    vi ~/privacera/privacera-manager/config/custom-vars/connectors/lakeformation/instance1/vars.connector.lakeformation.push.yml

  3. Set the following properties:

    YAML
    1
2
    CONNECTOR_LAKEFORMATION_API_MAX_RETRY_LIMIT: "3"
CONNECTOR_LAKEFORMATION_API_RETRY_INTERVAL: "30"

    Usage Guide

    • CONNECTOR_LAKEFORMATION_API_MAX_RETRY_LIMIT: Maximum number of retry attempts. Default is 3.
    • CONNECTOR_LAKEFORMATION_API_RETRY_INTERVAL: Wait time between retries in seconds. Increases linearly with each failure (e.g., 30s, 60s, 90s). Default is 30.

  4. After updating the values, apply the changes by running the following commands:

    Step 1 - Setup which generates the helm charts. This step usually takes few minutes.

    Bash
    1
2
    cd ~/privacera/privacera-manager
./privacera-manager.sh setup
    Step 2 - Apply the Privacera Manager helm charts.
    Bash
    1
2
    cd ~/privacera/privacera-manager
./pm_with_helm.sh upgrade
    Step 3 - Post-installation step which generates Plugin tar ball, updates Route 53 DNS and so on.

    Bash
    1
2
    cd ~/privacera/privacera-manager
./privacera-manager.sh post-install

Comments