Skip to content

Tag Reconcile Loader Configuration

Goal

Configure the BigQuery connector to automatically reconcile tags between the connector and Apache Ranger. This loader runs periodically to detect any differences by comparing the tags loaded by the connector with those present in Ranger, and it creates or deletes tags in Ranger as needed to maintain consistency.

Prerequisites

Before you begin, ensure the following:

  • Privacera Manager is installed and the base installation is operational.
  • The BigQuery connector is configured and running.
  • Apache Ranger is properly configured and accessible.

Tag Masking Requirement

These tag reconciliation loader properties will only work if tag masking is enabled in the connector.

Configuration Steps

The tag reconcile loader helps maintain tag consistency between your BigQuery connector and Apache Ranger by:

  • Monitoring tag differences: Compares tags loaded by the connector with those present in Ranger
  • Automatic reconciliation: Creates missing tags in Ranger or removes orphaned tags
  • Configurable intervals: Runs at specified intervals to ensure ongoing consistency

Tag Reconcile Loader Properties:

Property Name Description Default Value Supported Values
CONNECTOR_BIGQUERY_TAG_RECONCILE_LOADER_ENABLED Enable or disable the tag reconcile loader functionality true true, false
CONNECTOR_BIGQUERY_SERVICE_TAG_RECONCILE_SYNC_INTERVAL Set the interval for tag reconcile sync process in seconds 540 Any numeric value in seconds

Configuration Steps

Warning

  • Enabling this feature will automatically create and delete tags in Ranger based on connector data.
  • Lower interval values result in more frequent reconciliation but may increase system load.
  • Be sure to replace the example values with your actual configuration values.

Restart Required

Any changes to these properties require a restart of the BigQuery connector application for the updates to take effect.

  1. Navigate to Settings → Applications in the Self-Managed Portal.

  2. Select BigQuery from the list of Connected Applications.

  3. Click on the application name or the icon, then click on Access Management → ADVANCED tab.

  4. To update the tag reconcile sync interval, add the following property under the Add New Custom Properties section:

    Bash
    1
    ranger.policysync.connector.0.sync.servicetag.reconcile.interval.sec=540

  5. To disable the tag reconcile loader, add the following property under the Add New Custom Properties section:

    Bash
    1
    ranger.policysync.connector.0.tag.reconcile.loading.enabled=false

  6. Click SAVE to apply the changes.

  1. SSH to the instance where Privacera Manager is installed.

  2. Run the following command to open the .yml file to be edited.

    If you have multiple connectors, then replace instance1 with the appropriate connector instance name.

    Bash
    1
    vi ~/privacera/privacera-manager/config/custom-vars/connectors/bigquery/instance1/vars.connector.bigquery.yml

  3. To update the tag reconcile sync interval, add or modify the following property:

    YAML
    1
2
    # Set tag reconcile sync interval (in seconds)
CONNECTOR_BIGQUERY_SERVICE_TAG_RECONCILE_SYNC_INTERVAL: "540"

  4. To disable the tag reconcile loader, add or modify the following property:

    YAML
    1
2
    # Disable tag reconcile loader
CONNECTOR_BIGQUERY_TAG_RECONCILE_LOADER_ENABLED: "false"

  5. Once the properties are configured, run the following commands to update your Privacera Manager platform instance:

    Step 1 - Setup which generates the helm charts. This step usually takes few minutes.

    Bash
    1
2
    cd ~/privacera/privacera-manager
./privacera-manager.sh setup
    Step 2 - Apply the Privacera Manager helm charts.
    Bash
    1
2
    cd ~/privacera/privacera-manager
./pm_with_helm.sh upgrade
    Step 3 - Post-installation step which generates Plugin tar ball, updates Route 53 DNS and so on.

    Bash
    1
2
    cd ~/privacera/privacera-manager
./privacera-manager.sh post-install

  1. In PrivaceraCloud, navigate to Settings → Applications.

  2. Select BigQuery from the list of Connected Applications.

  3. Click on the application name or the icon, then click on Access Management → ADVANCED tab.

  4. To update the tag reconcile sync interval, add the following property under the Add New Custom Properties section:

    Bash
    1
    ranger.policysync.connector.0.sync.servicetag.reconcile.interval.sec=540

  5. To disable the tag reconcile loader, add the following property under the Add New Custom Properties section:

    Bash
    1
    ranger.policysync.connector.0.tag.reconcile.loading.enabled=false

  6. Click SAVE.

  7. Once saved and enabled, the BigQuery connector will start. You can hover on the VIEW LOGS button to check the status, either Running or Stopped.

Note

Perform the following steps only if the connector does not reflect the updated configuration and requires a restart.

Restart the BigQuery Connector:

  1. Go to Settings → Applications → select the BigQuery connector application.

  2. Edit the application → Disable it → and Save it.

  3. Open the same application again and then: Enable it and Save it.

Comments