Skip to content

Synchronization Intervals Configuration

You can configure the BigQuery connector to control the timing and frequency of various synchronization processes. These settings determine how often the connector synchronizes resources, principals, policies, and audit data with BigQuery and your identity providers.

Proper interval configuration is crucial for balancing real-time data consistency with system performance. Lower intervals provide more up-to-date synchronization but increase system load, while higher intervals reduce system load but may result in delays in policy enforcement and audit data collection.

Synchronization Interval Properties:

Property Name Description Default Value Supported Values
RESOURCE SYNC INTERVAL Set the interval for resource sync process in seconds 60 Any numeric value in seconds
PRINCIPAL SYNC INTERVAL Set the interval for principal sync process in seconds 420 Any numeric value in seconds
PERMISSION SYNC INTERVAL Set the interval for existing policies sync process in seconds 540 Any numeric value in seconds
AUDIT SYNC INTERVAL Set the interval for getting access audits process in seconds 30 Any numeric value in seconds

Configuration Steps

Warning

  • Higher intervals reduce system load but may delay policy enforcement.
  • Replace the example values with your actual configuration values.

Restart Required

Any changes to these properties require restarting the BigQuery connector application for the updates to take effect.

  1. Navigate to SettingsApplications in the Self-Managed Portal.

  2. Select BigQuery from the list of Connected Applications.

  3. Click on the application name or the icon, then click on Access ManagementADVANCED tab.

  4. Add the following properties under the Add New Custom Properties section:

    Bash
    1
    2
    3
    4
    ranger.policysync.connector.0.sync.interval.sec=30
    ranger.policysync.connector.0.sync.serviceuser.interval.sec=300
    ranger.policysync.connector.0.sync.servicepolicy.interval.sec=300
    ranger.policysync.connector.0.audit.interval.sec=15
    

  5. Click SAVE to apply the changes.

  1. SSH to the instance where Privacera Manager is installed.

  2. Run the following command to open the .yml file to be edited.

    If you have multiple connectors, then replace instance1 with the appropriate connector instance name.

    Bash
    vi ~/privacera/privacera-manager/config/custom-vars/connectors/bigquery/instance1/vars.connector.bigquery.yml
    
  3. Add or modify the following properties:

    YAML
    # Resource synchronization 
    CONNECTOR_BIGQUERY_RESOURCE_SYNC_INTERVAL: "30"
    
    # Principal synchronization 
    CONNECTOR_BIGQUERY_PRINCIPAL_SYNC_INTERVAL: "300"
    
    # Policy synchronization 
    CONNECTOR_BIGQUERY_PERMISSION_SYNC_INTERVAL: "300"
    
    # Audit synchronization 
    CONNECTOR_BIGQUERY_AUDIT_SYNC_INTERVAL: "15"
    

  4. Once the properties are configured, run the following commands to update your Privacera Manager platform instance:

    Step 1 - Setup which generates the helm charts. This step usually takes few minutes.

    Bash
    cd ~/privacera/privacera-manager
    ./privacera-manager.sh setup
    
    Step 2 - Apply the Privacera Manager helm charts.
    Bash
    cd ~/privacera/privacera-manager
    ./pm_with_helm.sh upgrade
    
    Step 3 - Post-installation step which generates Plugin tar ball, updates Route 53 DNS and so on.

    Bash
    cd ~/privacera/privacera-manager
    ./privacera-manager.sh post-install
    
  1. In PrivaceraCloud, go to SettingsApplications.

  2. Select BigQuery from the list of Connected Applications.

  3. Click on the application name or the icon, then click on Access ManagementADVANCED tab.

  4. Add the following properties under the Add New Custom Properties section:

    Bash
    1
    2
    3
    4
    ranger.policysync.connector.0.sync.interval.sec=30
    ranger.policysync.connector.0.sync.serviceuser.interval.sec=300
    ranger.policysync.connector.0.sync.servicepolicy.interval.sec=300
    ranger.policysync.connector.0.audit.interval.sec=15
    

  5. Click SAVE.

  6. Once saved and enabled, the BigQuery connector will start. You can hover on the VIEW LOGS button to check the status, either Running or Stopped.

Note

Perform the following steps only if the connector does not reflect the updated configuration and requires a restart.

Restart the BigQuery Connector:

  1. Go to SettingsApplications → select the BigQuery connector application.

  2. Edit the application → Disable it → and Save it.

  3. Open the same application again and then: Enable it and Save it.

Comments