Skip to content

Progress Reporting Configuration

You can configure the BigQuery connector to control progress reporting across different components. These settings determine how the connector tracks and reports progress for various operations including permission loading, principal synchronization, resource loading, and change processing activities.

Progress reporting provides operational visibility into connector activities and helps with monitoring, troubleshooting, and performance optimization. You can enable detailed tracking for specific components or configure completion reports to get periodic status updates.

Progress Reporting Properties:

Property Name Description Default Value Supported Values
PERMISSIONLOADER PROGRESS REPORT TRACKER ENABLE Enables progress report tracker for permission loader logs false true, false
PERMISSIONLOADER PROGRESS COMPLETE REPORT ENABLE Enables progress report for completed permission loader logs true true, false
PERMISSIONLOADER PROGRESS COMPLETE REPORT INTERVAL SEC Sets the interval for progress report for completed permission loader logs in seconds 1800 Any numeric value in seconds
PRINCIPALLOADER PROGRESS REPORT TRACKER ENABLE Enables progress report tracker for principal loader logs false true, false
PRINCIPALLOADER PROGRESS COMPLETE REPORT ENABLE Enables progress report for completed principal loader logs true true, false
PRINCIPALLOADER PROGRESS COMPLETE REPORT INTERVAL_SEC Sets the interval for progress report for completed principal loader logs in seconds 1800 Any numeric value in seconds
RESOURCELOADER PROGRESS REPORT TRACKER ENABLE Enables progress report tracker for resource loader logs false true, false
RESOURCELOADER PROGRESS COMPLETE REPORT ENABLE Enables progress report for completed resource loader logs true true, false
RESOURCELOADER PROGRESS COMPLETE REPORT INTERVAL SEC Sets the interval for progress report for completed resource loader logs in seconds 1800 Any numeric value in seconds
CHANGELOG PROGRESS REPORT TRACKER ENABLE Enables progress report tracker for change log loader logs true true, false
CHANGEPROCESSOR PROGRESS REPORT TRACKER ENABLE Enables progress report tracker for change processor logs true true, false
RESOURCESINK PROGRESS REPORT TRACKER ENABLE Enables progress report tracker for resource sink logs true true, false
RANGERPRINCIPALLOADER PROGRESS REPORT ENABLE Enables progress report tracker for ranger principal loader logs true true, false
RANGERPRINCIPALLOADER PROGRESS REPORT INTERVAL SEC Sets the interval for progress report for completed ranger principal loader logs in seconds 1800 Any numeric value in seconds

Configuration Steps

Warning

  • Replace the example values with your actual configuration values.

Restart Required

Any changes to these properties require restarting the BigQuery connector application for the updates to take effect.

  1. Navigate to Settings → Applications in the Self-Managed Portal.

  2. Select BigQuery from the list of Connected Applications.

  3. Click on the application name or the icon, then click on Access Management → ADVANCED tab.

  4. Add the following properties under the Add New Custom Properties section:

    Bash
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
    ranger.policysync.connector.0.permissionloader.progress.report.tracker.enable=true
ranger.policysync.connector.0.permissionloader.progress.complete.report.enable=true
ranger.policysync.connector.0.permissionloader.progress.complete.report.interval.sec=900
ranger.policysync.connector.0.principalloader.progress.report.tracker.enable=true
ranger.policysync.connector.0.principalloader.progress.complete.report.enable=true
ranger.policysync.connector.0.principalloader.progress.complete.report.interval.sec=900
ranger.policysync.connector.0.resourceloader.progress.report.tracker.enable=true
ranger.policysync.connector.0.resourceloader.progress.complete.report.enable=true
ranger.policysync.connector.0.resourceloader.progress.complete.report.interval.sec=900
ranger.policysync.connector.0.changelog.progress.report.tracker.enable=true
ranger.policysync.connector.0.changeprocessor.progress.report.tracker.enable=true
ranger.policysync.connector.0.resourcesink.progress.report.tracker.enable=true
ranger.policysync.connector.0.rangerprincipalloader.progress.report.enable=true
ranger.policysync.connector.0.rangerprincipalloader.progress.report.interval.sec=900

  5. Click SAVE to apply the changes.

  1. SSH to the instance where Privacera Manager is installed.

  2. Run the following command to open the .yml file to be edited.

    If you have multiple connectors, then replace instance1 with the appropriate connector instance name.

    Bash
    1
    vi ~/privacera/privacera-manager/config/custom-vars/connectors/bigquery/instance1/vars.connector.bigquery.yml

  3. Add or modify the following properties:

    YAML
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
    # Permission loader progress tracking
CONNECTOR_BIGQUERY_PERMISSIONLOADER_PROGRESS_REPORT_TRACKER_ENABLE: "true"
CONNECTOR_BIGQUERY_PERMISSIONLOADER_PROGRESS_COMPLETE_REPORT_ENABLE: "true"
CONNECTOR_BIGQUERY_PERMISSIONLOADER_PROGRESS_COMPLETE_REPORT_INTERVAL_SEC: "300"

# Principal loader progress tracking
CONNECTOR_BIGQUERY_PRINCIPALLOADER_PROGRESS_REPORT_TRACKER_ENABLE: "true"
CONNECTOR_BIGQUERY_PRINCIPALLOADER_PROGRESS_COMPLETE_REPORT_ENABLE: "true"
CONNECTOR_BIGQUERY_PRINCIPALLOADER_PROGRESS_COMPLETE_REPORT_INTERVAL_SEC: "300"

# Resource loader progress tracking
CONNECTOR_BIGQUERY_RESOURCELOADER_PROGRESS_REPORT_TRACKER_ENABLE: "true"
CONNECTOR_BIGQUERY_RESOURCELOADER_PROGRESS_COMPLETE_REPORT_ENABLE: "true"
CONNECTOR_BIGQUERY_RESOURCELOADER_PROGRESS_COMPLETE_REPORT_INTERVAL_SEC: "300"

# Change processing progress tracking
CONNECTOR_BIGQUERY_CHANGELOG_PROGRESS_REPORT_TRACKER_ENABLE: "true"
CONNECTOR_BIGQUERY_CHANGEPROCESSOR_PROGRESS_REPORT_TRACKER_ENABLE: "true"

# Resource Sink  progress tracking
CONNECTOR_BIGQUERY_RESOURCESINK_PROGRESS_REPORT_TRACKER_ENABLE: "true"

#Ranger Sync Principal progess tracking
CONNECTOR_BIGQUERY_RANGERPRINCIPALLOADER_PROGRESS_REPORT_ENABLE: "true"
CONNECTOR_BIGQUERY_RANGERPRINCIPALLOADER_PROGRESS_REPORT_INTERVAL_SEC: "300"

  4. Once the properties are configured, run the following commands to update your Privacera Manager platform instance:

    Step 1 - Setup which generates the helm charts. This step usually takes few minutes.

    Bash
    1
2
    cd ~/privacera/privacera-manager
./privacera-manager.sh setup
    Step 2 - Apply the Privacera Manager helm charts.
    Bash
    1
2
    cd ~/privacera/privacera-manager
./pm_with_helm.sh upgrade
    Step 3 - Post-installation step which generates Plugin tar ball, updates Route 53 DNS and so on.

    Bash
    1
2
    cd ~/privacera/privacera-manager
./privacera-manager.sh post-install

  1. In PrivaceraCloud, go to Settings → Applications.

  2. Select BigQuery from the list of Connected Applications.

  3. Click on the application name or the icon, then click on Access Management → ADVANCED tab.

  4. Add the following properties under the Add New Custom Properties section:

    Bash
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
    ranger.policysync.connector.0.permissionloader.progress.report.tracker.enable=false
ranger.policysync.connector.0.permissionloader.progress.complete.report.enable=true
ranger.policysync.connector.0.permissionloader.progress.complete.report.interval.sec=1800
ranger.policysync.connector.0.principalloader.progress.report.tracker.enable=false
ranger.policysync.connector.0.principalloader.progress.complete.report.enable=true
ranger.policysync.connector.0.principalloader.progress.complete.report.interval.sec=1800
ranger.policysync.connector.0.resourceloader.progress.report.tracker.enable=false
ranger.policysync.connector.0.resourceloader.progress.complete.report.enable=true
ranger.policysync.connector.0.resourceloader.progress.complete.report.interval.sec=1800
ranger.policysync.connector.0.changelog.progress.report.tracker.enable=true
ranger.policysync.connector.0.changeprocessor.progress.report.tracker.enable=true
ranger.policysync.connector.0.resourcesink.progress.report.tracker.enable=true
ranger.policysync.connector.0.rangerprincipalloader.progress.report.enable=true
ranger.policysync.connector.0.rangerprincipalloader.progress.report.interval.sec=1800

  5. Click SAVE.

  6. Once saved and enabled, the BigQuery connector will start. You can hover on the VIEW LOGS button to check the status, either Running or Stopped.

Note

Perform the following steps only if the connector does not reflect the updated configuration and requires a restart.

Restart the BigQuery Connector:

  1. Go to Settings → Applications → select the BigQuery connector application.

  2. Edit the application → Disable it → and Save it.

  3. Open the same application again and then: Enable it and Save it.

Comments