Skip to content

Access Audits Management

The BigQuery connector allows you to enable and manage access audits using PolicySync, providing detailed tracking and filtering of access events. You can configure how audit logs are collected and processed, with key settings including log collection frequency, the maximum interval for loading audit data, and progress reporting for audit operations.

This guide provides complete instructions for configuring audit collection and processing in the BigQuery connector, covering both basic setup and advanced options.

Prerequisites

Before you begin, ensure the following:

  • Privacera Manager is installed and the base installation operational.
  • The BigQuery connector is configured or currently being configured.

Basic Audit Configuration

Configure the BigQuery connector to collect access audit logs from your BigQuery environment.

Basic Audit Properties:

Property Name Description Required Supported Values
AUDIT ENABLE Enables access audit data fetching from BigQuery Yes true, false
AUDIT PROJECT ID Project ID where BigQuery audit logs are stored Yes Valid GCP project ID
AUDIT DATASET NAME Dataset name containing the audit data Yes Valid BigQuery dataset name
AUDIT EXCLUDED USERS Comma-separated list of users to exclude from audit logs No Email addresses

Configuration Steps

Warning

  • Replace the example values with your actual configuration values.

  1. Navigate to Settings → Applications in the Self-Managed Portal.

  2. Select BigQuery from the list of Connected Applications.

  3. Click on the application name or the icon, then click on Access Management.

  4. To enable access audits for BigQuery, enter the following values in the respective fields under BASIC tab:

    • Enable access audits: Enable to fetch access audits from BigQuery.
    • Project id having the bigquery audits datasets configured: Enter the project ID where the BigQuery audit are stored.
    • Dataset name to fetch the access audits: Enter the dataset name to fetch the access audits.

  5. Under the ADVANCED tab, optionally configure:

    • Users to exclude when fetching access audits: Enter a comma-separated list of user email addresses to exclude from audit logs.

  6. Click SAVE to apply the changes.

  1. SSH to the instance where Privacera Manager is installed.

  2. Run the following command to open the .yml file to be edited.

    If you have multiple connectors, then replace instance1 with the appropriate connector instance name.

    Bash
    1
    vi ~/privacera/privacera-manager/config/custom-vars/connectors/bigquery/instance1/vars.connector.bigquery.yml

  3. Add or modify the following properties:

    YAML
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
    # Enable access audit fetching from BigQuery
CONNECTOR_BIGQUERY_AUDIT_ENABLE: "true"

# Set the list of users whose access audits should be ignored
CONNECTOR_BIGQUERY_AUDIT_EXCLUDED_USERS: "user1@example.com, user2@example.com"

# Set the project ID to fetch BigQuery audits
CONNECTOR_BIGQUERY_AUDIT_PROJECT_ID: "your-bigquery-project-id"

# Set the dataset name to fetch BigQuery audits
CONNECTOR_BIGQUERY_AUDIT_DATASET_NAME: "your-bigquery-dataset-name"

  4. Once the properties are configured, run the following commands to update your Privacera Manager platform instance:

    Step 1 - Setup which generates the helm charts. This step usually takes few minutes.

    Bash
    1
2
    cd ~/privacera/privacera-manager
./privacera-manager.sh setup
    Step 2 - Apply the Privacera Manager helm charts.
    Bash
    1
2
    cd ~/privacera/privacera-manager
./pm_with_helm.sh upgrade
    Step 3 - Post-installation step which generates Plugin tar ball, updates Route 53 DNS and so on.

    Bash
    1
2
    cd ~/privacera/privacera-manager
./privacera-manager.sh post-install

  1. In PrivaceraCloud, go to Settings → Applications.

  2. Select BigQuery from the list of Connected Applications.

  3. Click on the application name or the icon, then click on Access Management.

  4. Under the BASIC tab, enter the values for:

    • Enable access audits: Enable to fetch access audits from BigQuery.
    • Project id having the bigquery audits datasets configured: Enter the project ID where the BigQuery audit logs are stored.
    • Dataset name to fetch the access audits: Enter the dataset name to fetch the access audits.

  5. Under the ADVANCED tab, optionally configure:

    • Ignore Audit for Users: Enter a comma-separated list of user email addresses to exclude from audit logs.

  6. Click SAVE.

  7. Once saved and enabled, the BigQuery connector will start. You can hover on the VIEW LOGS button to check the status, either Running or Stopped.

Advanced Audit Configuration

Configure advanced settings to control audit log collection intervals, processing behavior, and progress reporting.

Advanced Audit Properties:

Property Name Description Default Value Supported Values
AUDIT LOAD MAX INTERVAL MINUTES Sets the maximum interval for loading audit logs in minutes 30 Any numeric value in minutes
AUDIT PROGRESS REPORT TRACKER ENABLE Enables progress report tracker for audit logs false true, false
AUDIT PROGRESS COMPLETE REPORT ENABLE Enables progress report for completed audit logs true true, false
AUDIT PROGRESS COMPLETE REPORT INTERVAL SEC Sets the interval for progress report for completed audit logs in seconds 1800 Any numeric value in seconds

Configuration Steps

Warning

  • Replace the example values with your actual configuration values.

Restart Required

Any changes to these properties require restarting the BigQuery connector application for the updates to take effect.

  1. Navigate to Settings → Applications in the Self-Managed Portal.

  2. Select BigQuery from the list of Connected Applications.

  3. Click on the application name or the icon, then click on Access Management → ADVANCED tab.

  4. Add the following properties under the Add New Custom Properties section:

    Bash
    1
2
3
4
    ranger.policysync.connector.0.audit.log.load.max.interval.minutes=30
ranger.policysync.connector.0.audit.progress.report.tracker.enable=false
ranger.policysync.connector.0.audit.progress.complete.report.enable=true
ranger.policysync.connector.0.audit.progress.complete.report.interval.sec=1800

  5. Click SAVE to apply the changes.

  1. SSH to the instance where Privacera Manager is installed.

  2. Run the following command to open the .yml file to be edited.

    If you have multiple connectors, then replace instance1 with the appropriate connector instance name.

    Bash
    1
    vi ~/privacera/privacera-manager/config/custom-vars/connectors/bigquery/instance1/vars.connector.bigquery.yml

  3. Add or modify the following properties:

    YAML
    1
2
3
4
5
6
7
    # Access audits management
CONNECTOR_BIGQUERY_AUDIT_LOAD_MAX_INTERVAL_MINUTES: "30"

# Audit Loader Progress Reporting
CONNECTOR_BIGQUERY_AUDIT_PROGRESS_REPORT_TRACKER_ENABLE: "false"
CONNECTOR_BIGQUERY_AUDIT_PROGRESS_COMPLETE_REPORT_ENABLE: "true"
CONNECTOR_BIGQUERY_AUDIT_PROGRESS_COMPLETE_REPORT_INTERVAL_SEC: "1800"

  4. Once the properties are configured, run the following commands to update your Privacera Manager platform instance:

    Step 1 - Setup which generates the helm charts. This step usually takes few minutes.

    Bash
    1
2
    cd ~/privacera/privacera-manager
./privacera-manager.sh setup
    Step 2 - Apply the Privacera Manager helm charts.
    Bash
    1
2
    cd ~/privacera/privacera-manager
./pm_with_helm.sh upgrade
    Step 3 - Post-installation step which generates Plugin tar ball, updates Route 53 DNS and so on.

    Bash
    1
2
    cd ~/privacera/privacera-manager
./privacera-manager.sh post-install

  1. In PrivaceraCloud, go to Settings → Applications.

  2. Select BigQuery from the list of Connected Applications.

  3. Click on the application name or the icon, then click on Access Management → ADVANCED tab.

  4. Add the following properties under the Add New Custom Properties section:

    Bash
    1
2
3
4
    ranger.policysync.connector.0.audit.log.load.max.interval.minutes=30
ranger.policysync.connector.0.audit.progress.report.tracker.enable=false
ranger.policysync.connector.0.audit.progress.complete.report.enable=true
ranger.policysync.connector.0.audit.progress.complete.report.interval.sec=1800

  5. Click SAVE.

  6. Once saved and enabled, the BigQuery connector will start. You can hover on the VIEW LOGS button to check the status, either Running or Stopped.

Note

Perform the following steps only if the connector does not reflect the updated configuration and requires a restart.

Restart the BigQuery Connector:

  1. Go to Settings → Applications → select the BigQuery connector application.

  2. Edit the application → Disable it → and Save it.

  3. Open the same application again and then: Enable it and Save it.

Comments