Setup for BigQuery Connector - Access Management¶

1. Navigate to Settings → Applications in the Self-Managed Portal.

2. Select the BigQuery connector under Available Connections.

3. Enter the application Name and Description, then click Save

   • The name can be any meaningful identifier, such as bigquery-connector-1.
   • The description can provide additional context, e.g. BigQuery Connector for dev account xyz.

4. Click on the application name or the icon. Then, enable the Access Management toggle.

5. Under the BASIC tab, enter values for the following fields:

   Note

   • To view the property details, click the icon next to the property name.
   • Fields with asterisks (*) are mandatory.
   • Requires restart indicates that the connector pod will restart after updating the property.
   • Cleans RocksDB: When this property is updated, the connector pod initiates a cleanup process to remove cached data and other internal resources from RocksDB. The duration of this process may vary depending on the volume of data and resources being managed.
   • If you modify any of the following infrastructure-related properties, make sure to download the updated ZIP file and repeat the setup process:
     1. Use Default GCP Credentials when no service account JSON
     2. Service Account Email
     3. Or both properties

   Field	Description
   BigQuery Project Location	Add the project location, e.g. us.
   BigQuery Project ID	Use the BigQuery Project ID from GCP, e.g. projectXXX.
   Service Account Email	Provide the email address of the service account with the required GCP permissions.
   Projects to Set Access Control Policies	Enter the Project IDs to be managed.
   Native Public Group Identity Name	Specify the identity for access grants in policies that reference public groups: ALL_AUTHENTICATED_USERS or ALL_USERS.
   Use Default GCP Credentials when no Service account json is provided	Used when K8S_PROVIDER is GCP and when no Service account json is provided

6. Click the SAVE button.

7. Download the configuration as a YML file by clicking the icon in the Action column.

8. Copy the downloaded zip file to the following location on Privacera Manager instance:

   Bash
   1	~/privacera/privacera-manager/config/custom-vars/

9. SSH into the Privacera Manager instance, navigate to the location where the zip file was copied, and unzip the file:

   Bash
   1 2	cd ~/privacera/privacera-manager/config/custom-vars/ unzip <file_name>.zip

   Note

   • Backup existing configurations if needed.
   • Ensure unzip is installed on the system.
   • The replacement will only replace files with the same name; other files within the folder remain unchanged.

10. Once all changes are complete, you can start the connector by running the following commands:

    Step 1 - Setup which generates the helm charts. This step usually takes few minutes.

    Bash
    1 2	cd ~/privacera/privacera-manager ./privacera-manager.sh setup

    Step 2 - Apply the Privacera Manager helm charts.

    Bash
    1 2	cd ~/privacera/privacera-manager ./pm_with_helm.sh upgrade

    Step 3 - Post-installation step which generates Plugin tar ball, updates Route 53 DNS and so on.

    Bash
    1 2	cd ~/privacera/privacera-manager ./privacera-manager.sh post-install

11. Once the connector is started, you can manage and monitor it through the Self-Managed portal.

Please perform the following steps to configure the GCP BigQuery connector:

1. SSH into the instance where Privacera is installed.

2. Run the following command to navigate to the /config directory.

   Bash
   1	cd ~/privacera/privacera-manager/config

3. Run the following command to create a new directory:

   Bash
   1	mkdir -p custom-vars/connectors/bigquery/instance1

4. Run the following command to copy the sample vars:

   Bash
   1	cp sample-vars/vars.connector.bigquery.yml custom-vars/connectors/bigquery/instance1/

5. Run the following command to open the .yml file to be edited.

   Bash
   1	vi custom-vars/connectors/bigquery/instance1/vars.connector.bigquery.yml

6. Transfer the Service Account key JSON file, obtained during Attach IAM Role to Service Account, to the /custom-vars/connectors/bigquery/instance1/ directory. If using Workload Identity, ensure that the CONNECTOR_BIGQUERY_USE_VM_CREDENTIALS variable is set to true, and you do not need the JSON file. If not using Workload Identity, update the vars.connector.bigquery.yml file by setting the CONNECTOR_BIGQUERY_OAUTH_PRIVATE_KEY_FILE_NAME variable to the name of the newly transferred JSON file.

7. Modify the following properties:

   • CONNECTOR_BIGQUERY_PROJECT_ID - Enter the GCP Bigquery Project ID you want to use.

   • CONNECTOR_BIGQUERY_PROJECT_LOCATION - Set GCP Bigquery Project Location.

   • CONNECTOR_BIGQUERY_USE_VM_CREDENTIALS - Set this true if you are running the connector on a GCP instance and have performed the steps mentioned here.

   • CONNECTOR_BIGQUERY_OAUTH_SERVICE_ACCOUNT_EMAIL - Set the service account email with access to the GCP resources required by PolicySync. If CONNECTOR_BIGQUERY_USE_VM_CREDENTIALS is set to true, the service account from the Workload Identity setup will be used.

   • CONNECTOR_BIGQUERY_OAUTH_PRIVATE_KEY_FILE_NAME - Specify the name of the service account's private key file (in JSON format) if CONNECTOR_BIGQUERY_USE_VM_CREDENTIALS is set to false. If CONNECTOR_BIGQUERY_USE_VM_CREDENTIALS is true, the private key file is not required, as Workload Identity will be used.

   • CONNECTOR_BIGQUERY_CUSTOM_IAM_ROLES_SCOPE - Set the scope of the custom IAM roles to be created. Allowed values: "project" or "org".

   • CONNECTOR_BIGQUERY_ORGANIZATION_ID - Enter your GCP organization ID if you want to create IAM roles at the organization level.

   • CONNECTOR_BIGQUERY_AUDIT_ENABLE - Set to "true" to enable auditing for BigQuery.

   • CONNECTOR_BIGQUERY_AUDIT_EXCLUDED_USERS - Enter a comma-separated list of users whose access audits you want to exclude. If not set, it excludes the service account set in CONNECTOR_BIGQUERY_OAUTH_SERVICE_ACCOUNT_EMAIL by default.

   • CONNECTOR_BIGQUERY_MANAGE_PROJECT_LIST - Enter a comma-separated list of project IDs that PolicySync will manage.

   • CONNECTOR_BIGQUERY_MANAGE_DATASET_LIST - Enter a comma-separated list of datasets that PolicySync will manage. Format: <PROJECT_ID>.<DATASET_NAME>.

   • CONNECTOR_BIGQUERY_GRANT_UPDATES - Set to "true" to enable PolicySync to perform grant and revoke updates on access control.

   • CONNECTOR_BIGQUERY_NATIVE_PUBLIC_GROUP_IDENTITY_NAME - Enter the public group identity name for access grants. Allowed values: "ALL_AUTHENTICATED_USERS", "ALL_USERS", or any valid Google group name.

   • CONNECTOR_BIGQUERY_ENABLE - Set to "true" to enable PolicySync for BigQuery.

   • CONNECTOR_BIGQUERY_MANAGE_ENTITY_PREFIX - Enter a comma-separated list of prefixes for users, groups, and roles that PolicySync will manage. For example: dev_*, sa_*. Leave blank to manage all entities.

8. Once the properties have been configured, execute the following commands to update your Privacera Manager platform instance:

   Bash
   1 2 3	cd ~/privacera/privacera-manager ./privacera-manager.sh update

1. In PrivaceraCloud, navigate to Settings -> Applications.

2. On the Applications screen, select BigQuery.

3. Enter the application Name and Description, then click Save. The name can be any designation of your choice, such as BigQuery Connector for account 123456.

4. Open the BigQuery application.

5. Enable the Access Management option using the toggle button.

6. Under the BASIC tab, enter the values for:

   • BigQuery Project Location : us

   • BigQuery Project Id : projectXXX

   • Service Account Email : Use the service account email which have access to GCP resources required by PolicySync.

   • Service Account Json Key Content : Use Service Account JSON

   • Projects to Manage Access Control Policies : Project IDs you want to manage

   • Native Public Group Identity Name : Specifies the public group identity to be used for access grants in policies referring to public groups. Allowed values: ALL_AUTHENTICATED_USERS (All GCP project authenticated users) or ALL_USERS (All Google authenticated users).

   • Enable Audit : Specifies whether Privacera fetches access audit data from the data source. Allowed values: true or false.

7. Click SAVE.

8. The configured BigQuery connector appears under Applications.

9. Once saved and enabled, the BigQuery connector will start. Then you can hover on the VIEW LOGS button to check the status, either Running or Stopped.

10. Perform following steps to restart the BigQuery connector application:

    1. Go to Settings → Applications → select the****BigQuery** connector application** .

    2. Edit the application → Disable it → and Save it.

    3. Open the same application again and then: Enable it → and Save it.