Skip to content

Prerequisites for BigQuery Connector - Access Management

GCP BigQuery connector with Privacera requires the following prerequisites:

Mandatory Prerequisites

Prerequisites Detail
IAM Role Permissions for managing BigQuery Refer This is used to create Permissions for GCP IAM Role.
Create PrivaceraPolicySyncRole IAM Role for Project Refer This is used create GCP IAM role at project-level for service account.
Create PrivaceraPolicySyncRole IAM Role for organization Refer This is used to create GCP IAM role at organization-level for service account.
Attach IAM Role to Service Account Refer This is used to connect policysync with GCP bigquery.
Configure Logs for Auditing Refer This is used to store audits logs of GCP BigQuery to load by policysync.

Optional Prerequisites

Prerequisites Detail
Basic Authentication for PolicySync Refer (optional) This is used optionally enable basic authenticate for PolicySync to Google BigQuery.

Note

Managing Multiple GCP Projects with a Single BigQuery Connector

If you have multiple projects within your GCP organization and wish to manage them using a single BigQuery connector, you can follow one of the two options below. Choose the option that best fits your needs.

Option 1: Service Account at One Project with Organization-level Role

  1. Create Organization-level IAM Role:

  2. Assign the Role to the Service Account:

    • Create a service account in any one of the projects within your organization.
    • Assign the organization-level IAM role to the service account, granting it the necessary permissions to manage BigQuery across the organization.
    • This ensures the service account can access BigQuery resources in all projects within the organization.
    • Attach IAM Role to Service Account

Option 2: Service Account at One Project with Project-level Role for Each Project

  1. Create IAM Role at Every Project:

  2. Assign the Role to the Service Account:

    • Create a service account in any one of the projects within your organization.
    • Assign the project-level IAM role to the service account for each individual project. This ensures the service account has the required permissions to access BigQuery in all the relevant projects.
    • Attach IAM Role to Service Account

Appendix

IAM Role Permissions for managing BigQuery

You need to give Privacera PolicySync basic access to GCP. To grant that access, create PrivaceraPolicySyncRole IAM role in your GCP project or GCP organization using the following commands on Google Cloud's shell (gcloud). The shell can be installed and accessed locally or through Google Console.

Run the following command to create the file containing the permissions required for the PrivaceraPolicySyncRole role:
Text Only
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
ROLE_NAME="PrivaceraPolicySyncRole"

cat << EOF > ${ROLE_NAME}.yaml
title: "${ROLE_NAME}"
description: "${ROLE_NAME}"
stage: "ALPHA"
includedPermissions:
- resourcemanager.projects.get
- resourcemanager.projects.getIamPolicy
- resourcemanager.projects.setIamPolicy
- iam.roles.list
- iam.roles.get
- iam.roles.create
- iam.roles.update
- bigquery.jobs.create
- bigquery.datasets.get
- bigquery.datasets.create
- bigquery.datasets.update
- bigquery.datasets.delete
- bigquery.datasets.getIamPolicy
- bigquery.datasets.setIamPolicy
- bigquery.tables.list
- bigquery.tables.get
- bigquery.tables.getData
- bigquery.tables.create
- bigquery.tables.update
- bigquery.tables.delete
- bigquery.tables.getIamPolicy
- bigquery.tables.setIamPolicy
- bigquery.rowAccessPolicies.list
- bigquery.rowAccessPolicies.create
- bigquery.rowAccessPolicies.update
- bigquery.rowAccessPolicies.delete
- bigquery.rowAccessPolicies.getIamPolicy
- bigquery.rowAccessPolicies.setIamPolicy
- datacatalog.categories.getIamPolicy
- datacatalog.categories.setIamPolicy
- datacatalog.taxonomies.get
- datacatalog.taxonomies.getIamPolicy
- datacatalog.taxonomies.list
- datacatalog.taxonomies.setIamPolicy
- datacatalog.taxonomies.update
- bigquery.dataPolicies.create
- bigquery.dataPolicies.delete
- bigquery.dataPolicies.get
- bigquery.dataPolicies.getIamPolicy
- bigquery.dataPolicies.list
- bigquery.dataPolicies.setIamPolicy
- bigquery.dataPolicies.update
- bigquery.tables.setCategory

EOF

Create Project-level Access IAM Role for Service Account

To create the PrivaceraPolicySyncRole in your GCP project, execute the following command. Replace <GCP_PROJECT_ID> with your actual GCP project ID:
PROJECT_ID="<GCP_PROJECT_ID>"

To create PrivaceraPolicySyncRole role in your GCP project, run the following command.
Text Only
1
gcloud iam roles create ${ROLE_NAME} --project=${PROJECT_ID} --file=${ROLE_NAME}.yaml

Create Organization-level IAM Role for Service Account

To create the PrivaceraPolicySyncRole role in your GCP organization, run the following command. Replace <GCP_ORGANIZATION_ID> with your GCP organization ID:
ORGANIZATION_ID="<GCP_ORGANIZATION_ID>"

To create PrivaceraPolicySyncRole role in your GCP organization, run the following command.
Text Only
1
 gcloud iam roles create ${ROLE_NAME} --organization=${ORGANIZATION_ID} --file=${ROLE_NAME}.yaml

Attach IAM Role to Service Account

To attach the PrivaceraPolicySyncRole IAM role created above, please follow the below steps:

  1. Log in to your GCP console.

  2. Navigate to IAM & admin > Service accounts, and click + CREATE SERVICE ACCOUNT.

  3. Enter the required values in the fields and click CREATE.

  4. In the Grant this service account access to project section, select the role PrivaceraPolicySyncRole.

  5. On the Service Account Page, locate the newly created service account and copy its email address for use in a later step.

  6. If you are using a Google VM instance to configure Google BigQuery for PolicySync, you may attach the service account created above to your VM instance and skip the following steps.

  7. On the Service Account Page, navigate to the Keys tab, click Add Key, and then select Create New Key.

  8. Select the JSON key type and click CREATE. A JSON key file will be downloaded to your system. Please store the file in an accessible location, as it will be used to configure PolicySync in Privacera Manager.

For more detailed information on creating a service account, see the Google documentation.

Configure Logs for Auditing

A sink is required to collect all logs from Google BigQuery. To create a sink, please follow the below steps:

  1. In the search bar, type Logging, then click on Logs Router and select Create Sink.

  2. Enter the sink name as PolicySyncBigQueryAuditSink, and then click Next.

  3. Enter the sink destination.

  4. In the Select sink service section, choose BigQuery.

  5. In the Select BigQuery dataset section, click Create new BigQuery dataset.

  6. Enter the Dataset ID as bigquery_audits and click Create Dataset.

  7. Click Next.

  8. Add the BigQuery logs in the sink:

    • In the Build an inclusion filter, add the following line:
      Text Only
      1
      resource.type="bigquery_resource"

  9. Click Create Sink.

For more detailed information on creating a sink, see the Google documentation.

Basic Authentication for PolicySync (Optional)

To optionally enable basic authentication for PolicySync with Google BigQuery, you may create a JSON file in the subdirectory of your connector instance. The name of the file must be XXX.json.

An example of the contents of XXX.json.
JSON
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
 {
  "type": "service_account",
  "project_id": "your_project_id",
  "private_key_id": "autogenerated_value",
  "private_key": "-----BEGIN PRIVATE KEY-----autogenerated_value-----END PRIVATE KEY-----\n",
  "client_email": "autogenerated_value",
  "client_id": "autogenerated_value",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/autogenerated_value"
}

Comments