Skip to content

Manage Resources List

You can configure the GCP BigQuery connector to manage access control policies for specific projects, datasets, tables, and policy tags. By specifying inclusion and exclusion lists, you can control which resources are managed. The connector applies access control policies to resources in the include list and ignores those in the exclude list. If a resource appears in both the include and exclude lists, the connector will not manage it.

Use the following properties to specify comma-separated projects, datasets, tables, and policy tags whose access control should be managed by PolicySync. To manage all resources, simply leave these properties unspecified. Wildcard characters (*) can be used to match multiple projects, datasets, tables, and policy tags.

Example:

  • Project: gcp-project-123, gcp-project*
  • Dataset: gcp-project-123.analytics_db, gcp-project*.test_db*
  • Table: gcp-project-123.analytics_db.customer_table, gcp-project*.test_db*.*
  • Policy Tag: org/530746474/projects/gcp-project-123/locations/us/taxonomies/123456789/policyTags/Mask1, org/53074647454/projects/gcp-project*/locations/us*/taxonomies/*/policyTags/*

Policy Tag Management Guidelines

Policy Loss on Tag Deletion

When a policy tag is deleted, all associated tag-based policies are permanently deleted and cannot be recovered. These policies must be manually recreated after tag restoration.

Guidelines to Prevent Tag Deletion

When managing policy tags through the BigQuery connector, it's important to follow these guidelines to prevent data loss and ensure proper tag management.

  1. Single Connector Management

    Each policy tag must be managed by only one connector to prevent conflicts or accidental deletions.

    Important

    Do not manage the same set of tags using multiple connectors. This can lead to unintended tag deletions and policy loss.

  2. Connector Tag Management List Changes

    When a connector's tag management list is modified, it may delete previously managed tags from the portal—even if those tags are also managed by another connector.

    Reconcile Flow Behavior

    During the reconcile flow, another connector may recreate deleted tags if they are still applicable. However, any tag-based policies associated with those deleted tags will be permanently lost and must be manually recreated after the tags are restored.

  3. Disable Tag Deletion by Connectors

    To prevent a connector from deleting any tags from the portal—regardless of service-side changes or updates to the managed list—configure the following properties:

    Recommended Configuration

    Setting these properties to false ensures that the connector does not delete tags from the portal under any circumstances, preventing accidental policy loss.

    Bash
    ranger.policysync.connector.0.tag.delete.enabled: "false"
    YAML
    CONNECTOR_BIGQUERY_DELETE_SERVICE_TAG_ENABLED: "false"
    Bash
    ranger.policysync.connector.0.tag.delete.enabled: "false"

Important Considerations

Key Points

  • Tag Recreation: A connector will recreate any tag that was deleted manually or by another connector during its reconcile flow, as long as it is still managing that tag.
  • Policy Impact: Tag-based policies are permanently deleted when the associated tag is deleted. These policies cannot be recovered automatically and must be manually recreated after tag restoration.
  • Manual Recovery Required: If tag-based policies are deleted due to tag removal, they must be manually recreated after the tags are restored.
  • Multiple Connector Risk: Ensure that the same tag is not managed by multiple connectors to avoid unintentional policy deletions.

Best Practices

  • Assign each policy tag to only one connector
  • Configure tag deletion prevention settings
  • Have a recovery plan for manually recreating policies if tags are accidentally deleted

Prerequisites

  1. You have successfully installed Privacera Manager and have the base installation operational.
  2. You have configured the connector for BigQuery or are in the process of doing so.

Configuration Steps

Warning

  • Values are case-sensitive.
  • Provide fully qualified names for project, datasets, tables, and policy tags where applicable.
  • Example (for table resource only): gcp-project-123.analytics_db.customer_table
  • Updating policy tag fields restart the service.
  • Replace the example values with your actual resource names.

  1. Navigate to Settings → Applications in the Self-Managed Portal.

  2. Select BigQuery from the list of Connected Applications.

  3. Click on the application name or the icon, then click on Access Management.

  4. For including resources, enter the following values in the respective fields under BASIC tab:

    • Projects to manage access control policies : <your-project-name>
      • Example: gcp-project-123
    • Datasets to manage access control policies : <your-project-name.your-dataset-name>
      • Example: gcp-project-123.test_db
    • Tables to manage access control policies : <your-project-name.your-dataset-name.your-table-name>
      • Example: gcp-project-123.analytics_db.customer_table, gcp-project-123.analytics_db.finance_*

    Note

    • If manage.table.list is either empty or undefined, the connector will manage all tables present within the datasets specified in manage.dataset.list.
    • If any table patterns are defined in manage.table.list, the connector will only manage tables that match those patterns, even if their parent dataset is included in manage.dataset.list.
    • To manage all tables within specific datasets when table patterns are in use, you must explicitly include wildcard entries such as dataset_name.* in manage.table.list. Without these wildcards, tables from other datasets listed in manage.dataset.list will be skipped unless their individual patterns are also explicitly specified.

  5. For including policy tags resources, enter the following value in the respective field under ADVANCED tab:

    • Policy tags to manage access control policies : org/<your-org-id>/projects/<your-project-name>/locations/<your-location>/taxonomies/<your-taxonomy-id>/policyTags/<your-policy-tag-id>
      • Example: org/530746474/projects/gcp-project-123/locations/us/taxonomies/123456789/policyTags/Mask1, org/530746474/projects/gcp-project*/locations/us/taxonomies/*/policyTags/*

  6. For excluding resources, enter the following values in the respective fields under ADVANCED tab:

    • Projects to ignore while setting access control policies : <your-project-name>
      • Example: gcp-project-123
    • Datasets to ignore while setting access control policies : <your-project-name.your-dataset-name>
      • Example: gcp-project-123.test_db
    • Tables to ignore while setting access control policies : <your-project-name.your-dataset-name.your-table-name>
      • Example: gcp-project-123.analytics_db.customer_table, gcp-project-123.analytics_db.finance_*
    • Policy Tags to ignore while setting access control policies : org/<your-org-id>/projects/<your-project-name>/locations/<your-location>/taxonomies/<your-taxonomy-id>/policyTags/<your-policy-tag-id>
      • Example: org/530746474/projects/gcp-project-123/locations/us/taxonomies/123456789/policyTags/123456789, org/530746474/projects/gcp-project*/locations/*/taxonomies/*/policyTags/*

  7. Click SAVE to apply the changes.

  1. SSH to the instance where Privacera Manager is installed.

  2. Run the following command to open the .yml file to be edited.

    If you have multiple connectors, then replace instance1 with the appropriate connector instance name.

    Bash
    1
    vi ~/privacera/privacera-manager/config/custom-vars/connectors/bigquery/instance1/vars.connector.bigquery.yml

  3. Add or modify the following properties :

    YAML
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
    CONNECTOR_BIGQUERY_MANAGE_PROJECT_LIST: `<your-project-name>`
     - Example: `gcp-project-123`
CONNECTOR_BIGQUERY_MANAGE_DATASET_LIST: `<your-project-name.your-dataset-name>`
     - Example: `gcp-project-123.test_db`
CONNECTOR_BIGQUERY_MANAGE_TABLE_LIST: `<your-project-name.your-dataset-name.your-table-name>`
     - Example: `gcp-project-123.analytics_db.customer_table, gcp-project-123.analytics_db.finance_*`
CONNECTOR_BIGQUERY_MANAGE_POLICY_TAG_LIST: `org/<your-org-id>/projects/<your-project-name>/locations/<your-location>/taxonomies/<your-taxonomy-id>/policyTags/<your-policy-tag-id>`
     - Example: `org/530746474/projects/gcp-project-123/locations/us/taxonomies/123456789/policyTags/123456789, org/530746474/projects/gcp-project*/locations/*/taxonomies/*/policyTags/*`

CONNECTOR_BIGQUERY_IGNORE_PROJECT_LIST: `<your-project-name>`
     - Example: `gcp-project-123`
CONNECTOR_BIGQUERY_IGNORE_DATASET_LIST: `<your-project-name.your-dataset-name>`
     - Example: `gcp-project-123.test_db`
CONNECTOR_BIGQUERY_IGNORE_TABLE_LIST: `<your-project-name.your-dataset-name.your-table-name>`
     - Example: `gcp-project-123.analytics_db.customer_table, gcp-project-123.analytics_db.finance_*`
CONNECTOR_BIGQUERY_IGNORE_POLICY_TAG_LIST: `org/<your-org-id>/projects/<your-project-name>/locations/<your-location>/taxonomies/<your-taxonomy-id>/policyTags/<your-policy-tag-id>`
     - Example: `org/530746474/projects/gcp-project-123/locations/us/taxonomies/123456789/policyTags/123456789, org/530746474/projects/gcp-project*/locations/*/taxonomies/*/policyTags/*`

  4. Once the properties are configured, run the following commands to update your Privacera Manager platform instance:

    Step 1 - Setup which generates the helm charts. This step usually takes few minutes.

    Bash
    1
2
    cd ~/privacera/privacera-manager
./privacera-manager.sh setup
    Step 2 - Apply the Privacera Manager helm charts.
    Bash
    1
2
    cd ~/privacera/privacera-manager
./pm_with_helm.sh upgrade
    Step 3 - Post-installation step which generates Plugin tar ball, updates Route 53 DNS and so on.

    Bash
    1
2
    cd ~/privacera/privacera-manager
./privacera-manager.sh post-install

  1. In PrivaceraCloud, go to Settings -> Applications.

  2. Select BigQuery from the list of Connected Applications.

  3. Click on the application name or the icon, then click on Access Management → ADVANCED tab.

  4. For including resources, enter the following values in the respective fields:

    • Projects to manage access control policies : <your-project-name>
      • Example: gcp-project-123
    • Datasets to Set Access Control Policies : <your-project-name.dataset-name>
      • Example: gcp-project-123.test_db
    • Tables to Set Access Control Policies : <your-project-name.dataset-name.table-name>
      • Example: gcp-project-123.analytics_db.test_table

  5. For including policy tags resources, enter the following value in the respective field under ADVANCED tab:

    • Policy Tags to Set Access Control Policies : org/<your-org-id>/projects/<your-project-name>/locations/<your-location>/taxonomies/<your-taxonomy-id>/policyTags/<your-policy-tag-id>
      • Example: org/530746474/projects/gcp-project-123/locations/us/taxonomies/123456789/policyTags/123456789

  6. For excluding resources, enter the following values in the respective fields:

    • Projects to Ignore While Setting Access Control Policies : <your-project-name>
      • Example: gcp-project-123
    • Datasets to Ignore While Setting Access Control Policies : <your-project-name.dataset-name>
      • Example: gcp-project-123.admin_database
    • Tables to Ignore While Setting Access Control Policies : <your-project-name.dataset-name.table-name>
      • Example: gcp-project-123.admin-database.*
    • Policy Tags to Ignore While Setting Access Control Policies : org/<your-org-id>/projects/<your-project-name>/locations/<your-location>/taxonomies/<your-taxonomy-id>/policyTags/<your-policy-tag-id>
      • Example: org/530746474/projects/gcp-project-123/locations/us/taxonomies/123456789/policyTags/123456789

  7. Click SAVE.

  8. Once saved and enabled, the BigQuery connector will start. Then you can hover on the VIEW LOGS button to check the status, either Running or Stopped.

Note

Perform the following steps only if the connector does not reflect the updated configuration and requires a restart.

Restart the BigQuery Connector:

  1. Go to Settings > Applications > select the BigQuery connector application .

  2. Edit the application > Disable it > and Save it.

  3. Open the same application again and then: Enable it and Save it.