Multi-Connector Tag Permission Coordination¶

How this works and when you need it¶

BigQuery APIs are subject to per-project quotas and rate limits. When multiple Policy Sync connectors run against the same GCP project, each connector can issue calls to create or update tags and apply tag-based policies. If every connector performs that work, API usage stacks up and you are more likely to see throttling or quota-related errors.

This page describes an optional split of duties: one connector handles tag lifecycle and permission application for the shared project, and the other connectors only map their managed resources to those tags. Use this pattern when you operate several connectors on one project and need to stay within BigQuery API limits while keeping full coverage.

Use case¶

When running multiple BigQuery connectors within the same GCP project, you must designate one connector to manage lifecycle operations (creating/deleting tags) while the others handle resource mapping.

Connector role	Property	Behavior
Tag and permissions (one connector)	`CONNECTOR_BIGQUERY_APPLY_PERMISSIONS_TO_ALL_TAGS`: `true`	Creates and deletes tags and applies tag-based permissions (and masking) as needed.
Tag Mapping only (all other connectors on that project)	`CONNECTOR_BIGQUERY_TAG_MASKING_MAPPING_ONLY`: `true`	Maps resources to tags only for their managed resources; does not create tags or apply tag-based permissions.

Leave both properties false on a connector when you run a single BigQuery connector for the project (default behavior).

Configuration Properties¶

Important

Set CONNECTOR_BIGQUERY_APPLY_PERMISSIONS_TO_ALL_TAGS to true on exactly one connector per GCP project. Set it to false on every other connector for that project. This ensures only one instance performs tag and permission updates, which reduces BigQuery API usage. Misconfiguration can increase API usage or cause duplicate work.

Property	Description	Default
CONNECTOR_BIGQUERY_TAG_MASKING_MAPPING_ONLY	`true`: Tag Mapping only mode for that connector’s managed resources—no tag create/delete or permission application.	`"false"`
CONNECTOR_BIGQUERY_APPLY_PERMISSIONS_TO_ALL_TAGS	`true`: this connector coordinates tag create/delete and applies permissions for all relevant tags in scope.	`"false"`
CONNECTOR_BIGQUERY_MAPPING_ONLY_TAG_SYNC_WAIT_SECONDS	Extra delay (seconds) before a Tag Mapping only connector refreshes tag metadata; gives the tag-permission connector time to publish tags.	`"30"`

Restart required

Changes to these properties require a restart of the BigQuery connector.

RocksDB for the tag-permission connector

When enabling CONNECTOR_BIGQUERY_APPLY_PERMISSIONS_TO_ALL_TAGS, you must clear the RocksDB of that specific connector and restart it. This ensures the connector reloads tag data from a clean state and applies permissions using an up-to-date view of the environment.

Users, groups, and principals

The tag-permission connector must manage all users, groups, and principals that require tag access or masking. Any principal not managed by this connector will be ignored — it will not receive tag permissions or masking.

Configuration¶

Self Managed (Portal Configuration)Self Managed (YAML Configuration) and Data PlanePrivacera Cloud

Navigate to Settings → Applications in the Self-Managed Portal.
Select BigQuery from the list of Connected Applications.
Click on the application name or the icon, then click on Access Management → ADVANCED tab.

Under the Add New Custom Properties, add the following (adjust the connector index as needed):

Note

Configure each BigQuery connector instance as required. Use the Tag-permission connector properties for exactly one coordinator; use the Tag Mapping only connectors properties for the other connectors that share the same GCP project.

Tag-permission connector:

Bash
ranger.policysync.connector.0.apply.permissions.to.all.tags=true
ranger.policysync.connector.0.tag.masking.mapping.only=false

Tag Mapping only connectors:

Bash
ranger.policysync.connector.0.tag.masking.mapping.only=true
ranger.policysync.connector.0.apply.permissions.to.all.tags=false
ranger.policysync.connector.0.mapping.only.tag.sync.wait.seconds=60

Click SAVE to apply the changes.

SSH to the instance where Privacera Manager is installed.
Run the following command to open the .yml file to be edited.

If you have multiple connectors, then replace instance1 with the appropriate connector instance name.
Bash
1
vi ~/privacera/privacera-manager/config/custom-vars/connectors/bigquery/instance1/vars.connector.bigquery.yml
Add or update the following properties:

Note

Edit the vars file for each BigQuery connector instance separately. Use the Tag-permission connector block for the single coordinator instance; use the Tag Mapping only connectors block for the other instances that share the same GCP project.

Tag-permission connector:
YAML
1 2
CONNECTOR_BIGQUERY_APPLY_PERMISSIONS_TO_ALL_TAGS: "true" CONNECTOR_BIGQUERY_TAG_MASKING_MAPPING_ONLY: "false"
Tag Mapping only connectors:
YAML
1 2 3
CONNECTOR_BIGQUERY_TAG_MASKING_MAPPING_ONLY: "true" CONNECTOR_BIGQUERY_APPLY_PERMISSIONS_TO_ALL_TAGS: "false" CONNECTOR_BIGQUERY_MAPPING_ONLY_TAG_SYNC_WAIT_SECONDS: "60"
Once the properties are configured, run the following commands to update your Privacera Manager platform instance:

Step 1 - Setup which generates the helm charts. This step usually takes few minutes.
Bash
1 2
cd ~/privacera/privacera-manager ./privacera-manager.sh setup
Step 2 - Apply the Privacera Manager helm charts.
Bash
1 2
cd ~/privacera/privacera-manager ./pm_with_helm.sh upgrade
Step 3 - (Optional) Post-installation step which generates Plugin tar ball, updates Route 53 DNS and so on. This step is not required if you are updating only connector properties.
Bash
1 2
cd ~/privacera/privacera-manager ./privacera-manager.sh post-install

In PrivaceraCloud, go to Settings → Applications.
Select BigQuery from the list of Connected Applications.
Click on the application name or the icon, then click on Access Management → ADVANCED tab.

Under the Add New Custom Properties, add the following (adjust the connector index as needed):

Note

Configure each BigQuery connector instance as required. Use the Tag-permission connector properties for exactly one coordinator; use the Tag Mapping only connectors properties for the other connectors that share the same GCP project.

Tag-permission connector:

Bash
ranger.policysync.connector.0.apply.permissions.to.all.tags=true
ranger.policysync.connector.0.tag.masking.mapping.only=false

Tag Mapping only connectors:

Bash
ranger.policysync.connector.0.tag.masking.mapping.only=true
ranger.policysync.connector.0.apply.permissions.to.all.tags=false
ranger.policysync.connector.0.mapping.only.tag.sync.wait.seconds=60

Click SAVE.
Once saved and enabled, the BigQuery connector will start. You can hover on the VIEW LOGS button to check the status, either Running or Stopped.
If the connector does not reflect the updated configuration and requires a restart, restart the BigQuery connector as follows:
1. Go to Settings → Applications and select the BigQuery connector application.
2. Edit the application → Disable it → and Save it.
3. Open the same application again and then: Enable it and Save it.

Prev topic: Advanced Configuration