Skip to content

Native Tag Masking in BigQuery

This document provides information on how to use the native tag masking feature in BigQuery Connector.

BigQuery's native masking feature empowers users to implement data masking at the column level seamlessly. This functionality enables selective obscuring of column data for specific user groups while maintaining their access to the column itself.

Service Tags

Service Tags are read-only metadata tags that originate from external data service platforms and are automatically synchronized with Privacera. These tags cannot be created, modified, or deleted through the Privacera Portal, as they are managed exclusively at the service level.

Service Tags can be used in tag-based access control and data masking policies, allowing administrators to enforce service-specific access rules, implement consistent data protection strategies, and align access controls with the native tagging mechanisms of each integrated service.

BigQuery Native Masking Workflow. File Explorer

Google Documentation

Tag Synchronization from BigQuery to Privacera

The BigQuery Connector automatically synchronizes policy tags from BigQuery into the Privacera Portal. These tags are imported as Service Tags, which are read-only within Privacera. Service Tags are used in evaluating access control and data masking policies.

How It Works

  • The connector pulls tags from BigQuery and creates them in Privacera using the following format:
Text Only
1
org/{org-id}/projects/{project-id}/locations/us/taxonomies/{taxonomy-id}/policyTags/{tag-name}

Hierarchical Tag Example

  • If BigQuery tags are structured hierarchically, the connector flattens the hierarchy while preserving the full path of each tag. This approach ensures tag uniqueness and maintains consistent policy evaluation within Privacera.

For example, a taxonomy may be structured as:

Text Only
1
2
3
parent-tag
└── child-tag
    └── grand-child-tag

The connector will create three separate tags in Privacera as follows:

Text Only
1
2
3
org/{org-id}/projects/{project-id}/locations/us/taxonomies/{taxonomy-id}/policyTags/parent-tag
org/{org-id}/projects/{project-id}/locations/us/taxonomies/{taxonomy-id}/policyTags/parent-tag/child-tag
org/{org-id}/projects/{project-id}/locations/us/taxonomies/{taxonomy-id}/policyTags/parent-tag/child-tag/grand-child-tag

This flattened structure preserves the original tag hierarchy while ensuring compatibility with Privacera’s Service Tag model.

📖 Reference: Working with Tag Hierarchies in BigQuery

Viewing Synced Service Tags from BigQuery

You can view all service tags synchronized from BigQuery to Privacera by following these steps:

  1. Navigate to Access Management → Tag Management
  2. Select the SERVICE TAG tab to view all tags retrieved from BigQuery and synchronized by the connector.

This section allows users to verify that tags from BigQuery, including hierarchical tags, are properly imported and available for policy creation in Privacera.

Prerequisites

  1. Taxonomy and Policy Tag Creation

    • The BigQuery connector pulls tags directly from Google BigQuery, where all tags are organized within a taxonomy. Therefore, creating a taxonomy along with corresponding policy tags is a mandatory prerequisite for using this feature.
    • Follow the links below for detailed steps to create them manually in BigQuery:
      Create Policy Tag and Taxonomy in BigQuery

  2. Policy Tag-to-Column Association in BigQuery

    • The BigQuery connector enforces tag-based access and masking policies using policy tags defined in BigQuery, together with the policies configured in Privacera.
    • To ensure these policies take effect for end-users, the corresponding policy tag must be associated with the relevant column in BigQuery.
      Note: Before creating tag-based access or masking policies in Privacera, ensure the policy tag is properly associated with the target column in BigQuery.

  3. Organization id is Mandatory

    • The organization ID is a mandatory field for the connector to function correctly for tag based masking. It is used to identify the organization in which policy tags are created.
    • Ensure that the organization ID is accurately configured in the connector settings to avoid any issues with tag synchronization and policy enforcement.
    • The Following property should not be empty or null.
      • For Portal GCP organization id
      • For PM Configuration
        Bash
        1
           CONNECTOR_BIGQUERY_ORGANIZATION_ID: 123456 `<pls replace with your organization id>`

  4. Connector Configuration
    To enable this feature, ensure the connector is properly configured using the advanced configuration .
    Follow the configuration guide here: Native Row Filter and Tag Masking Setup

  5. Tag Resource Mapping For Privacera to enforce tag-based access and masking policies, the customer must manually associate policy tags with the appropriate columns in BigQuery.

  6. Custom Masking: To create a masking policy using a custom routine (UDF), the routine must already exist in the project where the masking policy is to be applied. Additionally, users are required to provide the routine path.

The Path of routine must follow following format:
Text Only
1
projects/<project-id>/datasets/<dataset-name>/routines/<routine-Name>

Key Considerations:

  • The routine must be present within the same project where the masking policy is being created.
  • The path format should be strictly followed to ensure proper integration of the routine with the masking policy.
    • Ensure that <project-id> is accurate.
    • Ensure that <dataset-name> is valid in the project.
    • Ensure that <routine-Name> matches exactly with the routine created in the dataset.
  • If the provided routine path is not supported or the routine is not found, the connector will throw an error and create the masking policy in BigQuery with the default masking.

Use Cases

Applying Tag-Based Masking Policy for End User

This guide outlines the steps to create a tag-based masking policy in Privacera for end users, including applying access and masking policies.

📌 Assumption
Before proceeding, ensure you have completed the steps listed in the Prerequisites section and that the following setup exists in BigQuery:

  • A policy tag test_tag is created under the taxonomy test_Taxonomy with taxonomy ID 12345678.
  • The tag exists in project test_project-12345 under organization ID 123123.
  • The policy tag test_tag is associated with the SSN column of the test_table in BigQuery.
  • Security Zone Consideration: If you plan to control permissions using Security Zones, note that the Security Zone should be configured only at the project level (i.e., test_project-12345) to ensure consistent enforcement across all tagged resources within the project.

Step 1: Create Resource-Based Policies in Privacera

  1. Create Resource-Based Policy for Job Creation Access:
    • Policy Type: Resource-Based Policy
    • Resource: Test_Project
    • Permission: Job Create
  2. Create Resource-Based Policy for Query Permission on Table:
    • Policy Type: Resource-Based Policy:
    • Resource: Test_Table
    • Permission: Query

Step 2: Create Tag-Based Masking Policy in Privacera

  1. Navigate to the tag Masking policy creation page
  2. Locate and enable the Service Based Tags toggle switch
  3. Once enabled, three dropdown menus will appear in sequence:
    • Service Type: Select the type of service as Google BigQuery)
    • Service: Choose the specific service instance by default its 'privacera_bigquery'
    • Tag Type: Select the type of tag as Policy Tag.
  4. Add service tag ,users and masking option
    • Tag: org/123123/projects/test_project-12345/locations/us/taxonomies/12345678/policyTags/test_tag
    • Users: test_user1
    • Component: bigquery
    • Masking Option: Nullify
  5. For more details please refer Tag policy Creation

This policy masks the SSN column in the test_table for test_user1 by replacing its values with NULL. This ensures sensitive data remains protected for the specified user.

Step 3: Create Tag-Based Access Policy in Privacera

  1. Navigate to the tag Access policy creation page
  2. Locate and enable the Service Based Tags toggle switch
  3. Once enabled, three dropdown menus will appear in sequence:
    • Service Type: Select the type of service as Google BigQuery)
    • Service: Choose the specific service instance by default its 'privacera_bigquery'
    • Tag Type: Select the type of tag as Policy Tag.
  4. Add service tag ,users and masking option
    • Tag: org/123123/projects/test_project-12345/locations/us/taxonomies/12345678/policyTags/test_tag
    • Users: test_user2
    • Component: bigquery
    • Permission: Query
  5. For more details please refer Tag policy Creation

This policy will grant test_user2 access to query the data tagged with forward_camera_186122_priv_tag_id. Since test_user2 has the appropriate access policy, they will be able to see the unmasked data, regardless of any masking policies applied for other users.

Note: The access granted to test_user2 will bypass any masking policies for that specific tag, allowing full visibility of the data.

⚠ Limitations

  1. BigQuery Column-Level Security Behavior: In BigQuery, if an end user needs access to a column that has a tag attached, they must either be included in the tag-based masking policy or have permissions through a tag-based access policy. This is a key requirement for managing column-level security with tags in BigQuery.
    Reference: BigQuery Column-Level Security

  2. Location Supported in Connector’s Configuration:

    • BigQuery resources and taxonomies can be created across various locations. However, the PolicySync connector currently supports access control management for resources within a single location.
    • The connector does not support the management of resources across multiple locations concurrently.

Comments