Skip to content

Discovery Configuration - Self Managed and Data Plane

Ensure that you have completed the Prerequisites before proceeding with the Discovery configuration.

Enabling Privacera Discovery

Run the following commands on the Privacera Manager host to enable Discovery features in Self-Managed and PrivaceraCloud Data Plane deployments.

Step 1: Copy the vars.discovery.aws.yml from config/sample-vars to config/custom-vars and edit the file.

Bash
1
2
3
cd ~/privacera/privacera-manager
cp -n config/sample-vars/vars.discovery.aws.yml config/custom-vars/
vi config/custom-vars/vars.discovery.aws.yml

Warning

  • Do not modify default values.
  • Only uncomment or edit the properties described in the sections below.
  • Do not change any other properties that already have default values. Changing them can affect Discovery functionality.

Step 2: Uncomment required properties

Uncomment the following properties in the configuration file to enable Discovery features:

Property Purpose
DISCOVERY_K8S_SPARK_DYNAMIC_ALLOCATION_ENABLED Enables dynamic resource allocation for Spark executors in Kubernetes.
DISCOVERY_USE_POD_IAM_ROLE Allows Discovery driver and executor pods to use IAM roles for AWS authentication.
DISCOVERY_CONSUMER_ENABLE Enables the Discovery consumer component for scanning and classification.
DISCOVERY_CONSUMER_USE_POD_IAM_ROLE Allows Discovery consumer pods to use IAM roles for AWS authentication.

Step 3: Add your configuration values

Edit the following properties and replace <PLEASE_CHANGE> with your actual values. You created these resources when you completed the Prerequisites

Property Description Example
DISCOVERY_BUCKET_NAME Name of the S3 bucket used for Discovery configuration. Do not include the s3:// prefix. You can optionally include a folder path. The bucket is created by Privacera Manager using Terraform. "my-discovery-bucket" or "my-discovery-bucket/my-path"
DISCOVERY_IAM_ROLE_ARN ARN of the IAM role for Discovery driver, executor, and Portal pods. arn:aws:iam::aws:role/AmazonS3Access
DISCOVERY_CONSUMER_IAM_ROLE_ARN ARN of the IAM role for Discovery consumer pods. arn:aws:iam::aws:role/AmazonS3Access

Step 1: Copy the vars.discovery.azure.yml from config/sample-vars to config/custom-vars and edit the file.

Bash
1
2
3
cd ~/privacera/privacera-manager  
cp -n config/sample-vars/vars.discovery.azure.yml config/custom-vars
vi config/custom-vars/vars.discovery.azure.yml

Warning

  • Do not modify default values.
  • Only uncomment or edit the properties described in the sections below.
  • Do not change any other properties that already have default values. Changing them can affect Discovery functionality

Step 2: Uncomment required properties

Uncomment the following properties in the configuration file to enable Discovery features:

Property Purpose
CREATE_AZURE_RESOURCES Set to "true" if you want Privacera Manager to create Azure resources for you; set to "false" if resources are created manually.
DISCOVERY_CONSUMER_ENABLE Enables the Discovery consumer component for scanning and classification.
DISCOVERY_COSMOSDB_SERVERLESS Set to "true" if you are using serverless CosmosDB; otherwise set to "false".

Step 3: Add your configuration values

Edit the following properties and replace <PLEASE_CHANGE> with your actual values. You created these resources when you completed the Prerequisites

Property Description
DISCOVERY_FS_PREFIX Discovery configuration container name.
DISCOVERY_AZURE_STORAGE_ACCOUNT_NAME Discovery configuration storage account name.
DISCOVERY_AZURE_STORAGE_ACCOUNT_KEY Key for discovery configuration storage account if created manually; otherwise comment it.
DISCOVERY_AZURE_LOCATION Region for discovery resources. Default is East US if Privacera Manager creates resources; otherwise use the region where resources were created manually.
DISCOVERY_AZURE_COSMOS_DB_ACCOUNT Azure CosmosDB account name (created by Privacera Manager during installation). Comment it if you created CosmosDB manually.
DISCOVERY_COSMOSDB_URL URL for Azure CosmosDB if created manually; otherwise comment it.
DISCOVERY_COSMOSDB_KEY Key for Azure CosmosDB if created manually; otherwise comment it.
DISCOVERY_AZURE_APPLICATION_ID Application ID for Azure Subscription Principal. Leave commented if using Managed Service Identity (MSI).
DISCOVERY_AZURE_TENANTID Tenant ID for Azure AD. Leave commented if using MSI.
DISCOVERY_AZURE_APP_CLIENT_SECRET_BASE64 Application Secret for Azure AD [Base64 encoded]. Leave commented if using MSI.

Step 1: Copy the vars.discovery.gcp.yml from config/sample-vars to config/custom-vars and edit the file.

Bash
1
2
3
cd ~/privacera/privacera-manager
cp -n config/sample-vars/vars.discovery.gcp.yml config/custom-vars/
vi config/custom-vars/vars.discovery.gcp.yml

Warning

  • Do not modify default values.
  • Only uncomment or edit the properties described in the sections below.
  • Do not change any other properties that already have default values. Changing them can affect Discovery functionality

Step 2: Uncomment required properties

Uncomment the following property in the configuration file to enable Discovery feature:

Property Purpose
DISCOVERY_CONSUMER_ENABLE Enables the Discovery consumer component for scanning and classification.

Step 3: Add your configuration values

Edit the following properties and replace <PLEASE_CHANGE> with your actual values. You created these resources when you completed the Prerequisites

Property Description
BIGTABLE_INSTANCE_ID Bigtable instance ID for Discovery.
DISCOVERY_BUCKET_NAME Name of the GCS bucket used for Discovery configuration. Do not include the gs:// prefix

Enable Kafka for Discovery

Kafka is required for inter process communication between the Discovery components.

Copy the vars.kafka.yml from config/sample-vars to config/custom-vars and edit the file.

Bash
1
2
3
cd ~/privacera/privacera-manager
cp -n config/sample-vars/vars.kafka.yml config/custom-vars/
vi config/custom-vars/vars.kafka.yml

Add or edit the following variables:

Bash
1
2
3
# Add or edit the following variables in the file
USE_KAFKA_SPECIFIC_STORAGE_CLASS: "true"
KAFKA_ENABLE_KRAFT_MODE: "true"

Copy the vars.kafka.yml from config/sample-vars to config/custom-vars and edit the file.

Bash
1
2
3
cd ~/privacera/privacera-manager
cp -n config/sample-vars/vars.kafka.yml config/custom-vars/
vi config/custom-vars/vars.kafka.yml

Add or edit the following variables:

Bash
1
2
3
# Add or edit the following variables in the file
USE_KAFKA_SPECIFIC_STORAGE_CLASS: "true"
KAFKA_ENABLE_KRAFT_MODE: "true"

Kafka is not required for GCP as it uses pubsub. Please refer prerequisites for GCP for more details

Apply the configuration

To apply the configuration you should run Privacera Manager to restart the services.

Bash
1
2
3
cd ~/privacera/privacera-manager
./privacera-manager.sh setup
./pm_with_helm.sh upgrade