Skip to content

Enabling for Realtime Discovery

Discovery supports Realtime discovery to monitor and scan data in real-time. For enabling Realtime discovery, there are a few prerequisites and configurations that you need to set up.

Prerequisites

Prerequisite Description
Setting up PKakfa Service This service listens to messaging queue for audit events. The configuration for each Cloud slightly differnt and mentioned in the Setup section

Even though the service name is called PKafka, it supports multiple messaging services like AWS SQS, Azure Event Hub, and GCP Pub/Sub

Each cloud provider requires additional prerequisites and configurations. Follow the steps based on the cloud provider.

For configuring PKafka with AWS, you need to set up an Amazon SQS queue and IAM role. This steps are covered in the section for installing the base Privacera Discovery service. Refer to the Prerequisites -> AWS section.

Prerequisite Description
AWS SQS Queue Name of the AWS SQS to fetch the change events for AWS S3 and DynamoDB
AWS IAM Role ARN of the AWS IAM Role which has permissions to the SQS Queue. E.g. privacera-discovery-role-privacera-prod

Setup

Copy the vars.pkafka.aws.yml from config/sample-vars to config/custom-vars and edit the file.

Bash
1
2
3
cd ~/privacera/privacera-manager
cp -n config/sample-vars/vars.pkafka.aws.yml config/custom-vars/
vi config/custom-vars/vars.pkafka.aws.yml

Replace the following placeholders

PKAFKA_SQS_ENDPOINT: Amazon SQS Queue name URL. It would have this format, where DEPLOYMENT_ENV_NAME is the name of the deployment environment .e.g privacera-prod: https://sqs.<AWS_REGION>.amazonaws.com/<ACCOUNT_ID>/privacera_bucket_sqs_DEPLOYMENT_ENV_NAME

PKAFKA_IAM_ROLE_ARN: ARN of the IAM role created for Privacera Discovery Service. E.g. arn:aws:iam::<ACCOUNT_ID>:role/privcera-discovery-role-privacera-prod

Add or edit the following variables:

Bash
1
2
3
4
PKAFKA_SQS_ENDPOINT: "<PLEASE_CHANGE>"

PKAFKA_USE_POD_IAM_ROLE: "true"
PKAFKA_IAM_ROLE_ARN: "<PLEASE_CHANGE>"

Copy the vars.pkafka.azure.yml from config/sample-vars to config/custom-vars and edit the file.

Bash
1
2
3
cd ~/privacera/privacera-manager
cp -n config/sample-vars/vars.pkafka.azure.yml config/custom-vars/
vi config/custom-vars/vars.pkafka.azure.yml

Add or edit the following variables:

Bash
1
2
3
4
PKAFKA_EVENT_HUB: "<PLEASE_CHANGE>"
PKAFKA_EVENT_HUB_NAMESPACE: "<PLEASE_CHANGE>"
PKAFKA_EVENT_HUB_CONSUMER_GROUP: "<PLEASE_CHANGE>"
PKAFKA_EVENT_HUB_CONNECTION_STRING: "<PLEASE_CHANGE>"

Copy the vars.pkafka.gcp.yml from config/sample-vars to config/custom-vars and edit the file.

Bash
1
2
3
cd ~/privacera/privacera-manager
cp -n config/sample-vars/vars.pkafka.gcp.yml config/custom-vars/
vi config/custom-vars/vars.pkafka.gcp.yml

Add or edit the following variables:

Bash
1
PKAFKA_GCP_SINK_DESTINATION_PUBSUB_SUBSCRIPTION_NAME: "<PLEASE_CHANGE>"

Restart Privacera Services

Bash
1
2
3
cd ~/privacera/privacera-manager
./privacera-manager.sh setup
./pm_with_helm.sh upgrade

Comments