Skip to main content

Privacera Documentation

Customize topic and table names on Privacera Platform

By default, topic and table names are assigned and managed internally by Privacera Discovery. Also, the deployment environment name is attached as a suffix to the topic and table names.

For example, the default name for a Classification Topic in Privacera Discovery is shown as below:

CLASSIFICATION_TOPIC: "privacera_classification_info_{{DEPLOYMENT_ENV_NAME}}"

To customize the name of a topic or table, you can do one of the following:

  • Remove the {{DEPLOYMENT_ENV_NAME}} variable as suffix.

  • Re-define a new topic/table name.

If you want to customize any topic or table name, refer to the property in the the following table.

  • Uncomment the topic, and enter a name, along with the {{DEPLOYMENT_ENV_NAME}} as the suffix.

  • To remove the {{DEPLOYMENT_ENV_NAME}} as the suffix, refer to the DISCOVERY_DEPLOYMENT_SUFFIX_ID property in this table.

  • {{DEPLOYMENT_ENV_NAME}} is the name of the environment you have given in the vars.privacera.yml

Property

Description

Example customization

PRIVACERA_PORTAL_TOPIC_DYNAMIC_PREFIX

Uncomment and enter a custom name to add a prefix to the real-time topic for Data Sources in Privacera Portal.

PRIVACERA_PORTAL_TOPIC_DYNAMIC_PREFIX="privacera_scan_worker"

CLASSIFICATION_TOPIC

Streams Privacera Discovery classification information generated after scanning for consumers to post-process, such as writing the data to Solr

CLASSIFICATION_TOPIC: "privacera_classification_info_{{DEPLOYMENT_ENV_NAME}}"

ALERT_TOPIC

Streams alert data which consumers to post-process, such as writing the data to Solr

ALERT_TOPIC: "privacera_alerts_{{DEPLOYMENT_ENV_NAME}}"

SPARK_EVENT_TOPIC

Streams Spark events for debugging purpose

SPARK_EVENT_TOPIC: "privacera_spark_events_{{DEPLOYMENT_ENV_NAME}}"

RESULT_TOPIC

Streams error logs consumers to post-process, such as writing the data to Solr for displaying on the Privacera Portal diagnostic page

RESULT_TOPIC: "privacera_results_{{DEPLOYMENT_ENV_NAME}}"

OFFLINE_SCAN_TOPIC

Streams batch file events after listing, which is consumed by Privacera Discovery to initiate scanning of batch file

OFFLINE_SCAN_TOPIC: "privacera_offline_scan_{{DEPLOYMENT_ENV_NAME}}"

AUDITS_TOPIC

Streams real-time audit events consumed by Privacera Discovery for real-time scanning

AUDITS_TOPIC: "privacera_audits_{{DEPLOYMENT_ENV_NAME}}"

SCAN_RESOURCE_INFO_TOPIC

Streams data for scan summary information reporting about scan request jobs

SCAN_RESOURCE_INFO_TOPIC: "privacera_scan_resources_info_{{DEPLOYMENT_ENV_NAME}}"

RIGHT_TO_PRIVACY_TOPIC

Streams events for triggering the Right to Privacy compliance policy

RIGHT_TO_PRIVACY_TOPIC: "privacera_right_to_privacy_{{DEPLOYMENT_ENV_NAME}}"

DELAY_QUEUE_TOPIC

Streams real-time events to HDFS for delayed processing

DELAY_QUEUE_TOPIC: "privacera_delay_queue_{{DEPLOYMENT_ENV_NAME}}"

APPLY_SCHEME_TOPIC

Streams events for triggering the de-identification compliance policy

APPLY_SCHEME_TOPIC: "privacera_apply_scheme_{{DEPLOYMENT_ENV_NAME}}"

ML_CLASSIFY_TAG_TOPIC

Streams events for triggering tag detection via Machine Learning Models

ML_CLASSIFY_TAG_TOPIC: "privacera_ml_classify_tag_{{DEPLOYMENT_ENV_NAME}}"

LINEAGE_TOPIC

Streams lineage information for consumers for writing the data to Solr

LINEAGE_TOPIC: "privacera_lineage_{{DEPLOYMENT_ENV_NAME}}"

RESOURCE_TABLE

ALERT_TABLE

SCAN_REQUEST_TABLE

ACTIVE_SCANS_TABLE

MLRESOURCE_TABLE

LINEAGE_TABLE

AUDIT_SUMMARY_TABLE

STATE_TABLE

SCAN_STATUS_TABLE

You can customize the table names.

Uncomment the table, and enter a name, along with the {{DEPLOYMENT_ENV_NAME}} as the suffix.

To remove the {{DEPLOYMENT_ENV_NAME}} as the suffix, refer to the DISCOVERY_DEPLOYMENT_SUFFIX_ID property in this table.

RESOURCE_TABLE: "privacera_resource_v2_{{DEPLOYMENT_ENV_NAME}}"

ALERT_TABLE: "privacera_alert_{{DEPLOYMENT_ENV_NAME}}"

SCAN_REQUEST_TABLE: "privacera_scan_request_{{DEPLOYMENT_ENV_NAME}}"

ACTIVE_SCANS_TABLE: "privacera_active_scans_{{DEPLOYMENT_ENV_NAME}}"

MLRESOURCE_TABLE: "privacera_mlresource_v2_{{DEPLOYMENT_ENV_NAME}}"

LINEAGE_TABLE:"privacera_lineage_{{DEPLOYMENT_ENV_NAME}}"

AUDIT_SUMMARY_TABLE: "privacera_audit_summary_{{DEPLOYMENT_ENV_NAME}}"

STATE_TABLE: "privacera_state_{{DEPLOYMENT_ENV_NAME}}"

SCAN_STATUS_TABLE: "privacera_scan_status_{{DEPLOYMENT_ENV_NAME}}"

DISCOVERY_DEPLOYMENT_SUFFIX_ID

Use this property to remove the {{DEPLOYMENT_ENV_NAME}} variable as suffix from the topic/table names.

Note

This is a custom property, and has to be added separately to the YAML file.

DISCOVERY_DEPLOYMENT_SUFFIX_ID: ""

DISCOVERY_BUCKET_SQS_NAME

You can customize the SQS bucket name.

Uncomment the table, and enter a name, along with the {{DISCOVERY_DEPLOYMENT_SUFFIX_ID}} as the suffix.

DISCOVERY_BUCKET_SQS_NAME: "privacera_bucket_sqs_{{DISCOVERY_DEPLOYMENT_SUFFIX_ID}}"