Table Properties#
Discovery#
Expand
| Property | Description | Example |
|---|---|---|
| DISCOVERY_BUCKET_NAME | Set the bucket name where Discovery will store its metadata files | container1 |
| Properties of Topic and Table names | Topic and Table names are assigned by default in Privacera Discovery. To customize any topic or table name, refer to the link. |
Expand
| Property | Description | Example |
|---|---|---|
| DISCOVERY_ENABLE | In the Basic tab, enable/disable Privacera Discovery. | |
| DISCOVERY_REALTIME_ENABLE |
In the Basic tab, enable/disable real-time scan in Privacera Discovery. For real-time scan to work, ensure the following:
|
|
| DISCOVERY_FS_PREFIX | Enter the container name. Get it from the Prerequisites section. | container1 |
| DISCOVERY_AZURE_STORAGE_ACCOUNT_NAME | Enter the name of the Azure Storage account. Get it from the Prerequisites section. | azurestorage |
| DISCOVERY_COSMOSDB_URL DISCOVERY_COSMOSDB_KEY |
Enter the Cosmos DB URL and Primary Key. Get it from the Prerequisites section. | DISCOVERY_COSMOSDB_URL: "https://url1.documents.azure.com:443/" DISCOVERY_COSMOSDB_KEY: "xavosdocof" |
| DISCOVERY_AZURE_STORAGE_ACCOUNT_KEY | Enter the Access Key of the storage account. Get it from the Prerequisites section. | GMi0xftgifp== |
| Properties of Topic and Table names | Topic and Table names are assigned by default in Privacera Discovery. To customize any topic or table name, refer to the link. | |
| PKAFKA_EVENT_HUB | In the Advanced > Pkafka Configuration section, enter the Event Hub name. Get it from the Prerequisites section. | eventhub1 |
| PKAFKA_EVENT_HUB_NAMESPACE | In the Advanced > Pkafka Configuration section, enter the name of the Event Hub namespace. Get it from the Prerequisites section. | eventhubnamespace1 |
| PKAFKA_EVENT_HUB_CONSUMER_GROUP | In the Advanced > Pkafka Configuration section, enter the name of the Consumer Group. Get it from the Prerequisites section. | congroup1 |
| PKAFKA_EVENT_HUB_CONNECTION_STRING | In the Advanced > Pkafka Configuration section, enter the connection string. Get it from the Prerequisites section. | Endpoint=sb://eventhub1.servicebus.windows.net/; SharedAccessKeyName=RootManageSharedAccessKey; SharedAccessKey=sAmPLEP/8PytEsT= |
Spark from Databricks#
Expand
| Property | Description | Example |
|---|---|---|
| PRIVACERA_DISCOVERY_DATABRICKS_DOWNLOAD_URL | Set the Discover Databricks download URL. See your Privacera Sales Account Manager for this information. | https://s3.amazonaws.com/privacera/release/hdp314/rel_x.x.x.x |
| DATABRICKS_DRIVER_INSTANCE_TYPE |
For AWS driver's instance type can be "m5.xlarge" or "m5.2xlarge" For Azure driver's instance type can be "Standard_DS3_v2" |
m5.xlarge |
| DATABRICKS_INSTANCE_TYPE |
For AWS driver's instance type can be "m5.xlarge" or "m5.2xlarge" For Azure driver's instance type can be "Standard_DS3_v2" |
m5.xlarge |
| SETUP_DATABRICKS_JAR | ||
| USE_DATABRICKS_SPARK | ||
| DATABRICKS_ELASTIC_DISK | ||
| DATABRICKS_DISCOVERY_MANAGE_INIT_SCRIPT | Set to true if you want to create databricks init script. | false |
| DATABRICKS_DISCOVERY_WORKERS | ||
| DATABRICKS_DISCOVERY_JOB_NAME | ||
| DATABRICKS_DISCOVERY_SPARK_VERSION |
Spark version can be as follows:
|
7.3.x-scala2.12 |
| DATABRICKS_DISCOVERY_INSTANCE_PROFILE |
Property is used for the instance role, for the Databricks instance node where your discovery will be running |
arn:aws:iam::1234564835:instance-profile/privacera_databricks_cluster_iam_role |
| DISCOVERY_AWS_CLOUD_ASSUME_ROLE |
Property to grant Discovery access to AWS services to perform the scanning operation. |
true |
| DISCOVERY_AWS_CLOUD_ASSUME_ROLE_ARN |
ARN of the AWS IAM Role |
arn:aws:iam::12345671758:role/DiscoveryCrossAccAssumeRole_k |
Pkafka#
Expand
| Property | Description | Example |
|---|---|---|
| PKAFKA_EVENT_HUB | Enter the Event Hub name. Get it from the Prerequisites section above. | eventhub1 |
| PKAFKA_EVENT_HUB_NAMESPACE | Enter the name of the Event Hub namespace. Get it from the Prerequisites section above. | eventhubnamespace1 |
| PKAFKA_EVENT_HUB_CONSUMER_GROUP | Enter the name of the Consumer Group. Get it from the Prerequisites section above. | congroup1 |
| PKAFKA_EVENT_HUB_CONNECTION_STRING | Enter the connection string. Get it from the Prerequisites section above. | Endpoint=sb://eventhub1.servicebus.windows.net/; SharedAccessKeyName=RootManageSharedAccessKey; SharedAccessKey=sAmPLEP/8PytEsT= |
|
DISCOVERY_REALTIME_ENABLE
|
Add this property to enable/disable real-time scan. By default, it is set to false. Note: This is a custom property, and has to be added separately to the YAML file. For real-time scan to work, ensure the following:
|
true |