Skip to content

Prerequisites for Discovery on AZURE

Note

The prerequisites for Privacera Discovery are the same for both Self-Managed and PrivaceraCloud Data Plane deployments.

The Privacera Discovery module leverages Azure services such as ADLS, Event Hub, and Cosmos DB for data scanning. Privacera Manager can create these resources for you, or you can create them manually. During installation, you will need to provide the details of the manually created resources in the vars.discovery.azure.yml file.

Here are the prerequisites for setting up Privacera Discovery on AZURE:

Prerequisites Description
Create an Azure Storage Account and Container A storage account and container where the configurations and temporary files for Discovery are stored.
Create an Azure Cosmos DB Used to store metadata and tags..
Create an Event Hub namespace, Event Hub and Consumer Group Used for real-time scanning, capturing change events for resources and used for parallel processing of events.
Create an Event Subscription Defines events route from a source to a target.
IAM for Privacera Manager IAM Role to create Azure resources required by Privacera Discovery.

Create an Azure Storage Account and Container

A dedicated Azure storage account and container are required to store the configuration for Privacera Discovery. Privacera Manager can create the storage account and container for you during setup, or you can create them manually and provide the storage account name, container name, and access key to Privacera Manager during installation. This container can be used to store configurations and temporary files for Discovery. Azure provides an access key for each storage account, which is used for authentication and to access resources such as Blob Storage, File Storage, and other Azure services.

Tip

  1. Provide value of DISCOVERY_AZURE_STORAGE_ACCOUNT_NAME and DISCOVERY_FS_PREFIX if you want privacera manager to create it for you during installation or if you have created it manually then provide that values in vars.discovery.azure.yml file.

  2. Provide value of DISCOVERY_AZURE_STORAGE_ACCOUNT_KEY if you created storage account manually else comment it in .yml file.

Example :

DISCOVERY_AZURE_STORAGE_ACCOUNT_NAME: "discoverystorage"

DISCOVERY_FS_PREFIX: "discoverycontainer"

DISCOVERY_AZURE_STORAGE_ACCOUNT_KEY: "XXXXXXXXXXXXXXXXXXXXXXXXXXX"

Steps for creating Storage Account
  1. Sign in to the Azure Portal.
  2. Navigate to Storage Accounts in the left sidebar.
  3. Click + Create.

  4. Enter the Basic Configuration Details::

    Tip

    The storage account name must be globally unique, between 3-24 characters long, and contain only lowercase letters and numbers.

    • Subscription: Select your Azure subscription.
    • Resource Group: Choose an existing resource group or create a new one.
    • Storage Account Name: Provide a globally unique name.
    • Region: Select the region where the storage account will be hosted.
    • Performance: Choose Standard or Premium based on the use case.
    • Redundancy: Select LRS (Locally Redundant Storage) or GRS (Geo-Redundant Storage).

  5. Click Review + Create and then Create to deploy the storage account.

Steps to create a Blob Container in Storage Account
  1. Open the Azure Portal and navigate to your newly created Storage Account (e.g., discoverystorage).
  2. Under Data Storage, click + Containers.
  3. Enter a Container Name (e.g., discoverycontainer).
  4. Set Anonymous Access Level to Private (no anonymous access) for security.
  5. Click Create to finalize the container setup.
Steps to view and fetch access key
  1. Sign in to the Azure Portal.
  2. Navigate to your Storage Account (e.g., discoverystorage).
  3. In the left-hand menu, locate the Security + Networking section.
  4. Click on Access Keys.
  5. In the Access Keys page, two keys will be displayed.
  6. Copy the first key from the Value field for authentication purposes.
Reference links

For more detailed information, you can refer to Azure documentation from the links given below:

Create an Azure storage account

Create a container

View account access keys

Create an Azure Cosmos DB

Azure Cosmos DB is required to store metadata for Privacera Discovery. The recommended naming convention for these tables is privacera_*_<DEPLOYMENT_ENV_NAME>. Privacera Manager can create a Cosmos DB instance with the recommended format for you during installation, or you can create the Cosmos DB instance manually, and Privacera Discovery will create the necessary tables during the Discovery installation.

Steps for creating Azure Cosmos DB
  1. Sign in to the Azure Portal.
  2. Navigate to Azure Cosmos DB in the left sidebar.
  3. Click on + Create.
  4. Select an API based on your use case:
    • Core (SQL API) – For NoSQL document-based databases (Recommended).
  5. Choose the appropriate Subscription.
  6. Select an existing Resource Group or create a new one.
  7. Enter a unique Account Name for Cosmos DB.
  8. Select the nearest Azure Region for optimal performance.
  9. Choose the Capacity Mode:
    • Provisioned Throughput (fixed RU/s).
    • Serverless (pay-per-request model).
  10. Configure additional settings and click Review + Create.
  11. Set the consistency level to Strong under Settings > Default Consistency.
Reference link

For more detailed information, you can refer to Azure documentation from the links given below:

Create an Azure Cosmos DB

After creating Azure Cosmos DB manually, provide the following values in the vars.discovery.azure.yml file during the Discovery installation. You can retrieve the URI and Key from Settings > Keys of the newly created Azure Cosmos DB instance.

Tip

Provide the values for DISCOVERY_COSMOSDB_URL and DISCOVERY_COSMOSDB_KEY if you have created the Azure Cosmos DB manually. Otherwise, if you want Privacera Manager to create it for you, keep the entries commented in the vars.discovery.azure.yml file.

DISCOVERY_COSMOSDB_URL: "https://XXXXXX-cosmosdb-XXX.documents.azure.com:443/" DISCOVERY_COSMOSDB_KEY: "6mTUMOKDdmXXXXXCCCCCCnzBvnFIXUNP8RzBlIqakw=="

Important

If you want to setup serverless Azure Cosmos DB, add the following property in the vars.discovery.azure.yml file: DISCOVERY_COSMOSDB_SERVERLESS: "true"

Azure Cosmos DB is required to store metadata for Privacera Discovery. During installation, Privacera Discovery will create the necessary Azure Cosmos DB tables. If creating the tables manually, follow the recommended naming convention by replacing DEPLOYMENT_ENV_NAME with your actual deployment environment name.

Tip

It is recommended to suffix table names with the DEPLOYMENT_ENV_NAME (e.g., privacera-prod) to avoid conflicts with other deployments.

Table Names
privacera_scan_request_<DEPLOYMENT_ENV_NAME>
privacera_resource_v2_<DEPLOYMENT_ENV_NAME>
privacera_alert_<DEPLOYMENT_ENV_NAME>
privacera_audit_summary_<DEPLOYMENT_ENV_NAME>
privacera_lineage_<DEPLOYMENT_ENV_NAME>
privacera_scan_status_<DEPLOYMENT_ENV_NAME>

Create an Event Hub Namespace, Event Hub, Connection String and Consumer Group (Optional)

Note

This is only required if we want to enable real-time scanning in Privacera Discovery.

An Azure Event Hub is set up to receive change events (such as object creation, deletion, or modification) from ADLS via Event Grid. Discovery pods listen to the Azure Event Hub for incoming change events. Each event message contains metadata about the changed object, which is then processed by Discovery. By integrating Event Hub with ADLS, Privacera ensures continuous, automated detection and protection of sensitive data in near real-time.

Steps to create an Event Hub Namespace
  1. Sign in to the Azure Portal.
  2. In the left sidebar, navigate to Event Hubs.
  3. Click + Create.
  4. Configure the following details:
    • Subscription: Select your Azure subscription.
    • Resource Group: Choose an existing resource group or create a new one.
    • Namespace Name: Enter a globally unique name (e.g., discovery-eventhub-namespace).
    • Location: Select the nearest Azure region.
    • Pricing Tier: Choose Basic, Standard, or Dedicated, based on requirements.
  5. Click Review + Create, then click Create.
Steps to create an Event Hub
  1. Open the Event Hub Namespace that you just created.
  2. Click + Event Hub to create a new Event Hub.
  3. Configure the following settings:
    • Name: Enter a unique name that can include letters, numbers, periods (.), hyphens (-), and underscores (_), up to 256 characters. The name must begin and end with a letter or number (e.g., discovery-eventhub).
    • Cleanup Policy: Default is Delete.
    • Retention Time (hours): Default is 1 (increase if longer retention is needed).
  4. Click Review + Create, then click Create.
Steps to retrieve the Connection String
  1. Sign in to the Azure Portal.
  2. Search for and select Event Hubs.
  3. Choose the namespace where your Event Hub is located.
  4. Under the left menu, click Shared access policies.
  5. Click on an existing policy (e.g., RootManageSharedAccessKey) or create a new one with required permissions.
  6. Under the selected policy, copy the Primary Connection String and use it to authenticate and connect to the Event Hub.

Note

  • The Primary Connection String contains credentials, so store it securely.
  • If needed, create a new access policy with restricted permissions instead of using RootManageSharedAccessKey for security best practices.
Steps to create a Consumer Group
  1. Sign in to the Azure Portal.
  2. Navigate to Event Hubs in the left sidebar.
  3. Click on the Event Hub Namespace where your Event Hub is located.
  4. Under Entities, select your Event Hub.
  5. In the Event Hub settings page, go to Consumer Groups in the left menu.
  6. You can see $Default consumer group which is automatically created by Event Hub by default.
  7. Click + Consumer Group and you can enter a unique name for the consumer group (e.g., discovery-consumer-group) if you want to create new consumer group instead of default.
  8. Click Create.
Reference links

For more detailed information, you can refer to Azure documentation from the links given below:

Create an Event Hub namespace

Create an Event Hub

Consumer Groups - Create Or Update

Create an Event Subscription (Optional)

Note

This is only required if we want to enable real-time scanning in Privacera Discovery.

This enables Privacera Discovery to listen to those events and support automatic, real-time scanning of resources such as files in the Azure storage account.

Steps for creating an Event Subscription
  1. Sign in to the Azure Portal.
  2. Navigate to Storage Accounts in the left sidebar.
  3. Select your Storage Account which you wish to monitor for real time events and click on Events.
  4. Click on + Event Subscription.

  5. Enter the following Event Subscription details:

    a. Subscription Name: Provide a unique name for the event subscription, e.g., discovery-event-sub.

    b. Event Schema: Choose Event Grid Schema (default).

  6. The following Topic Details will be auto-populated based on the selected storage account:

    a. Topic Type: Storage Account.

    b. Source Resource: Selected Storage Account name.

    c. System Topic Name: Enter a new name or select an existing one.

  7. Event Types → Select specific event types to filter (or choose All). It is recommended to select Blob Created and Blob Deleted.

  8. Endpoint Type → Select how events will be processed. It is recommended to choose Event Hub.

  9. Configure the Endpoint with the following details (use the same values as during configuration):

    a. Subscription

    b. Resource Group

    c. Event Hub Namespace

    d. Event Hub

  10. Review the settings and click Create.

Important

When creating an Event Grid Subscription, ensure that the checkbox Enable subject filtering under the Filters tab in Subject Filters is cleared.

Reference link

For more detailed information, you can refer to Azure documentation from the links given below:

Subscribe to the Blob storage

IAM for Privacera Manager (Optional)

Using MSI (Managed Service Identity)

Steps to provide Cosmos DB access
  1. Sign in to the Azure Portal.
  2. Navigate to the Subscription.
  3. In the left sidebar, click on Access Control (IAM).
  4. Click on + Add > Add role assignment.
  5. Select Cosmos DB Account Contributor as the role.
  6. Under Assign access to, select User, group, or service principal.
  7. In the Select box, search for the Managed Identity of your VM.
  8. Click Save to assign the role.
Steps to provide Storage Account access
  1. Sign in to the Azure Portal.
  2. Navigate to the Subscription.
  3. In the left sidebar, click on Access Control (IAM).
  4. Click on + Add > Add role assignment.
  5. Select Storage Account Contributor as the role.
  6. Under Assign access to, select User, group, or service principal.
  7. In the Select box, search for the Managed Identity of your VM.
  8. Click Save to assign the role.

Using Azure Subscription Principal

Steps for creating Enterprise Application
  1. Sign in to the Azure Portal.
  2. Navigate to Identity > Applications > Enterprise applications > All applications.
  3. Click on + New application at the top.
  4. Choose Create your own application to register a custom application (Service Principal).
  5. Provide a name for your new application (e.g., "discovery-service-principal").
  6. Select Integrate any other application you don’t find in the gallery.
  7. Click Create.
Steps to fetch credentials
  1. Sign in to the Azure Portal.
  2. Navigate to Azure Active Directory and select App registrations within the left menu.
  3. Click on All applications and search for the application you created.
  4. Click on the application
  5. Copy the Application (client) ID and Directory (tenant) ID.
  6. In the left menu, click on Certificates & secrets.
  7. Click on + New client secret.
  8. Enter a description for the secret and select the expiry duration.
  9. Click Add.
  10. Copy the value of the secret as it will not be displayed again.

Comments