Skip to content

AWS

Overview

PrivaceraCloud Data Plane

When running Privacera Discovery in AWS, the following cloud resources are required -

TODO: Only the S3 buckets and IAM roles are required for Privacera Discovery.

Self Managed and PrivaceraCloud Data Plane

When running Privacera Discovery in Self Managed and PrivaceraCloud Data Plane deploynent mode on AWS, additional cloud resources are required such as S3 bucket, DynamoDB tables, SQS and IAM roles for EKS Service Accounts. These resources can be created by Privacera Manager if you give it the necessary permissions, otherwise you can create these resources manually and configure their ARN in Privacera Manager.

Prerequisite Description
Additional IAM policy on PM EC2 instance Additional IAM policies in the Privacera Manager EC2 host to be able to create AWS resources for Discovery such as DynamoDB tables and SQS.
AWS S3 bucket and path One bucket to store the configuration and metadata. Buckets containing data to be scanned.
AWS DynamoDB Tables To store the metadata
AWS SQS For bucket change notifications
IAM Role for Discovery and Portal pods IAM Role for Service Account(IRSA) for Discovery driver, executor, Consumer and Portal Pods to access AWS resources.

Additional IAM policy on PM EC2 instance

Additional IAM policy on PM EC2 instance

The following additional IAM policy needs to be attached to the Privacera Manager EC2 instance to be able to create AWS resources for Discovery such as DynamoDB tables, S3 and SQS.

Text Only
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
You can skip this if you do not want Privacera Manager to create these resources, but you will have to create
these resources manually and provide the ARN of these resources to Privacera Manager.

```json
     {
    "Version":"2012-10-17",
    "Statement":[
        {
            "Sid":"CreateDynamodb",
            "Effect":"Allow",
            "Action":[
                "dynamodb:CreateTable",
                "dynamodb:DescribeTable",
                "dynamodb:ListTables",
                "dynamodb:TagResource",
                "dynamodb:UntagResource",
                "dynamodb:UpdateTable",
                "dynamodb:UpdateTableReplicaAutoScaling",
                "dynamodb:UpdateTimeToLive",
                "dynamodb:DescribeTimeToLive",
                "dynamodb:ListTagsOfResource",
                "dynamodb:DescribeContinuousBackups"
            ],
            "Resource":"arn:aws:dynamodb:AWS_REGION:ACCOUNT_ID:table/privacera*"
        },
        {
            "Sid":"CreateS3Bucket",
            "Effect":"Allow",
            "Action":[
                "s3:CreateBucket",
                "s3:ListAllMyBuckets",
                "s3:GetBucketLocation"

            ],
            "Resource":[
                "arn:aws:s3:::DISCOVERY_BUCKET"
            ]
        },
        {
            "Sid":"CreateSQSMessages",
            "Effect":"Allow",
            "Action":[
                "sqs:CreateQueue",
                "sqs:ListQueues"
            ],
            "Resource":[
                "arn:aws:sqs:AWS_REGION:ACCOUNT_ID:privacera*"
            ]
        }
    ]
    }
```
!!! note "Replace the following placeholders"
    __AWS_REGION__: The AWS region where the resources are created.

    __ACCOUNT_ID__: The AWS account ID where the resources are created.

    __DISCOVERY_BUCKET__: The S3 bucket name where the Privacera meta-data is stored.

AWS S3 bucket and path

An AWS S3 bucket and path is required to store the configuration for Privacera Discovery. You can use an existing bucket and provide a path, or create a new bucket manually or let Privacera Manager create it for you. Note down the bucket name and path to be configured in Privacera Manager.

The IAM role for Privacera Discovery pods and Privacera Portal pod will need read/write access to this bucket and, they will need read access to the buckets containing data to be scanned.

AWS DynamoDB Tables

AWS DynamoDB tables are required to store the metadata for Privacera Discovery. Privacera Manager will create these for you and append the DEPLOYMENT_ENV_NAME to the table names.

If you want to create them manually, you can use the following naming convention and schema -

DynamoDB Table schema and naming convention
Override Variable Table Name Hash Key Hash Key Type Range Key Range Key Type
SCAN_REQUEST_TABLE privacera_scan_requests_DEPLOYMENT_ENV_NAME scan_id S id S
RESOURCE_TABLE privacera_resource_v2_DEPLOYMENT_ENV_NAME appCode S id S
ALERT_TABLE privacera_alert_DEPLOYMENT_ENV_NAME d S id S
AUDIT_SUMMARY_TABLE privacera_audit_summary_DEPLOYMENT_ENV_NAME appCode S id S
ACTIVE_SCANS_TABLE privacera_active_scans_DEPLOYMENT_ENV_NAME topicName S id S
STATE_TABLE privacera_state_DEPLOYMENT_ENV_NAME id S

Table Naming Convention

The table names should be suffixed with the DEPLOYMENT_ENV_NAME to avoid conflicts with other deployments.

You can create DynamoDB tables using the above schema and user your own names for the tables. In that case you will have to provide the table names to Privacera Manager.

AWS SQS

An AWS SQS queue is used for bucket change notifications. Privacera Manager will create these for you and append the DEPLOYMENT_ENV_NAME to the queue name.

If you want to create it manually then follow the following naming convention

SQS Queue naming convention
Bash
1
privacera_bucket_sqs_DEPLOYMENT_ENV_NAME

You can use a different name for the SQS queue and provide the queue name to Privacera Manager.

IAM Role for Discovery and Portal pods

IAM Role for Service Account for Discovery driver, executor, Consumer and Portal pods

Pod level IAM roles are supported since Privacera Platform version 9.0.0.1. Prior to that you had to give these IAM policies to the nodes of the Kubernetes cluster

  1. Create an IAM role named privacera_discovery_consumer_DEPLOYMENT_ENV_NAME and attach these policies to it.

    • IAM Policy to access DynamoDB tables
    • IAM policy to access S3 bucket for Discovery configuration

    Assign this policy to EKS service account discovery-consumer-privacera-sa for the Discovery consumer pod. Note down the ARN of this IAM role to be used to set value of DISCOVERY_CONSUMER_IAM_ROLE_ARN in

Text Only
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
2. Create an IAM role named `privacera_discovery_DEPLOYMENT_ENV_NAME` and attach these policies to it.

    - IAM Policy to access DynamoDB tables
    - IAM policy to access S3 bucket for Discovery configuration
    - IAM policy to access S3 bucket containing data to be scanned by Discovery (optional)
    - IAM policy to access DynamoDB tables containing data to be scanned by Discovery (optional)

    Assign this policy to EKS service account `discovery-privacera-sa` for the Discovery consumer pod,
    and to EKS service account `portal-privacera-sa` for Portal. Note down the ARN of this IAM role to be 
    used to set value of `DISCOVERY_IAM_ROLE_ARN` and `PORTAL_IAM_ROLE_ARN`
Text Only
1
2
3
4
5
6
3. Create an IAM role named `privacera_discovery_pkafka_DEPLOYMENT_ENV_NAME` and attach these policies to it.

    - IAM Policy to access SQS queue

    Assign this policy to EKS service account `pkafka-privacera-sa` for the Discovery consumer pod. 
    Note down the ARN of this IAM role to be used to set value of `PKAFKA_IAM_ROLE_ARN` in
Text Only
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
You can follow the instructions here for creating the 
[IAM role for service accounts](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html){target=_blank}.

The IAM policies are given below.

??? note "IAM Policy to access DynamoDB tables (Discovery driver, executor and Consumer)" 
    ```json
    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Sid": "Dynamodb ",
                "Effect": "Allow",
                "Action": [
                    "dynamodb:BatchGet*",
                    "dynamodb:DescribeStream",
                    "dynamodb:DescribeTable",
                    "dynamodb:Get*",
                    "dynamodb:Query",
                    "dynamodb:Scan",
                    "dynamodb:BatchWrite*",
                    "dynamodb:Update*",
                    "dynamodb:Put*"
                ],
                "Resource": "arn:aws:dynamodb:AWS_REGION:AWS_ACCOUNT_ID:table/privacera_*_DEPLOYMENT_ENV_NAME"
            }
        ]
    }
    ```

??? note "IAM policy to access S3 bucket for Discovery configuration (Discovery driver, executor and Consumer)"
    ```json
    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Sid": "S3ObjectAllpermissions",
                "Effect": "Allow",
                "Action": [
                    "s3:List*",
                    "s3:Put*",
                    "s3:Get*",
                    "s3:Delete*"
                ],
                "Resource": [
                    "arn:aws:s3:::DISCOVERY_CONFIG_BUCKET_NAME/PATH/*",
                    "arn:aws:s3:::DISCOVERY_CONFIG_BUCKET_NAME"
                ]
            },
            {
                "Sid": "S3ObjectListAllpermissions",
                "Effect": "Allow",
                "Action": [
                    "s3:ListAllMyBuckets"
                ],
                "Resource": [
                    "arn:aws:s3:::*"
                ]
            }
        ]
    }
    ```

??? note "IAM policy to access SQS queue (Discovery pKafka)"
    ```json
        {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Sid": "ManageSQSMessages",
                    "Effect": "Allow",
                    "Action": [
                        "sqs:DeleteMessage",
                        "sqs:ReceiveMessage"
                    ],
                    "Resource": [
                        "arn:aws:sqs:AWS_REGION:AWS_ACCOUNT_ID:privacera_*_DEPLOYMENT_ENV_NAME"
                    ]
                }
            ]
        }
    ```

??? note "IAM policy to scan S3 bucket containing data (Discovery driver, executor)(optional)"
    ```json
        {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Sid": "S3ObjectScanOnlypermissions",
                    "Effect": "Allow",
                    "Action": [
                        "s3:List*",
                        "s3:Put*",
                        "s3:Get*",
                        "s3:Delete*"
                    ],
                    "Resource": [
                        "arn:aws:s3:::DISCOVERY_SCAN_BUCKET_NAME1/*",
                        "arn:aws:s3:::DISCOVERY_SCAN_BUCKET_NAME1",
                        "arn:aws:s3:::DISCOVERY_SCAN_BUCKET_NAME2/*",
                        "arn:aws:s3:::DISCOVERY_SCAN_BUCKET_NAME2"
                    ]
                }
            ]
        }
    ```

??? note "IAM policy to scan DynamoDB tables containing data(optional)(Discovery driver, executor)"
    ```json
        {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Sid": "Dynamodb",
                    "Effect": "Allow",
                    "Action": [
                        "dynamodb:ListTables",
                        "dynamodb:BatchGet*",
                        "dynamodb:DescribeTable",
                        "dynamodb:Get*",
                        "dynamodb:Query",
                        "dynamodb:Scan"
                    ],
                    "Resource": "*"
                }
            ]
        }
    ```

!!! tip "Replace the following placeholders"
    __AWS_REGION__: The AWS region where the resources are created.

    __AWS_ACCOUNT_ID__: The AWS account ID where the resources are created.

    __DEPLOYMENT_ENV_NAME__: The Privacera deployment environment name.

    __DISCOVERY_CONFIG_BUCKET_NAME__: The S3 bucket name where the Privacera configuration is stored.

    __DISCOVERY_SCAN_BUCKET_NAME1__: The S3 bucket name where the data to be scanned is stored.

    __DISCOVERY_SCAN_BUCKET_NAME2__: The S3 bucket name where the data to be scanned is stored.

Comments