Advanced Configuration for AWS S3¶

To configure JWT, refer to the JWT Auth Configuration section in the Databricks Clusters - FGAC connector.

1. In PrivaceraCloud, navigate to Settings -> Applications.
2. On the Connected Applications screen, select S3.
3. On the screen, click the edit icon, click on Access Management -> ADVANCED tab.
4. Add the following properties:

   Properties

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

   # static jwt configuration privacera.jwt.oauth.enable=true privacera.jwt.0.token.issuer=https://example.com/issuer privacera.jwt.0.token.publickey=<PUBLIC KEY> privacera.jwt.0.token.userKey=client_id privacera.jwt.0.token.groupKey=scope privacera.jwt.0.token.parserType=PING_IDENTITY # dynamic jwt configuration privacera.jwt.1.token.parserType=PING_IDENTITY privacera.jwt.1.token.userKey=client_id privacera.jwt.1.token.groupKey=scope privacera.jwt.1.token.issuer=https://example.com/issuer privacera.jwt.1.token.publickey.provider.url=<public_key_provider_url> privacera.jwt.1.token.publickey.provider.auth.type=basic privacera.jwt.1.token.publickey.provider.auth.username=your_username privacera.jwt.1.token.publickey.provider.auth.password=your_password privacera.jwt.1.token.publickey.provider.response.key=x5c privacera.jwt.1.token.publickey.provider.key.id=kid

5. Click the Test Connection button to verify the connection.
6. Once the connection is successful, click Save.

1. SSH to the instance where Privacera Manager is installed.
2. Open the vars.dataserver.aws.yml file in an editor:

   Bash
   1	vi config/custom-vars/vars.dataserver.aws.yml

3. Modify the following properties:

   YAML

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

   DATASERVER_AWS_MULTI_ACCOUNT_PROFILE_ENABLE: "true" DATASERVER_AWS_PROFILE_NAME_DEFAULT: "aws_account1" DATASERVER_AWS_PROFILE_NAMES: "aws_account1, aws_account2" DATASERVER_AWS_PROFILE_PROPERTIES: - PROFILE_NAME: "aws_account1" PROFILE_PROPERTIES: - AUTHN_METHOD: "CUSTOM_IAM" - IAM_ARN: "arn:aws:iam::<account_id>:role/<role_name>" - REGION: "us-east-1" - PROFILE_NAME: "aws_account2" PROFILE_PROPERTIES: - AUTHN_METHOD: "CUSTOM_IAM" - IAM_ARN: "arn:aws:iam::<account_id>:role/<role_name>" - IAM_ROLE_EXTERNAL_ID: "test1234" - REGION: "us-east-1"

4. Once the properties are configured, refer to the Privacera Manager Quickstart.

To configure AWS S3 multiple account support in PrivaceraCloud, follow the Setup for AWS S3 steps for each account.

1. To enable access to Requester Pays buckets, include the following property in the vars.dataserver.aws.yml file:

   YAML
   1 2 3 4	DATASERVER_AWS_PROFILE_PROPERTIES: - PROFILE_NAME: "aws_account1" PROFILE_PROPERTIES: - S3_REQUESTER_PAYS_ENABLE: "true"

2. Once the properties are configured, refer to the Privacera Manager Quickstart.

1. In PrivaceraCloud, navigate to Settings -> Applications.
2. On the Connected Applications screen, select S3.
3. Click the edit icon next to the Account Name, then go to Access Management -> ADVANCED tab.
4. Add the following property with the appropriate value for <your_profile_name>:

   Properties
   1	dataserver.aws.<your_profile_name>.s3.requester-pays.enable=true

5. Click the Test Connection button to verify the connection.
6. Once the connection is successful, click Save.

The AUTHN_METHOD property determines which additional parameters are required for authentication. There are three different ways to configure the authentication method. To set it up, include the following property in the vars.dataserver.aws.yml file:

1. ACCESS KEY:
   • If AUTHN_METHOD is set to ACCESS_KEY, both ACCESS_KEY and SECRET_KEY are mandatory and should be non-null values.

     YAML
     1 2 3 4 5 6	DATASERVER_AWS_PROFILE_PROPERTIES: - PROFILE_NAME: "aws_account1" PROFILE_PROPERTIES: - AUTHN_METHOD: "ACCESS_KEY" - ACCESS_KEY: "your-access-key" - SECRET_KEY: "your-secret-key"

2. CUSTOM IAM:
   • If AUTHN_METHOD is set to CUSTOM_IAM, IAM_ARN is mandatory and should be a non-null value.

     YAML
     1 2 3 4 5	DATASERVER_AWS_PROFILE_PROPERTIES: - PROFILE_NAME: "aws_account1" PROFILE_PROPERTIES: - AUTHN_METHOD: "CUSTOM_IAM" - IAM_ARN: "arn:aws:iam::<account_id>:role/<role_name>"

3. INSTANCE IAM:
   • If AUTHN_METHOD is set to INSTANCE_IAM, no additional parameters are required.

     YAML
     1 2 3 4	DATASERVER_AWS_PROFILE_PROPERTIES: - PROFILE_NAME: "aws_account1" PROFILE_PROPERTIES: - AUTHN_METHOD: "INSTANCE_IAM"

4. Once the properties are configured, refer to the Privacera Manager Quickstart.

There are two different ways to configure the authentication method in PrivaceraCloud. Follow the steps below:

1. ACCESS KEY and SECRET KEY:

   • In PrivaceraCloud, navigate to Settings -> Applications.
   • On the Connected Applications screen, select S3.
   • Click the edit icon next to the Account Name, then go to Access Management -> BASIC tab.
   • Disable the Use IAM Role toggle button.
   • In the AWS Access Key field, enter your_access_key. In the AWS Secret Key field, enter your_secret_key.
   • Click the Test Connection button to verify the connection.
   • Once the connection is successful, click Save.

2. IAM ROLE:

   • In PrivaceraCloud, navigate to Settings -> Applications.
   • On the Connected Applications screen, select S3.
   • Click the edit icon next to the Account Name, then go to Access Management -> BASIC tab.
   • Enable the Use IAM Role toggle button.
   • In the AWS IAM Role field, enter arn:aws:iam::<account_id>:role/<role_name>.
   • Click the Test Connection button to verify the connection.
   • Once the connection is successful, click Save.

1. To enable Encryption for AWS S3 bucket, include the following property in the vars.dataserver.aws.yml file:

   YAML
   1 2 3 4 5	DATASERVER_AWS_PROFILE_PROPERTIES: - PROFILE_NAME: "aws_account1" PROFILE_PROPERTIES: - S3_ENCRYPTION_ENABLE: "true" - S3_ENCRYPTION_BUCKET_DATA: "S3_ENCRYPTION_BUCKET_NAME|S3_ENCRYPTION_SSETYPE|S3_ENCRYPTION_SSEKEY"

2. There are three parts to the S3_ENCRYPTION_BUCKET_DATA property:
   1. S3_ENCRYPTION_BUCKET_NAME: Comma-separated list of S3 bucket names.
   2. S3_ENCRYPTION_SSETYPE: The type of server-side encryption to use for the S3 bucket. The supported values are SSE-C, SSE-KMS, and SSE-S3.
   3. S3_ENCRYPTION_SSEKEY: The server-side encryption key to use for the S3 bucket. This is optional based on the SSE_TYPE value:
      • For SSE_TYPE SSE-C SSE_KEY is mandatory, for SSE_TYPE SSE-KMS SSE_KEY is optional and for SSE_TYPE SSE-S3 there is no SSE_KEY.
3. To enable Encryption for Multiple S3 Buckets:
   • Use the S3_ENCRYPTION_BUCKET_DATA property to specify encryption details for multiple S3 buckets. The format is as follows:

     YAML
     1	S3_ENCRYPTION_BUCKET_DATA: "S3_ENCRYPTION_BUCKET_NAMES|S3_ENCRYPTION_SSETYPE|S3_ENCRYPTION_SSEKEY"

   • For example:

     YAML
     1	S3_ENCRYPTION_BUCKET_DATA: "bucket1,bucket2|SSE-C|jkabcxiuahsc"

4. Once the properties are configured, refer to the Privacera Manager Quickstart.

1. In PrivaceraCloud, navigate to Settings -> Applications.
2. On the Connected Applications screen, select S3.
3. Click the edit icon next to the Account Name, then go to Access Management -> ADVANCED tab.
4. Add the following property with the appropriate value for <your_profile_name>, <your_s3_bucket_name>, <sse_type>, and <sse_key>:

   Properties
   1 2 3 4	dataserver.aws.<your_profile_name>.s3.encryption.enable=true dataserver.aws.<your_profile_name>.s3.encryption.bucketname=<your_s3_bucket_name> dataserver.aws.<your_profile_name>.s3.encryption.ssetype=<sse_type> dataserver.aws.<your_profile_name>.s3.encryption.ssekey=<sse_key>

   1. S3_ENCRYPTION_BUCKET_NAME: Comma-separated list of S3 bucket names.
   2. S3_ENCRYPTION_SSETYPE: The type of server-side encryption to use for the S3 bucket. The supported values are SSE-C, SSE-KMS, and SSE-S3.
   3. S3_ENCRYPTION_SSEKEY: The server-side encryption key to use for the S3 bucket. This is optional based on the SSE_TYPE value:
      • For SSE_TYPE SSE-C SSE_KEY is mandatory, for SSE_TYPE SSE-KMS SSE_KEY is optional and for SSE_TYPE SSE-S3 there is no SSE_KEY.
5. Click the Test Connection button to verify the connection.
6. Once the connection is successful, click Save.

1. To configure the Databricks OLAC service principal, include the following property in the vars.dataserver.aws.yml file:

   YAML
   1	DATASERVER_DBX_OLAC_USE_DISPLAY_NAME: "true"

2. Once the properties are configured, refer to the Privacera Manager Quickstart.

1. In PrivaceraCloud, navigate to Settings -> Applications.
2. On the Connected Applications screen, select S3.
3. On the screen, click the edit icon, click on Access Management -> ADVANCED tab.
4. Add the following property:

   Properties
   1	dataserver.dbx.olac.use.displayname=true

5. Click the Test Connection button to verify the connection.
6. Once the connection is successful, click Save.

1. To configure Minio, include following properties in the vars.dataserver.aws.yml file:

   YAML

   1 2 3 4 5 6 7 8 9 10 11 12 13 14

   DATASERVER_AWS_PROFILE_PROPERTIES: - PROFILE_NAME: "aws_account1" PROFILE_PROPERTIES: - # enable to connect to a custom S3 endpoint such as, MINIO instead of AWS S3 - CUSTOM_S3_ENDPOINT_ENABLE: "true" - # set the S3 endpoint host to connect to instead of AWS S3 - CUSTOM_S3_ENDPOINT_HOST: "minio.example.com" - # set the S3 endpoint port to connect to instead of AWS S3 - CUSTOM_S3_ENDPOINT_PORT: "443" - # to enable S3 endpoint SSL - CUSTOM_S3_ENDPOINT_SSL: "true"

2. Once the properties are configured, refer to the Privacera Manager Quickstart.

1. In PrivaceraCloud, navigate to Settings -> Applications.
2. On the Connected Applications screen, select S3.
3. On the screen, click the edit icon, click on Access Management -> ADVANCED tab.
4. Add the following properties with the appropriate value for <your_profile_name>, <minio_host>, and <minio_port>:

   Properties
   1 2 3 4	dataserver.aws.<your_profile_name>.custom.s3.endpoint.enable=true dataserver.aws.<your_profile_name>.custom.s3.endpoint.host=<minio_host> dataserver.aws.<your_profile_name>.custom.s3.endpoint.port=<minio_port> dataserver.aws.<your_profile_name>.custom.s3.endpoint.ssl=true

5. Click the Test Connection button to verify the connection.
6. Once the connection is successful, click Save.

1. To configure the AWS region and endpoint, include the following properties in the vars.dataserver.aws.yml file with appropriate value for your-s3-bucket-region:

   YAML
   1 2 3 4 5 6 7 8	DATASERVER_AWS_PROFILE_PROPERTIES: - PROFILE_NAME: "aws_account1" PROFILE_PROPERTIES: - # Set the AWS region. Default is us-east-1 - REGION: "your-s3-bucket-region" - # Set true to force dataserver to use default region for all bucket - USE_REGIONAL_ENDPOINT: "true"

2. Once the properties are configured, refer to the Privacera Manager Quickstart.

1. In PrivaceraCloud, navigate to Settings -> Applications.
2. On the Connected Applications screen, select S3.
3. On the screen, click the edit icon, click on Access Management -> BASIC tab.
4. Add the AWS region for the S3 bucket in the AWS Region field.
5. Click on ADVANCED tab and add the following property with the appropriate value for <your_profile_name>:

   Properties
   1	dataserver.aws.<your_profile_name>.use.regional.endpoint=true

6. Click the Test Connection button to verify the connection.
7. Once the connection is successful, click Save.

1. To configure the AWS STS token, include the following property in the vars.dataserver.aws.yml file:

   YAML
   1 2 3 4 5	DATASERVER_AWS_PROFILE_PROPERTIES: - PROFILE_NAME: "aws_account1" PROFILE_PROPERTIES: - # Set true to enable STS token - STS_TOKEN_SIGNING_ENABLE: "true"

2. Once the properties are configured, refer to the Privacera Manager Quickstart.

1. In PrivaceraCloud, navigate to Settings -> Applications.
2. On the Connected Applications screen, select S3.
3. On the screen, click the edit icon, click on Access Management -> ADVANCED tab.
4. Add the following property with the appropriate value for <your_profile_name>:

   Properties
   1	dataserver.aws.<your_profile_name>.signer.ststoken.enable=true

5. Click the Test Connection button to verify the connection.
6. Once the connection is successful, click Save.

1. To add Privacera user info into AWS outgoing request for CloudWatch, include the following property in the vars.dataserver.aws.yml file:

   YAML
   1 2 3 4	DATASERVER_AWS_PROFILE_PROPERTIES: - PROFILE_NAME: "aws_account1" PROFILE_PROPERTIES: - REQUEST_INCLUDE_USERINFO: "true"

2. Once the properties are configured, refer to the Privacera Manager Quickstart.

1. In PrivaceraCloud, navigate to Settings -> Applications.
2. On the Connected Applications screen, select S3.
3. On the screen, click the edit icon, click on Access Management -> ADVANCED tab.
4. Add the following property with the appropriate value for <your_profile_name>:

   Properties
   1	dataserver.aws.<your_profile_name>.request.include.userinfo=true

5. Click the Test Connection button to verify the connection.
6. Once the connection is successful, click Save.

1. To configure the Ranger service name for S3, include the following property in the vars.dataserver.aws.yml file:

   YAML
   1 2 3 4	DATASERVER_AWS_PROFILE_PROPERTIES: - PROFILE_NAME: "aws_account1" PROFILE_PROPERTIES: - RANGER_SERVICE_NAME_S3: "s3_ranger_policy_repository_name"

2. Once the properties are configured, refer to the Privacera Manager Quickstart.

1. In PrivaceraCloud, navigate to Settings -> Applications.
2. On the Connected Applications screen, select S3.
3. On the screen, click the edit icon, click on Access Management -> ADVANCED tab.
4. Add the following property with the appropriate value for <your_profile_name> and <s3_ranger_policy_repository_name>:

   Properties
   1	dataserver.aws.<your_profile_name>.ranger.service.name.s3=<s3_ranger_policy_repository_name>

5. Click the Test Connection button to verify the connection.
6. Once the connection is successful, click Save.

1. To set the External ID configured in the IAM role, include the following property in the vars.dataserver.aws.yml file:

   YAML
   1 2 3 4	DATASERVER_AWS_PROFILE_PROPERTIES: - PROFILE_NAME: "aws_account1" PROFILE_PROPERTIES: - IAM_ROLE_EXTERNAL_ID: "external_id_value"

2. Once the properties are configured, refer to the Privacera Manager Quickstart.

1. In PrivaceraCloud, navigate to Settings -> Applications.
2. On the Connected Applications screen, select S3.
3. Click the edit icon next to the Account Name, then go to Access Management -> BASIC tab.
4. In the AWS IAM Role External Id field, add external_id_value.
5. Click the Test Connection button to verify the connection.
6. Once the connection is successful, click Save.

1. To set the AWS Security Token Service (STS) role, include the following property in the vars.dataserver.aws.yml file:

   YAML
   1 2 3 4	DATASERVER_AWS_PROFILE_PROPERTIES: - PROFILE_NAME: "aws_account1" PROFILE_PROPERTIES: - STS_ROLE: "arn:aws:iam::<account_id>:role/<your_STS_role>"

2. Once the properties are configured, refer to the Privacera Manager Quickstart.

1. In PrivaceraCloud, navigate to Settings -> Applications.
2. On the Connected Applications screen, select S3.
3. On the screen, click the edit icon, click on Access Management -> ADVANCED tab.
4. Add the following property with the appropriate value for <your_profile_name>, <account_id>, and <your_STS_role>:

   Properties
   1	dataserver.aws.<your_profile_name>.sts.role=arn:aws:iam::<account_id>:role/<your_STS_role>

5. Click the Test Connection button to verify the connection.
6. Once the connection is successful, click Save.