Skip to content

Audit Fluentd

This topic covers how you can store the audits from AuditServer locally, or on a cloud, for example, AWS S3, Azure blob, and Azure ADLS Gen 2.

Prerequisites

Ensure the following prerequisites are met:

  • AuditServer must be up and running. For more information, refer to AuditServer.

  • If you're configuring Fluentd for an Azure environment and want to configure User Managed Service Identity (MSI), assign the following two IAM roles to the Azure Storage account for the User Managed Service Identity where the audits will be stored.

    • Owner or Contributor
    • Storage Blob Data Owner or Storage Blob Data Contributor

    Note

    If your Azure environment is Docker-based, then configure MSI on a virtual machine, whereas for a Kubernetes-based environment, configure MSI on a virtual machine scale set (VMSS).

CLI Configuration

  1. SSH to the instance where Privacera is installed.

  2. Run the following commands.

    cd ~/privacera/privacera-manager
cp config/sample-vars/vars.audit-fluentd.yml config/custom-vars/
vi config/custom-vars/vars.audit-fluentd.yml

  3. Modify the properties below. For property details and description, refer to the Configuration Properties below.

    You can also add custom properties that are not included by default. See Audit Fluentd.

  4. Run the following commands.

    cd ~/privacera/privacera-manager
./privacera-manager.sh update

Configuration Properties

Property Description Example
AUDIT_FLUENTD_AUDIT_DESTINATION 

Set the audit destination where the audits will be saved. If the value is set to S3, the audits get stored in the AWS S3 server. For S3, the default time interval to publish the audits is 3600s (1hr).

Local storage should be used only for development and testing purposes. All the audit received are stored in the same container/pod. 

Value: local, s3, azure-blob, azure-adls 

 s3
When the destination is `local`, edit the following property:
AUDIT_FLUENTD_LOCAL_FILE_TIME_INTERVAL This is the time interval after which the audits will be pushed to the local destination. 3600s
When the destination is `s3`, edit the following properties:
AUDIT_FLUENTD_S3_BUCKET 

Set the bucket name, if you set the audit destination above to S3.

Leave unchanged, if you set the audit destination to local. 

 bucket_1
AUDIT_FLUENTD_S3_REGION 

Set the bucket region, if you set the audit destination above to S3.

Leave unchanged, if you set the audit destination to local. 

 us-east-1
AUDIT_FLUENTD_S3_FILE_TIME_INTERVAL This is the time interval after which the audits will be pushed to the S3 destination. 3600s

AUDIT_FLUENTD_S3_ACCESS_KEY

AUDIT_FLUENTD_S3_SECRET_KEY 

Set the access and secret key, if you set the audit destination above to S3.

Leave unchanged, if you set the audit destination to local and are using AWS IAM Instance Role.

AUDIT_FLUENTD_S3_ACCESS_KEY: "AKIAIOSFODNN7EXAMPLE"

AUDIT_FLUENTD_S3_SECRET_KEY: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
AUDIT_FLUENTD_S3_BUCKET_ENCRYPTION_TYPE

Property to encrypt an S3 bucket. You can use the property, if you have set `S3` as the audit destination in the property, `AUDIT_FLUENTD_AUDIT_DESTINATION`.

You can assign one of the following values as the encryption types:

  • SSE-S3
  • SSE-KMS
  • SSE-C
  • NONE

SSE-S3 and SSE-KMS are encryptions managed by AWS. You need to enable the server-side encryption for the S3 bucket. For more information on how to enable SSE-S3 or SSE-KMS encryption types, [click here](https://docs.aws.amazon.com/AmazonS3/latest/userguide/default-bucket-encryption.html)

SSE-C is the custom encryption type, where the encryption key and MD5 have to generated separately.

NONE
AUDIT_FLUENTD_S3_BUCKET_ENCRYPTION_KEY

If you have set `SSE-C` encryption type in the `AUDIT_FLUENTD_S3_BUCKET_ENCRYPTION_TYPE` property, then the encryption key is mandatory. It is optional for `SSE-KMS` encryption type.
AUDIT_FLUENTD_S3_BUCKET_ENCRYPTION_KEY_MD5

If you have set `SSE-C` encryption type in the `AUDIT_FLUENTD_S3_BUCKET_ENCRYPTION_TYPE` property, then the MD5 encryption key is mandatory.

To get the MD5 hash for the encryption key, run the following command:

echo -n "<generated-key>" |  openssl dgst -md5 -binary | openssl enc -base64

When the destination is `azure-blob` or `azure-adls`, edit the following properties:

AUDIT_FLUENTD_AZURE_STORAGE_ACCOUNT

AUDIT_FLUENTD_AZURE_CONTAINER

Set the storage account and the container, if you set the audit destination above to Azure Blob or Azure ADLS.

To know how to get the ADLS properties, [click here](../pm-ig/get_adls_properties.md).

Leave unchanged, if you set the audit destination to local.

Note: Currently, it supports Azure blob storage only.

AUDIT_FLUENTD_AZURE_STORAGE_ACCOUNT: "storage_account_1"

AUDIT_FLUENTD_AZURE_CONTAINER: "container_1"
AUDIT_FLUENTD_AZURE_FILE_TIME_INTERVAL This is the time interval after which the audits will be pushed to the Azure ADLS/Blob destination. 3600s
AUDIT_FLUENTD_AUTH_TYPE Select an authentication type from the dropdown list.

AUDIT_FLUENTD_AZURE_STORAGE_ACCOUNT_KEY

AUDIT_FLUENTD_AZURE_STORAGE_SAS_TOKEN 

Configure this property, if you have selected `SAS Key` in the property, `AUDIT_FLUENTD_AUTH_TYPE`.

Set the storage account key and the SAS token, if you set the audit destination above to Azure Blob.

Leave unchanged, if you're using Azure's Managed Identity Service.

  

AUDIT_FLUENTD_AZURE_OAUTH_TENANT_ID

AUDIT_FLUENTD_AZURE_OAUTH_APP_ID

AUDIT_FLUENTD_AZURE_OAUTH_SECRET

Configure this property, if you have selected `OAUTH` in the property, `AUDIT_FLUENTD_AUTH_TYPE`.

Set the storage account key and the SAS token, if you set the audit destination above to Azure ADLS.

Leave unchanged, if you're using Azure's Managed Identity Service.

  

AUDIT_FLUENTD_AZURE_USER_MANAGED_IDENTITY_ENABLE

AUDIT_FLUENTD_AZURE_USER_MANAGED_IDENTITY

Configure this property, if you have selected `MSI (UserManaged)` in the property, `AUDIT_FLUENTD_AUTH_TYPE`.

Related Information

For further reading, see: