Audit Fluentd
This topic covers how you can store the audits from AuditServer locally, or on a cloud, for example, AWS S3, Azure blob, and Azure ADLS Gen 2.
Prerequisites
Ensure the following prerequisites are met:
-
AuditServer must be up and running. For more information, refer to AuditServer.
-
If you're configuring Fluentd for an Azure environment and want to configure User Managed Service Identity (MSI), assign the following two IAM roles to the Azure Storage account for the User Managed Service Identity where the audits will be stored.
- Owner or Contributor
- Storage Blob Data Owner or Storage Blob Data Contributor
Note
If your Azure environment is Docker-based, then configure MSI on a virtual machine, whereas for a Kubernetes-based environment, configure MSI on a virtual machine scale set (VMSS).
CLI Configuration
-
SSH to the instance where Privacera is installed.
-
Run the following commands.
cd ~/privacera/privacera-manager cp config/sample-vars/vars.audit-fluentd.yml config/custom-vars/ vi config/custom-vars/vars.audit-fluentd.yml
-
Modify the properties below. For property details and description, refer to the Configuration Properties below.
You can also add custom properties that are not included by default. See Audit Fluentd.
-
Run the following commands.
cd ~/privacera/privacera-manager ./privacera-manager.sh update
Configuration Properties
Property | Description | Example |
---|---|---|
AUDIT_FLUENTD_AUDIT_DESTINATION |
Set the audit destination where the audits will be saved. If the value is set to S3, the audits get stored in the AWS S3 server. For S3, the default time interval to publish the audits is 3600s (1hr). Local storage should be used only for development and testing purposes. All the audit received are stored in the same container/pod. Value: local, s3, azure-blob, azure-adls |
s3 |
When the destination is `local`, edit the following property: | ||
AUDIT_FLUENTD_LOCAL_FILE_TIME_INTERVAL | This is the time interval after which the audits will be pushed to the local destination. | 3600s |
When the destination is `s3`, edit the following properties: | ||
AUDIT_FLUENTD_S3_BUCKET |
Set the bucket name, if you set the audit destination above to S3. Leave unchanged, if you set the audit destination to local. |
bucket_1 |
AUDIT_FLUENTD_S3_REGION |
Set the bucket region, if you set the audit destination above to S3. Leave unchanged, if you set the audit destination to local. |
us-east-1 |
AUDIT_FLUENTD_S3_FILE_TIME_INTERVAL | This is the time interval after which the audits will be pushed to the S3 destination. | 3600s |
AUDIT_FLUENTD_S3_ACCESS_KEY AUDIT_FLUENTD_S3_SECRET_KEY |
Set the access and secret key, if you set the audit destination above to S3. Leave unchanged, if you set the audit destination to local and are using AWS IAM Instance Role. |
AUDIT_FLUENTD_S3_ACCESS_KEY: "AKIAIOSFODNN7EXAMPLE" AUDIT_FLUENTD_S3_SECRET_KEY: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" |
AUDIT_FLUENTD_S3_BUCKET_ENCRYPTION_TYPE |
Property to encrypt an S3 bucket. You can use the property, if you have set `S3` as the audit destination in the property, `AUDIT_FLUENTD_AUDIT_DESTINATION`. You can assign one of the following values as the encryption types:
SSE-S3 and SSE-KMS are encryptions managed by AWS. You need to enable the server-side encryption for the S3 bucket. For more information on how to enable SSE-S3 or SSE-KMS encryption types, [click here](https://docs.aws.amazon.com/AmazonS3/latest/userguide/default-bucket-encryption.html) SSE-C is the custom encryption type, where the encryption key and MD5 have to generated separately. |
NONE |
AUDIT_FLUENTD_S3_BUCKET_ENCRYPTION_KEY |
If you have set `SSE-C` encryption type in the `AUDIT_FLUENTD_S3_BUCKET_ENCRYPTION_TYPE` property, then the encryption key is mandatory. It is optional for `SSE-KMS` encryption type. |
|
AUDIT_FLUENTD_S3_BUCKET_ENCRYPTION_KEY_MD5 |
If you have set `SSE-C` encryption type in the `AUDIT_FLUENTD_S3_BUCKET_ENCRYPTION_TYPE` property, then the MD5 encryption key is mandatory. To get the MD5 hash for the encryption key, run the following command:
|
|
When the destination is `azure-blob` or `azure-adls`, edit the following properties: | ||
AUDIT_FLUENTD_AZURE_STORAGE_ACCOUNT AUDIT_FLUENTD_AZURE_CONTAINER |
Set the storage account and the container, if you set the audit destination above to Azure Blob or Azure ADLS. To know how to get the ADLS properties, [click here](../pm-ig/get_adls_properties.md). Leave unchanged, if you set the audit destination to local. Note: Currently, it supports Azure blob storage only. |
AUDIT_FLUENTD_AZURE_STORAGE_ACCOUNT: "storage_account_1" AUDIT_FLUENTD_AZURE_CONTAINER: "container_1" |
AUDIT_FLUENTD_AZURE_FILE_TIME_INTERVAL | This is the time interval after which the audits will be pushed to the Azure ADLS/Blob destination. | 3600s |
AUDIT_FLUENTD_AUTH_TYPE | Select an authentication type from the dropdown list. | |
AUDIT_FLUENTD_AZURE_STORAGE_ACCOUNT_KEY AUDIT_FLUENTD_AZURE_STORAGE_SAS_TOKEN |
Configure this property, if you have selected `SAS Key` in the property, `AUDIT_FLUENTD_AUTH_TYPE`. Set the storage account key and the SAS token, if you set the audit destination above to Azure Blob. Leave unchanged, if you're using Azure's Managed Identity Service. |
|
AUDIT_FLUENTD_AZURE_OAUTH_TENANT_ID AUDIT_FLUENTD_AZURE_OAUTH_APP_ID AUDIT_FLUENTD_AZURE_OAUTH_SECRET |
Configure this property, if you have selected `OAUTH` in the property, `AUDIT_FLUENTD_AUTH_TYPE`. Set the storage account key and the SAS token, if you set the audit destination above to Azure ADLS. Leave unchanged, if you're using Azure's Managed Identity Service. |
|
AUDIT_FLUENTD_AZURE_USER_MANAGED_IDENTITY_ENABLE AUDIT_FLUENTD_AZURE_USER_MANAGED_IDENTITY |
Configure this property, if you have selected `MSI (UserManaged)` in the property, `AUDIT_FLUENTD_AUTH_TYPE`. |