Skip to content

Send Audits in Self Managed deployment to S3 in AWS

Privacera Audit Server receives audits from different sources and uses Fluentd to send them to the configured destination. In a self-managed deployment, you can configure the Audit Server to send audit logs to Fluentd, which can then be configured to send the audits to external storage, such as Amazon Simple Storage Service (S3) on Amazon Web Services (AWS).

This is applicable only for self-managed deployments. For PrivaceraCloud, please reach out to your contact at Privacera.

Prerequisites

Prerequisite Description
AWS Account You must have an existing AWS account and login credentials with the necessary privileges to manage your S3 bucket.
Bucket in S3 You must have a bucket in S3 where you want to store the audit logs. E.g privacera_audits
AuditServer AuditServer must be enabled. This is enabled by default in Self Managed Deployment

Setup

To setup sending audit logs to S3, follow these steps:

  1. SSH into the instance where Privacera Manager is installed.
  2. Navigate to the config directory with the following command:
    Bash
    1
    cd ~/privacera/privacera-manager/config
  3. Run the following command to copy the file from sample vars. The -n flag ensures that the file is not overwritten if it already exists.
    Bash
    1
       cp -n sample-vars/vars.audit-fluentd.yml custom-vars/
  4. Run the following command to open the .yml file for editing.
    Bash
    1
    vi custom-vars/vars.audit-fluentd.yml

  5. Modify the following properties:

    Property Description
    AUDIT_FLUENTD_S3_ACCESS_KEY The AWS access key used for authenticating the Fluentd connection to the S3 bucket.
    AUDIT_FLUENTD_S3_SECRET_KEY The AWS secret key used in combination with the access key for authenticating the connection to the S3 bucket.
    AUDIT_FLUENTD_S3_BUCKET The name of the S3 bucket where logs are to be stored.
    AUDIT_FLUENTD_S3_PATH_PREFIX The path prefix in the S3 bucket where logs will be stored.
    AUDIT_FLUENTD_S3_REGION The AWS region where the S3 bucket is located (default: us-east-1).

  6. After configuring these properties, update your Privacera Manager platform instance by following the instructions in the Quick Start guide.

Validation

To confirm that the setup was successful, follow the steps above, perform an access operation, then navigate to the S3 bucket and verify that the audit logs are stored in the specified path.

Tip

Note that the audits are not sent to S3 immediately. The Fluentd buffer is flushed based on the configured time interval. If you want to update the time interval, refer to the Advanced Configuration section.

Here is a sample S3 bucket Access audit logs:

S3 bucket Access audit

Advanced Configuration

Here are some of the advanced configurations that you can do to customize the audit logs sent to S3

Configure S3 Encryption

Configure S3 Encryption

Prerequisites

Prerequisite Description
S3 Bucket Encryption Your S3 bucket must be encrypted.
Encryption Key You must have an encryption key for server-side encryption.

Configuration

If your S3 bucket is encrypted, you can configure the encryption key in the vars.audit-fluentd.yml file

  1. SSH into the instance where Privacera Manager is installed.
  2. Run the following command to open the .yml file for editing:
    Bash
    1
    vi config/custom-vars/vars.audit-fluentd.yml
  3. To enable encryption for S3, include the following property in the vars.audit-fluentd.yml file:
Property Description
AUDIT_FLUENTD_S3_BUCKET_ENCRYPTION_TYPE Specifies the encryption type for the S3 bucket (e.g., SSE-S3, SSE-KMS, SSE-C, or NONE).
AUDIT_FLUENTD_S3_BUCKET_ENCRYPTION_KEY Specifies the encryption key for SSE-KMS or SSE-C encryption types (required if SSE-KMS or SSE-C is used as encryption type).
AUDIT_FLUENTD_S3_BUCKET_ENCRYPTION_KEY_MD5 Specifies the MD5 hash of the encryption key for SSE-C encryption type. (required if SSE-C is used as encryption type).
Configure Time Interval for Sending Audits to S3

Configure Time Interval for Sending Audits to S3

If you want to change default time intervals for sending audits to S3, you can configure the following properties in the vars.audit-fluentd.yml file

  1. SSH into the instance where Privacera Manager is installed.
  2. Run the following command to open the .yml file for editing:
    Bash
    1
    vi config/custom-vars/vars.audit-fluentd.yml
  3. Modify the following properties:
Variable Definition
AUDIT_FLUENTD_S3_FILE_TIME_INTERVAL This parameter defines the time-based interval for Fluentd to create a new log file. The log file will be rotated or split based on this time interval.
AUDIT_FLUENTD_S3_FILE_TIME_WAIT Allows Fluentd to wait for a specified duration after the timekey(AUDIT_FLUENTD_S3_FILE_TIME_INTERVAL) interval to ensure all logs for that time period are written before rotating the log file.
AUDIT_FLUENTD_S3_FLUSH_INTERVAL Specifies how frequently Fluentd should push the audits to the S3 destination. Example: 30s.
  1. Once the properties are configured, update your Privacera Manager platform instance by following the Quick Start guide.

Comments