Access Audits¶
Access Audits track all data access attempts whether allowed or denied across all integrated data platforms. This provides visibility into who accessed what, when, and under what conditions. Privacera leverages Apache Ranger's audit framework to store and present this information.
Access Audit Fields¶
Privacera normalizes the audit logs from different data platforms and presents them in a consistent format. This normalization allows users to easily search and filter audit logs across different data platforms.
| Field | Description | Supported In Plugin/DataServer | Supported In PolicySync |
|---|---|---|---|
| Policy ID | ID of the policy that resulted in the decision |  Yes |  No Policy ID |
| Result | ALLOW or DENY | Yes | Yes |
| Event Time | Timestamp of the event | Yes | Yes |
| Application Name | Name of the application making the request | Yes | Yes |
| User / Principal | Identity that attempted the access | Yes | Yes |
| Service Name | Name of the service making the request | Yes | Yes |
| Service Type | Type of service (e.g., Hive, HDFS, S3) | Yes | Yes |
| Resource | Database, table, column, path, etc. | Yes |  Limited Audit Granularity |
| Access Type | Type of access (e.g., SELECT, INSERT, UPDATE). As defined in Privacera | Yes | Yes |
| Action Type | Type of action in the data platform (e.g., mread, write) | Yes | Yes |
| Agent Host Name | Hostname of the agent processing the request | Yes | Agent Hostname |
| Client IP | IP address of the user or service making the request | Client IP | Client IP |
| Cluster Name | Name of the cluster where the request was made. E.g. Databricks cluster or workspace | Cluster Name | Cluster Name |
| Zone Name | Name of the security zone for the resource | Yes | Zone Name |
| Event Count | Number of times the event occurred in a batch request | Event Count | Event Count |
| Tags (if applicable) | Tags used in the policy decision | Yes | Tags |
Limitations¶
There are some limitations to access audits based on the integration method and also the data platform:
Policy ID¶
The policy ID is not available in the audit logs for the data platforms that are integrated using the Privacera PolicySync connector. This is because the policies from Privacera are translated to the native permissions and these are enforced in the data platform. The audit logs are generated by the data platform and do not include the policy ID in the audit logs. For these audit logs, the Policy ID will be empty in the audit logs.
Audit Granularity¶
In some cases, the audit logs may not capture the most granular level of access. For example: in Databricks Unity Catalog Privacera's connector fetches the audits that are generated by Unity Catalog. The audit logs consists of the query the user ran and this does not include the individual tables or columns that were accessed. The SQL statement is stored in the resource field in Privacera. Here is an example of how the audit log looks like:
Agent Hostname¶
This is the hostname of the data platform or Privacera DataServer where Privacera's Plugin is running. So this field is not applicable for the PolicySync connector. The hostname will be empty in the audit logs for the PolicySync connector.
Client IP¶
Client IP is the IP address of the user or service making the request. This is not available for all data platforms.
Cluster Name¶
Cluster name is the name of the cluster or workspace where the request was made. This might not be applicable for all data platforms. In such cases, the cluster name will be empty in the audit logs.
Zone Name¶
Zone name is the name of the security zone for the resource. In Plugin and DataServer integrations, the security zone is determined when the access is checked and the audit is created. However, in PolicySync integrations, in most cases the resource granularity is not available. So the zone name will be empty in the audit logs for PolicySync integrations.
Event Count¶
Event count is the number of times the event occurred in a batch request. In high throughput environments like Apache Kafka, the same topic might be accessed multiple times in short durations. In such cases, to reduce the number of audit logs, the event count is used to indicate the number of times the event occurred. The event count might not be available for all data platforms. In such cases, the event count will be generated as 1.
Tags¶
Tags are used in the policy decision. In Privacera Policy integrations, the enforcement is done by the data platform and the tags won't be available in the audit logs. In such cases, the tags will be empty in the audit logs.
Audit Log Storage¶
Privacera provides the option to store the audit logs in different storage systems. The audit logs can be stored in
- Amazon S3
- Azure Blob Storage
- Google Cloud Storage
- Apache Solr
Audit Retention¶
Audits in Object Store can be retained as long as it is needed for compliance or other purposes. Since the audits are stored in the customer's Cloud, it is the customer's responsibility to manage the retention of the audit logs.
In Self Managed Privacera, Audits in Apache Solr are generally retained for 90 days. However, the retention period can be configured in the Privacera Audit Server. Refer to the Configuring Retention for Ranger Audits in Solr for more details.
In Privacera Cloud and Privacera Data Plane deployments, the audit logs are retained for 2 weeks. It is strongly recommended to configured to transfer the audit logs to customers' object store for long term retention.
Accessing Audits¶
Privacera provides a unified interface to access the audit logs across different data platforms. The audit logs can be accessed using the Privacera Portal. Privacera's Portal fetches the audit logs from Apache Solr.
Since the audits are available in Object Stores also, it is possible to access the audit logs using modern data platforms like Apache Spark and Trino. This requires additional configuration on the data platform side.