Configuring Retention for Ranger Audits in Solr¶

In Self Managed deployments, audit logs are stored in Apache Solr which is used by Privacera Portal to display audit logs. By default, audit logs are retained for 90 days in Solr. You can configure the retention period for Ranger audit logs in Solr.

Prerequisites¶

Setup¶

Follow these steps to configure the retention period for Ranger audits in Solr:

1. SSH into the instance where Privacera Manager is installed.
2. Navigate to the privacera-manager directory using the following command:

   Bash
   1	cd ~/privacera/privacera-manager/

3. Run the following command to copy the sample vars. The -n flag ensures that the file is not overwritten if it already exists.

   Bash
   1	cp -n sample-vars/vars.solr.yml custom-vars/

4. Run the following command to open the .yml file for editing.

   Bash
   1	vi custom-vars/vars.solr.yml

5. Add or update the following property:

1

MAX_AUDIT_RETENTION_DAYS: "90" #Audits will be automatically deleted after 90 days.

1
2
3

cd ~/privacera/privacera-manager
./privacera-manager.sh setup
./pm_with_helm.sh upgrade 

Validation¶

To confirm that the setup is successful, after running the above steps and make some access operation which will generate audit logs. The new audit logs will have the TTL set as per the configuration. If you have access to the Solr UI, you can verify the TTL configuration for the ranger_audits collection for the newly created audit logs.

Here is a sample solrconfig file with ranger-audits TTL set to +7DAYS:

Here is a sample ranger-audit with TTL set to +7DAYS:

Purging Historical Ranger Audits¶

Ranger access audits created before upgrade to Privacera version 9.0.7.1 might not have TTL set and needs to be manually deleted from Apache Solr. This can be done by making HTTP post requests to the Solr collection to delete the audits greater than the retention period.

Here is a sample script to delete ranger-audits from Solr manually. Please use this script as a reference and modify it as per your requirement.

Create a script file called delete_ranger_audits.sh with the following content. Update SOLR_URL with your Solr URL

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

#!/bin/bash

# Check if correct number of arguments are provided
if [ "$#" -ne 2 ]; then
  echo "Usage: $0 <start_date> <end_date>"
  echo "Dates should be in ISO 8601 format, e.g., 2024-01-01T00:00:00Z"
  exit 1
fi

# Assign input arguments to variables
START_DATE=$1
END_DATE=$2

# Solr server and collection details
# UPDATE THE SOLR_URL variable with your Solr URL. Make sure this URL is accessible from the machine where the script is executed.
SOLR_URL="https://localhost:8983/solr"  # Update with your Solr URL
COLLECTION_NAME="ranger_audits"      # Replace with your collection name
DATE_FIELD="evtTime"                 # Replace with your date field name

# Formulate the query for deletion
QUERY="${DATE_FIELD}:[${START_DATE} TO ${END_DATE}]"

# Endpoint for updating (deleting) documents
DELETE_URL="${SOLR_URL}/${COLLECTION_NAME}/update"

# Delete command payload
DELETE_PAYLOAD=$(cat <<EOF
{
  "delete": {
    "query": "${QUERY}"
  }
}
EOF
)

# Make the delete request
echo "Deleting documents between ${START_DATE} and ${END_DATE} in collection '${COLLECTION_NAME}'..."
curl -X POST -H "Content-Type: application/json" --data "${DELETE_PAYLOAD}" "${DELETE_URL}"

# Commit the changes to apply deletion
echo "Committing changes..."
curl "${DELETE_URL}?commit=true"

echo "Deletion process completed."      

1. Make the Script Executable

   Bash
   1	chmod +x delete_ranger_audits.sh

2. Run the Script

   Bash
   1 2	#./delete_ranger_audits.sh ${delete_doc_from_date} ${delete_doc_to_date} ./delete_ranger_audits.sh 2024-01-01T00:00:00Z 2024-01-31T23:59:59Z

Comments