Skip to main content

Privacera Platform

Configure Kafka destination
:

This topic covers how you can configure Kafka audit endpoint in AuditServer for the Ranger plugin and the Ranger Admin to send the audits.

Prerequisites

Ensure the following prerequisites are met:

  • AuditServer must be configured. For more information, click here.

CLI configuration

  1. SSH to an instance where Privacera is installed.

  2. Run the following commands.

    cd ~/privacera/privacera-manager
cp config/sample-vars/vars.auditserver.kafka.destination.yml config/custom-vars/
vi config/custom-vars/vars.auditserver.kafka.destination.yml

  3. Modify the properties. For property details and description, refer to the Configuration Properties below.

    AUDITSERVER_KAFKA_DESTINATION:"<PLEASE_CHANGE>"AUDITSERVER_KAFKA_BROKER_LIST:"<PLEASE_CHANGE>"AUDITSERVER_KAFKA_TOPIC_NAME:"<PLEASE_CHANGE>"AUDITSERVER_KAFKA_SECURITY_PROTOCOL:"<PLEASE_CHANGE>"AUDITSERVER_KAFKA_SSL_KEYSTORE_LOCATION:"<PLEASE_CHANGE>"AUDITSERVER_KAFKA_SSL_KEYSTORE_PASSWORD:"<PLEASE_CHANGE>"AUDITSERVER_KAFKA_SSL_KEY_PASSWORD:"<PLEASE_CHANGE>"AUDITSERVER_KAFKA_SSL_TRUSTSTORE_LOCATION:"<PLEASE_CHANGE>"AUDITSERVER_KAFKA_SSL_TRUSTSTORE_PASSWORD:"<PLEASE_CHANGE>"AUDITSERVER_KAFKA_SASL_JAAS_CONFIG:"<PLEASE_CHANGE>"AUDITSERVER_KAFKA_SASL_MECHANISM:"<PLEASE_CHANGE>"AUDITSERVER_KAFKA_SASL_LOGIN_CALLBACK_HANDLER_CLASS:"<PLEASE_CHANGE>"AUDITSERVER_KAFKA_OAUTH_TOKEN_ENDPOINT_URI:"<PLEASE_CHANGE>"AUDITSERVER_KAFKA_OAUTH_WITH_SSL:"<PLEASE_CHANGE>"AUDITSERVER_OAUTH_ACCEPT_UNSECURE_SERVER:"<PLEASE_CHANGE>"AUDITSERVER_OAUTH_LOGIN_GRANT_TYPE:"<PLEASE_CHANGE>"AUDITSERVER_KAFKA_OAUTH_CLIENT_ID:"<PLEASE_CHANGE>"AUDITSERVER_KAFKA_OAUTH_CLIENT_SECRET:"<PLEASE_CHANGE>"AUDITSERVER_KAFKA_BATCH_FILESPOOL_DIR:"/workdir/privacera-audit-server/kafka-spool"ADMIN_AUDITSERVER_KAFKA_DESTINATION:"<PLEASE_CHANGE>"ADMIN_AUDITSERVER_KAFKA_BROKER_LIST:"<PLEASE_CHANGE>"ADMIN_AUDITSERVER_KAFKA_TOPIC_NAME:"<PLEASE_CHANGE>"ADMIN_AUDITSERVER_KAFKA_SECURITY_PROTOCOL:"<PLEASE_CHANGE>"ADMIN_AUDITSERVER_KAFKA_SSL_KEYSTORE_LOCATION:"<PLEASE_CHANGE>"ADMIN_AUDITSERVER_KAFKA_SSL_KEYSTORE_PASSWORD:"<PLEASE_CHANGE>"ADMIN_AUDITSERVER_KAFKA_SSL_KEY_PASSWORD:"<PLEASE_CHANGE>"ADMIN_AUDITSERVER_KAFKA_SSL_TRUSTSTORE_LOCATION:"<PLEASE_CHANGE>"ADMIN_AUDITSERVER_KAFKA_SSL_TRUSTSTORE_PASSWORD:"<PLEASE_CHANGE>"ADMIN_AUDITSERVER_KAFKA_SASL_JAAS_CONFIG:"<PLEASE_CHANGE>"ADMIN_AUDITSERVER_KAFKA_SASL_MECHANISM:"<PLEASE_CHANGE>"ADMIN_AUDITSERVER_KAFKA_SASL_LOGIN_CALLBACK_HANDLER_CLASS:"<PLEASE_CHANGE>"ADMIN_AUDITSERVER_KAFKA_OAUTH_TOKEN_ENDPOINT_URI:"<PLEASE_CHANGE>"ADMIN_AUDITSERVER_KAFKA_OAUTH_WITH_SSL:"<PLEASE_CHANGE>"ADMIN_AUDITSERVER_OAUTH_ACCEPT_UNSECURE_SERVER:"<PLEASE_CHANGE>"ADMIN_AUDITSERVER_OAUTH_LOGIN_GRANT_TYPE:"<PLEASE_CHANGE>"ADMIN_AUDITSERVER_KAFKA_OAUTH_CLIENT_ID:"<PLEASE_CHANGE>"ADMIN_AUDITSERVER_KAFKA_OAUTH_CLIENT_SECRET:"<PLEASE_CHANGE>"ADMIN_AUDITSERVER_KAFKA_BATCH_FILESPOOL_DIR:"/workdir/privacera-audit-server/kafka-spool"

  4. Run the following commands.

    cd ~/privacera/privacera-manager
./privacera-manager.sh update
Configuration properties

The property names prefixed with ADMIN_ refer to Privacera Ranger Admin, whereas the others refer to Privacera Portal.

Property

Description

Example

AUDITSERVER_KAFKA_DESTINATION

ADMIN_AUDITSERVER_KAFKA_DESTINATION

Set to true if audit destination is kafka

AUDITSERVER_KAFKA_BROKER_LIST

ADMIN_AUDITSERVER_KAFKA_BROKER_LIST

A list of host/port pairs to use for establishing the initial connection to the Kafka cluster. This list should be in the form host1:port1,host2:port2,.... Since these servers are just used for the initial connection to discover the full cluster membership (which may change dynamically), this list need not contain the full set of servers (you may want more than one, though, in case a server is down).

10.xxx.xx.xxx:9093

AUDITSERVER_KAFKA_TOPIC_NAME

ADMIN_AUDITSERVER_KAFKA_TOPIC_NAME

Topic name to which audits are to be sent

topic-name

AUDITSERVER_KAFKA_SECURITY_PROTOCOL

ADMIN_AUDITSERVER_KAFKA_SECURITY_PROTOCOL

Protocol used to communicate with brokers.

Valid values are: PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL.

SASL_SSL

AUDITSERVER_KAFKA_SSL_KEYSTORE_LOCATION

ADMIN_AUDITSERVER_KAFKA_SSL_KEYSTORE_LOCATION

The location of the key store file.

Make sure key is copied in config/ssl folder. Provide name of the file.

kafka.server.keystore

AUDITSERVER_KAFKA_SSL_KEYSTORE_PASSWORD

ADMIN_AUDITSERVER_KAFKA_SSL_KEYSTORE_PASSWORD

The store password for the key store file.This is optional and only needed if AUDITSERVER_KAFKA_SSL_KEYSTORE_LOCATION is configured.

privacera

AUDITSERVER_KAFKA_SSL_KEY_PASSWORD

ADMIN_AUDITSERVER_KAFKA_SSL_KEY_PASSWORD

The password of the private key in the key store file. This is optional.

privacera

AUDITSERVER_KAFKA_SSL_TRUSTSTORE_LOCATION

ADMIN_AUDITSERVER_KAFKA_SSL_TRUSTSTORE_LOCATION

The location of the trust store file. Make sure the key is copied in config/ssl folder. Provide name of the file.

kafka.server.truststore

AUDITSERVER_KAFKA_SSL_TRUSTSTORE_PASSWORD

ADMIN_AUDITSERVER_KAFKA_SSL_TRUSTSTORE_PASSWORD

The password for the trust store file.

privacera

AUDITSERVER_KAFKA_SASL_JAAS_CONFIG

ADMIN_AUDITSERVER_KAFKA_SASL_JAAS_CONFIG

Kafka uses the Java Authentication and Authorization Service (JAAS) for SASL configuration. You must provide JAAS configurations for all SASL authentication mechanisms.

E.g "org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required ;" if AUDITSERVER_KAFKA_SASL_MECHANISM is "OAUTHBEARER

org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required ;

AUDITSERVER_KAFKA_SASL_MECHANISM

ADMIN_AUDITSERVER_KAFKA_SASL_MECHANISM

SASL mechanism used for connections. This may be any mechanism for which a security provider is available. GSSAPI is the default mechanism.

OAUTHBEARER

AUDITSERVER_KAFKA_SASL_LOGIN_CALLBACK_HANDLER_CLASS

ADMIN_AUDITSERVER_KAFKA_SASL_LOGIN_CALLBACK_HANDLER_CLASS

The LoginModule for the selected SASL_MECHANISM

E.g "io.strimzi.kafka.oauth.client.JaasClientOauthLoginCallbackHandler" if AUDITSERVER_KAFKA_SASL_MECHANISM is "OAUTHBEARER

io.strimzi.kafka.oauth.client.JaasClientOauthLoginCallbackHandler

AUDITSERVER_KAFKA_OAUTH_TOKEN_ENDPOINT_URI

ADMIN_AUDITSERVER_KAFKA_OAUTH_TOKEN_ENDPOINT_URI

OAUTH Token endpoint URL used by the application in order to get an access token or a refresh token

http://10.211.93.140:4444/oauth2/token

AUDITSERVER_KAFKA_OAUTH_WITH_SSL

ADMIN_AUDITSERVER_KAFKA_OAUTH_WITH_SSL

Set to true if SSL is applied on OAUTH.

AUDITSERVER_OAUTH_ACCEPT_UNSECURE_SERVER

ADMIN_AUDITSERVER_OAUTH_ACCEPT_UNSECURE_SERVER

Set to true if OAUTH accept unsecure server.

AUDITSERVER_OAUTH_LOGIN_GRANT_TYPE

ADMIN_AUDITSERVER_OAUTH_LOGIN_GRANT_TYPE

The authorization server needs to know which grant type the application wants to use since it affects the kind of credential it will issue

e.g client_credentials

client_credentials

AUDITSERVER_KAFKA_OAUTH_CLIENT_ID

ADMIN_AUDITSERVER_KAFKA_OAUTH_CLIENT_ID

The ID of the application that asks for authorization.

broker-kafka

AUDITSERVER_KAFKA_OAUTH_CLIENT_SECRET

ADMIN_AUDITSERVER_KAFKA_OAUTH_CLIENT_SECRET

The secret of the application that asks for authorization.

broker-kafka

AUDITSERVER_KAFKA_BATCH_FILESPOOL_DIR

ADMIN_AUDITSERVER_KAFKA_BATCH_FILESPOOL_DIR

If audit framework detects that an audit destination is down then it buffers the audit messages in memory. Once memory buffer fills up then it can be configured to spool the unsent messages to disk files to prevent or minimize the loss of audit messages. Local disk directory where spool files would be kept. This value must be specified.

Default location is "/workdir/privacera-audit-server/kafka-spool

/workdir/privacera-audit-server/kafka-spool