Dremio

Introduction

This section covers how you can integrate Dremio with Privacera. You can use Dremio for table-level access control with the native Ranger plugin.

By integrating Dremio with Privacera, you'll be provided with comprehensive data lake security and fine-grained access control across multi-cloud environments. Dremio works directly with data lake storage. Using Dremio's query engine and ability to democratize data access, Privacera implements fine-grained access control policies, then automatically enforces and audits them at enterprise scale.

Dremio is supported with the following data sources:

S3
ADLS
Hive
Redshift

Prerequisites

Ensure the following prerequisites are met:

A Privacera Manager host where Privacera services are running.
A Dremio host where Dremio Enterprise Edition is installed. (The Community Edition is not supported.)

Configuration

To configure Dremio:

Note

There are limitations in the Dremio native Hive plugin because Dremio uses Ranger 1.1.0.

Audit Server basic auth needs to be disabled because it's not supported.
Dremio does not support solr audits in SSL if it is enabled in the audit server.

Run the following commands:

cd ~/privacera/privacera-manager
cp config/sample-vars/vars.dremio.yml config/custom-vars/

Update the following properties:

AUDITSERVER_ENABLE: "true"
AUDITSERVER_AUTH_TYPE: "none"
AUDITSERVER_SSL_ENABLE: "false"

Run the following commands to configure the audit server for Dremio Native Hive Ranger Based authorization..
```
cd ~/privacera/privacera-manager 
cp config/sample-vars/vars.auditserver.yml config/custom-vars/ 
vi config/custom-vars/vars.auditserver.yml
```
After the update is completed, the Dremio plugin installation script privacera_dremio.sh and custom configuration archive privacera_custom_conf.tar.gz is generated at the location ~/privacera/privacera-manager/output/dremio
Configure Privacera plugin depending on how you have installed Dremio in your instance.
For a new or existing data source configured in Dremio Data Lake, ensure Enable external authorization plugin checkbox under Settings > Advanced Options of the data source is selected in the Dremio UI.
Restart the Dremio service.

Kubernetes

Depending on your cloud provider, you can set up Dremio in a Kubernetes container. For more information, see the following links.

After setting up Dremio, perform the following steps to deploy Privacera plugin. The steps assume that your Privacera Manager host instance is separate from your Dremio Kubernetes instance. If they are configured on the single instance, then modify the steps accordingly.

SSH to your instance where Dremio is installed containing the Dremio Kubernetes artifacts and change to the dremio-cloud-tools/charts/dremio_v2/ directory.
Copy the privacera_dremio.sh and privacera_custom_conf.tar.gz files from your Privacera Manager host instance to the dremio_v2 folder in your Dremio Kubernetes instance.

Run the following commands:

mkdir -p privacera_config 
mv privacera_dremio.sh privacera_config/ 
mv privacera_custom_conf.tar.gz privacera_config/

Update configmap.yml to add new configmap for Privacera configuration.
```
vi templates/dremio-configmap.yaml
```

Add the following configuration at the start of the file:

apiVersion: v1
kind: ConfigMap
metadata:
name: dremio-privacera-install
data:
privacera_dremio.sh: |- {{ .Files.Get "privacera_config/privacera_dremio.sh" | nindent 4 }}
binaryData:
privacera_custom_conf.tar.gz: {{ .Files.Get "privacera_config/privacera_custom_conf.tar.gz" | b64enc | nindent 4 }}
---

Update dremio-env to add Privacera jars and configuration in the Dremio classpath.
```
vi config/dremio-env
```

Add the following variable, or update it if it already exists:

DREMIO_EXTRA_CLASSPATH=/opt/privacera/conf:/opt/privacera/dremio-ext-jars/*

Update values.yaml.
```
vi values.yaml
        
```

Add the following configuration for extraInitContainers inside the coordinator section:

extraInitContainers:  |
    - name: install-privacera-dremio-plugin
    image: {{.Values.image}}:{{.Values.imageTag}}
    imagePullPolicy: IfNotPresent
    securityContext:
        runAsUser: 0
    volumeMounts:
    - name: dremio-privacera-plugin-volume
        mountPath: /opt/dremio/plugins/authorizer
    - name: dremio-ext-jars-volume
        mountPath: /opt/privacera/dremio-ext-jars
    - name: dremio-privacera-config
        mountPath: /opt/privacera/conf/
    - name: dremio-privacera-install
        mountPath: /opt/privacera/install/
    command:
        - "bash"
        - "-c"
        - "cd /opt/privacera/install/ && cp * /tmp/ && cd /tmp && ./privacera_dremio.sh"

Update or uncomment the extraVolumes section inside the coordinator section and add the following configuration:

extraVolumes:
- name: dremio-privacera-install
    configMap:
    name: dremio-privacera-install
    defaultMode: 0777
- name: dremio-privacera-plugin-volume
    emptyDir: {}
- name: dremio-ext-jars-volume
    emptyDir: {}
- name: dremio-privacera-config
    emptyDir: {}

Update or uncomment the extraVolumeMounts section inside the coordinator section and add the following configuration:

extraVolumeMounts:
- name: dremio-ext-jars-volume
    mountPath: /opt/privacera/dremio-ext-jars
- name: dremio-privacera-plugin-volume
    mountPath: /opt/dremio/plugins/authorizer
- name: dremio-privacera-config
    mountPath: /opt/privacera/conf

Upgrade your Helm release. Get the release name by running helm list command. The text under the Name column is your Helm release.
```
helm upgrade -f values.yaml <release-name>
```

RPM

To deploy RPM:

SSH to your instance where Dremio RPM is installed.
Copy the privacera_dremio.sh and privacera_custom_conf.tar.gz files from your Privacera Manager host instance to the Home folder in your Dremio instance.

Rum the following commands:

mkdir -p ~/privacera/install
mv privacera_dremio.sh ~/privacera/install
mv privacera_custom_conf.tar.gz ~/privacera/install

Launch the privacera_dremio.sh script.

cd ~/privacera/install
chmod +x privacera_dremio.sh
sudo ./privacera_dremio.sh

Update dremio-env to add Privacera jars and configuration in the Dremio classpath.
```
vi ${DREMIO_HOME}/conf/dremio-env
```