Dremio
This section covers how you can integrate Dremio with Privacera. You can use Dremio for table-level access control with native Ranger plugin.
By integrating Dremio with Privacera, you'll be provided with comprehensive data lake security and fine-grained access control across multi-cloud environments. Dremio works directly with data lake storage. Using Dremio's query engine and ability to democratize data access, Privacera implements fine-grained access control policies, then automatically enforces and audits them at enterprise scale.
Dremio is supported with the following data sources:
- S3
- ADLS
- Hive
- Redshift
Prerequisite
Ensure the following prerequisite is met:
-
A Privacera Manager host where Privacera services are running.
-
A Dremio host where Dremio Enterprise Edition is installed. (Community Edition is not supported)
Configuration
-
Run the following commands.
cd ~/privacera/privacera-manager cp config/sample-vars/vars.dremio.yml config/custom-vars/
-
Run the following commands.
cd ~/privacera/privacera-manager ./privacera-manager.sh update
After the update is completed, the Dremio plugin installation script
privacera_dremio.sh
and custom configuration archiveprivacera_custom_conf.tar.gz
is generated at the location ~/privacera/privacera-manager/output/dremio. -
Configure Privacera plugin depending on how you have installed Dremio in your instance.
Note
For a new/existing data source configured in Dremio Data Lake, ensure Enable external authorization plugin checkbox under Settings > Advanced Options of the data source is selected in the Dremio UI. Then, restart the Dremio service.
Depending on your cloud provider, set up Dremio in a Kubernetes environment. See the following links:
After setting up Dremio, perform the following steps to deploy Privacera plugin. The steps assume that your Privacera Manager host instance is separate from your Dremio Kubernetes instance. If they are configured on the single instance, then modify the steps accordingly.
-
SSH to your instance where Dremio is installed containing the Dremio Kubernetes artifacts and change to the dremio-cloud-tools/charts/dremio_v2/ directory.
-
Copy the
privacera_dremio.sh
andprivacera_custom_conf.tar.gz
files from your Privacera Manager host instance to the dremio_v2 folder in your Dremio Kubernetes instance. -
Run the following commands:
mkdir -p privacera_config mv privacera_dremio.sh privacera_config/ mv privacera_custom_conf.tar.gz privacera_config/
-
Update
configmap.yml
to add new configmap for Privacera configuration.vi templates/dremio-configmap.yaml
Add the following configuration at the start of the file:
apiVersion: v1 kind: ConfigMap metadata: name: dremio-privacera-install data: privacera_dremio.sh: |- {{ .Files.Get "privacera_config/privacera_dremio.sh" | nindent 4 }} binaryData: privacera_custom_conf.tar.gz: {{ .Files.Get "privacera_config/privacera_custom_conf.tar.gz" | b64enc | nindent 4 }} ---
-
Update
dremio-env
to add Privacera jars and configuration in the Dremio classpath.vi config/dremio-env
Update the following variable if it exists or add it.
DREMIO_EXTRA_CLASSPATH=/opt/privacera/conf:/opt/privacera/dremio-ext-jars/*
-
Update
values.yaml
.vi values.yaml
Add the following configuration for extraInitContainers inside the coordinator section.
extraInitContainers: | - name: install-privacera-dremio-plugin image: {{.Values.image}}:{{.Values.imageTag}} imagePullPolicy: IfNotPresent securityContext: runAsUser: 0 volumeMounts: - name: dremio-privacera-plugin-volume mountPath: /opt/dremio/plugins/authorizer - name: dremio-ext-jars-volume mountPath: /opt/privacera/dremio-ext-jars - name: dremio-privacera-config mountPath: /opt/privacera/conf/ - name: dremio-privacera-install mountPath: /opt/privacera/install/ command: - "bash" - "-c" - "cd /opt/privacera/install/ && cp * /tmp/ && cd /tmp && ./privacera_dremio.sh"
Update or uncomment the extraVolumes section inside the coordinator section and add the following configuration:
extraVolumes: - name: dremio-privacera-install configMap: name: dremio-privacera-install defaultMode: 0777 - name: dremio-privacera-plugin-volume emptyDir: {} - name: dremio-ext-jars-volume emptyDir: {} - name: dremio-privacera-config emptyDir: {}
Update or uncomment the extraVolumeMounts section inside the coordinator section and add the following configuration:
extraVolumeMounts: - name: dremio-ext-jars-volume mountPath: /opt/privacera/dremio-ext-jars - name: dremio-privacera-plugin-volume mountPath: /opt/dremio/plugins/authorizer - name: dremio-privacera-config mountPath: /opt/privacera/conf
-
Upgrade your Helm release. Get the release name by running
helm list
command. The text under the Name column is your Helm release.helm upgrade -f values.yaml <release-name>
-
SSH to your instance where Dremio RPM is installed.
-
Copy the
privacera_dremio.sh
andprivacera_custom_conf.tar.gz
files from your Privacera Manager host instance to the Home folder in your Dremio instance. -
Run the following commands:
mkdir -p ~/privacera/install mv privacera_dremio.sh ~/privacera/install mv privacera_custom_conf.tar.gz ~/privacera/install
-
Launch the
privacera_dremio.sh
script.cd ~/privacera/install chmod +x privacera_dremio.sh sudo ./privacera_dremio.sh
-
Update
dremio-env
to add Privacera jars and configuration in the Dremio classpath.vi ${DREMIO_HOME}/conf/dremio-env
Update the following variable if it exists or add it.
DREMIO_EXTRA_CLASSPATH=/opt/privacera/conf:/opt/privacera/dremio-ext-jars/*
-
Restart Dremio.
sudo service dremio restart
-