Skip to content

Configure Additional Destinations for Telemetry data

Introduction

OpenTelemetry (OTEL) allows exporting telemetry data — Metrics, Logs, and Traces — to remote backends like Prometheus, Loki, and Tempo.

This guide explains how to configure custom export targets in the vars.monitoring.custom.targets.yml file within the Privacera monitoring stack.

Prerequisites

Prerequisite Description
Accessible Endpoints All remote write endpoint URLs must be accessible from the Privacera monitoring stack
TLS Certificates Custom TLS certificates are currently not supported. Only use http or https endpoints that do not require custom certificate validation
Unique Names Each endpoint must have a unique name when defining multiple custom endpoints for each data category

Exporting Metrics (Only for Prometheus-Compatible Datasources)

Prerequisite Description
Prometheus Remote Write The remote write endpoint must be enabled and configured in your Prometheus server to receive metrics from external sources

Procedure

  1. SSH into the instance where Privacera Manager is installed.

  2. Navigate to the configuration directory:

    Bash
    1
    cd ~/privacera/privacera-manager/config/

  3. Copy the monitoring vars file if not already copied:

    If this file already exists in custom-vars folder then you can skip this step.

    Bash
    1
    cp sample-vars/vars.monitoring.custom.targets.yml custom-vars/

  4. Open the configuration file:

    Bash
    1
    vi custom-vars/vars.monitoring.custom.targets.yml

  5. Uncomment and configure the OTEL_METRICS_EXPORT_TARGETS block:

    YAML
    1
2
3
4
5
6
7
8
9
    OTEL_METRICS_EXPORT_TARGETS:
  - name: custom-metrics1
    properties:
      endpoint: "http://your-prometheus-endpoint:9090/api/v1/write"
      resource_to_telemetry_conversion_enabled: "true"
      remote_write_queue_enabled: "true"
      remote_write_queue_size: 50000
      max_batch_size: 5000000
      timeout: 30s

  6. Save the file and exit the editor.

Supported Properties

Property Description
endpoint Remote write URL
resource_to_telemetry_conversion_enabled Converts resource info into telemetry metadata
remote_write_queue_enabled Enables queueing for remote write
remote_write_queue_size Size of the write queue
max_batch_size Max size per batch in bytes
timeout Timeout duration (e.g., 30s)

Exporting Logs (for Loki-Compatible Data Sources)

Procedure

  1. Open the same configuration file:

    Bash
    1
    vi custom-vars/vars.monitoring.custom.targets.yml

  2. Uncomment and configure the OTEL_LOGS_EXPORT_TARGETS block:

    YAML
    1
2
3
4
    OTEL_LOGS_EXPORT_TARGETS:
  - name: custom-logs1
    properties:
      endpoint: "http://your-loki-endpoint:3100/loki/api/v1/push"

  3. Save and exit the file.

Supported Properties

Property Description
endpoint URL to push logs to Loki

Exporting Traces (Compatible with Tempo Trace Format)

Prerequisites

Prerequisite Description
Endpoint Protocol gRPC-based endpoints are currently not supported. Use OTLP over HTTP (http:// or https://) for successful trace export

Procedure

  1. Open the same configuration file:

    Bash
    1
    vi custom-vars/vars.monitoring.custom.targets.yml

  2. Uncomment and configure the OTEL_TRACES_EXPORT_TARGETS block:

    YAML
    1
2
3
4
    OTEL_TRACES_EXPORT_TARGETS:
  - name: custom-traces1
    properties:
      endpoint: "http://your-tempo-endpoint:4318"

  3. Save and exit the file.

Supported Properties

Property Description
endpoint http endpoint for Tempo traces

Redeploy Monitoring Components

Once you've added all desired custom endpoints in vars.monitoring.custom.targets.yml, apply the configuration by redeploying the monitoring stack:

a. Go to privacera-manager directory.

Bash
1
cd ~/privacera/privacera-manager.
b. Run setup to generate the required files.
Bash
1
./privacera-manager.sh setup
c. Install the monitoring components.
Bash
1
./pm_with_helm.sh install-monitoring
d. Run install to update the datasources in Grafana.
Bash
1
./pm_with_helm.sh install
e. Once done, run post-install.
Bash
1
./privacera-manager.sh post-install

Troubleshooting

If telemetry data is not exporting as expected, check the following:

  1. Endpoint Accessibility: Ensure all configured endpoint URLs are reachable from within the OTEL Receiver, Scraper, or Agent pod/container.

  2. Unique Target Names:Confirm that each export target has a unique name to avoid conflicts.

  3. YAML Syntax: Validate the YAML file for correct indentation and syntax. Improper formatting can prevent configuration from loading.

  4. Check Logs: Review the logs of the OTEL Receiver, Scraper, or Agent for any error messages related to export failures.

Summary

By enabling and configuring custom export targets for Metrics, Logs, and Traces, you can forward telemetry data to any supported remote backend system without changing the core OTEL configuration — just update the vars.monitoring.custom.targets.yml file.

Comments