Performance Optimization for Diagnostics Server¶

Prerequisites¶

Before tuning, ensure you have:

• A running Privacera Diagnostics Server deployment.
• Access to the vars.privacera-diagnostics.yml configuration file.

Choose a Database Backend¶

The server supports two backends. Choose before any other tuning — the active backend determines which sections below apply.

SQLite is the default and requires zero external infrastructure. It is the right choice for single-server deployments on local or EBS-backed storage.

MariaDB is required when:

• You run more than one Diagnostics Server replica.
• The storage volume is NFS or EFS (SQLite WAL does not work correctly over NFS).

1
2
3
4
5

# Use SQLite (default — no change needed)
# DIAG_SERVER_DB_TYPE: "sqlite"

# Switch to MariaDB
DIAG_SERVER_DB_TYPE: "mariadb"

SQLite Performance Tuning¶

Skip this section if you are using MariaDB.

SQLite performance is dominated by three factors: lock-wait tolerance, WAL checkpoint frequency, and how many requests can touch the database at the same time.

Lock Timeout¶

1
2

# Raise only when the SQLite file is on EFS/NFS
DIAG_SERVER_SQLITE_BUSY_TIMEOUT_MS: "60000"

WAL Checkpoint Frequency¶

1
2

# Reduce checkpoint frequency on EFS to avoid lock spikes
DIAG_SERVER_SQLITE_WAL_AUTOCHECKPOINT: "2000"

Concurrent Session Limit¶

The auto-calculation is: min(32, cpu_count + 4) - 2. This reserves 2 slots for readiness probes.

1
2

# Override only when semaphore-wait warnings appear in the logs
# DIAG_SERVER_SQLITE_MAX_CONCURRENT_SESSIONS: "12"

Readiness Probe Sensitivity¶

The pod reports Not Ready only when all of the last N sessions failed. Raising this value makes the readiness probe more tolerant of transient SQLite lock errors.

1

DIAG_SERVER_SQLITE_READINESS_WINDOW_SIZE: "5"

In-Memory Heartbeat Queue¶

Heartbeats bypass the concurrency semaphore and queue in memory. The pod returns 503 only when this queue fills up (sustained DB outage), not on transient slowness.

1

DIAG_SERVER_SQLITE_HEARTBEAT_QUEUE_MAX_SIZE: "500"

MariaDB Configuration¶

Skip this section if you are using SQLite.

Connection Details¶

1
2
3
4
5
6

DIAG_SERVER_DB_TYPE: "mariadb"
DIAG_SERVER_DB_HOST: "mariadb"
DIAG_SERVER_DB_PORT: "3306"
DIAG_SERVER_DB_USER: "<DB_USERNAME>"
DIAG_SERVER_DB_PASSWORD: "<DB_PASSWORD>"
DIAG_SERVER_DB_NAME: "diag_server"

Connection Pool¶

These settings control how many simultaneous database connections the server maintains. They apply only when DIAG_SERVER_DB_TYPE is mariadb — SQLite uses NullPool (no persistent connections).

1
2
3

DIAG_SERVER_DB_POOL_CAPACITY: "20"
DIAG_SERVER_DB_POOL_OVERFLOW: "30"
DIAG_SERVER_DB_POOL_TIMEOUT_SECS: "10"

SocketIO Performance¶

SocketIO settings affect real-time communication between the server and browser clients. Tune these when clients experience dropped connections or high-latency updates.

1
2
3
4
5

# Production-tuned SocketIO settings
DIAG_SERVER_SOCKETIO_PING_TIMEOUT: "120"
DIAG_SERVER_SOCKETIO_PING_INTERVAL: "15"
DIAG_SERVER_SOCKETIO_LOG_LEVEL: "ERROR"
DIAG_SERVER_SOCKETIO_ENGINEIO_LOG_LEVEL: "ERROR"

Data Purge and Retention¶

Background purge tasks remove stale records and prevent unbounded database growth. Schedule purges during off-peak hours to avoid I/O contention with live traffic.

Server-Side Purge Schedule¶

1
2
3
4
5

DIAG_SERVER_DATA_PURGE_ENABLED: "true"
DIAG_SERVER_DATA_PURGE_RUN_TIME: "00:00"
DIAG_SERVER_DATA_PURGE_ERROR_LOGS_RETENTION_DAYS: "7"
DIAG_SERVER_DATA_PURGE_POD_TEST_RESULT_RETENTION_DAYS: "30"
DIAG_SERVER_DATA_PURGE_RETRY_INTERVAL_SECONDS: "600"

Client-Side Test Result File Retention¶

These variables control how many historical test-result files the diagnostics client retains on disk before rotating.

Naming pattern: DIAG_TEST_RESULTS_RETENTION_{CONTAINER_NAME_IN_UPPERCASE}.

1
2
3
4
5

# Retain the last 10 result files for all services
DIAG_TEST_RESULTS_FILE_RETENTION_COUNT: "10"

# Override for a noisy connector that generates many results
DIAG_TEST_RESULTS_RETENTION_DATABRICKS_UNITY_CATALOG: "5"

Diagnostics Test Scheduling¶

These variables control when and how often the client sidecar runs its diagnostic test suite against each service.

1
2
3
4

DIAG_TESTS_INITIAL_WAIT_SECS: "120"
DIAG_TESTS_INTERVAL_SECS: "86400"
DIAG_TESTS_RETRY_ENABLED: "true"
DIAG_TESTS_RETRY_INTERVAL_SECS: "3600"

Apply the Changes¶

Save and close the file, then regenerate and apply the Helm charts:

1
2
3

cd ~/privacera/privacera-manager
./privacera-manager.sh setup
./pm_with_helm.sh upgrade

Quick Reference — Recommended Configurations¶