DataServer Scaling Guide¶

Overview¶

This guide provides guidelines on when and how to scale Privacera DataServer for S3 deployments, including scaling thresholds, configuration best practices, and operational considerations to ensure efficient performance and stability.

DataServer Architecture¶

For detailed information, please refer to the Privacera DataServer documentation

Prerequisites¶

Before scaling DataServer, ensure the following:

• Kubernetes cluster (EKS, AKS, or GKE) is deployed
• kubectl access is configured with appropriate permissions
• Kubernetes version meets the requirements. Refer to Compatibility & Versions for supported versions
• Privacera Manager is installed
• AWS S3 prerequisites (required only for DataServer with S3): IAM role configured with S3 access permissions. Refer to AWS S3 prerequisites
• Service Account with IAM Role (for AWS deployments): DataServer service account has IAM role annotation configured for accessing AWS services

Scaling Rationale for DataServer¶

Despite STS reducing data plane load, DataServer must scale for:

Scaling Approaches¶

Manual Replica Scaling¶

DataServer uses manual scaling by configuring the number of pods:

DATASERVER_K8S_CLUSTER_SIZE: "<NUMBER_OF_DATASERVER_POD>" 

• How it works
  • Set desired replica count in configuration
  • Apply using Privacera Manager
  • Kubernetes maintains specified number of pods

When to Scale¶

• Scale Horizontally (Add Replicas):

  • CPU consistently > 70%
  • High token generation rate
  • Many concurrent users
  • Request latency increasing

• Scale Vertically (More Resources per Pod):

  • Complex policy evaluations
  • Large policy cache needed
  • Individual pods hitting limits

Configuration Best Practices¶

All DataServer configuration variables should be added to the vars.sizing.yaml file in your Privacera Manager config directory.

Core Configuration Variables¶

• Key Principles:
  • Heap = 75% of container memory to leave room for non-heap JVM memory.
  • CPU min/max ratio allows bursting but prevents unlimited usage.

Deployment Size Templates¶

• Update Configuration

  Edit ~/privacera/privacera-manager/config/custom-vars/vars.sizing.yaml based on the configuration guidelines provided in the sections below.

  1. Test Environment (Small Scale)

     vars.sizing.yaml

     DATASERVER_K8S_CLUSTER_SIZE: "1"
     DATASERVER_HEAP_MIN_MEMORY_MB: "1536"
     DATASERVER_HEAP_MAX_MEMORY_MB: "2048"
     DATASERVER_CPU_MIN: "1"
     DATASERVER_CPU_MAX: "1"
     

  2. Production (Medium Scale)

     DATASERVER_K8S_CLUSTER_SIZE can be scaled between 2 to 4.

     vars.sizing.yaml

     DATASERVER_K8S_CLUSTER_SIZE: "2"
     DATASERVER_HEAP_MIN_MEMORY_MB: "8192"
     DATASERVER_HEAP_MAX_MEMORY_MB: "8192"
     DATASERVER_CPU_MIN: "2"
     DATASERVER_CPU_MAX: "4"
     

  3. Production (Large Scale)

     DATASERVER_K8S_CLUSTER_SIZE can be scaled between 3 to 20.

     vars.sizing.yaml

     DATASERVER_K8S_CLUSTER_SIZE: "4"
     DATASERVER_HEAP_MIN_MEMORY_MB: "8192"
     DATASERVER_HEAP_MAX_MEMORY_MB: "8192"
     DATASERVER_CPU_MIN: "4"
     DATASERVER_CPU_MAX: "8"
     

• Validation Checklist

  • All pods in Running state
  • All pods show 1/1 Ready
  • CPU utilization 50-65% average
  • Memory utilization 60-75%

Grafana Monitoring and Alerts¶

Monitor DataServer Dashboard¶

Use the Privacera Grafana monitoring dashboards to track DataServer resource utilization:

1. Access Grafana Dashboard

   • Navigate to: Dashboards → Infra-Dashboards → Cluster RAM and CPU Utilization

2. View DataServer Metrics

   • The dashboard provides the following views:

   Dashboard View	What to Check	Focus On
   Memory usage by namespace	Overall namespace memory	Identify if namespace hitting limits
   Memory usage by pod	Individual DataServer pod memory	Find which pods need attention
   CPU Usage by namespace	Overall namespace CPU	Track aggregate CPU consumption
   CPU Usage by pod	Individual DataServer pod CPU	Identify high CPU pods

3. Interpret Dashboard Colors

   • The dashboard uses a gradient color scale to indicate resource usage intensity:

   Color Indicator	Usage Level	What It Means	Action Required
   Light Yellow	Low usage	Resources are healthy	Monitor regularly
   Yellow-Orange	Moderate to high usage	Resources approaching limits	Plan scaling soon
   Red-Dark Red	Critical usage	Resources at or near capacity	Take immediate scaling action

Monitor DataServer Alerts¶

Privacera monitoring includes pre-configured alerts that automatically notify when DataServer pods exceed resource thresholds. These alerts help identify when scaling is needed.

1. Open your Grafana portal

   • Navigate to: Home → Alerting → Alert rules

2. Identify DataServer Alert Types

   In the Alert rules page, look for the following alert rules in the Common-Dashboards folder:

   • pod_cpu_usage
   • pod_memory_usage
   • node_cpu_usage
   • node_memory_usage
   • pod-container-not-ready-alert
   • pvc_usage

3. When an alert fires for DataServer

   • Alert identifies the issue - You receive notification that a threshold is breached.
   • Check the dashboard for details - Go to the Cluster RAM and CPU Utilization dashboard in Grafana.
   • For troubleshooting steps - Refer to Resource Utilization Issues section.