Self Managed and PrivaceraCloud Data Plane Compute Sizing¶

Compute Sizing¶

This section provides the compute sizing for the Privacera Platform deployment. The compute sizing is based on the deployment size of the Privacera Platform. The deployment size is categorized into three sizes: small, medium, and large. The compute sizing is provided for each pod in the Privacera Platform deployment.

Note

The compute sizing is for reference only. The actual compute sizing may vary based on the workload and the deployment environment. You can adjust the compute sizing based on the workload and the deployment environment.

Small Deployment (PoC)Medium DeploymentLarge Deployment

Pod	Memory	CPU	Disk	Replication Factor
Portal	2GB	0.5	NA	min=1 max=1
Maria DB	1GB	0.5	12
Data Server	2GB	1	NA	min=1 max=1
Discovery - Driver	2GB	1	32
Discovery - Executor	2GB	1	NA
PolicySync	2GB	2	32
Solr	1.5GB	1	64	1
Zookeeper	1GB	0.5	32	1
Ranger KMS	1GB	0.5	12	NA
Ranger UserSync	1GB	0.5	12	NA
Grafana	1GB	0.5	1
Graphite	1GB	0.5	32
Kafka	1GB	0.5	32
PEG	1GB	0.5	NA	min=1 max=2
pkafka	1GB	0.5	NA
Ranger Admin	2GB	1	NA
Audit Server	1GB	1	32
FluentD	1GB	1	32

Pod	Memory	CPU	Disk	Replication Factor
Portal	4GB	2	NA
Maria DB	4GB	2	12
Data Server	8GB	2	NA	min=2 max=4
Discovery - Driver	8GB	4	32
Discovery - Executor	2GB	2	NA
PolicySync	8GB	4	32
Solr	8GB	4	64	3
Zookeeper	2GB	1	32	3
Ranger KMS	2GB	2	12	NA
Ranger UserSync	4GB	2	12	NA
Grafana	4GB	2	1
Graphite	4GB	2	32
Kafka	4GB	2	32
PEG	4GB	2	NA	min=2 max=10
pkafka	4GB	2	NA
Ranger Admin	8GB	4	NA	min=2 max=4
Audit Server	4GB	2	32
FluentD	4GB	2	32

Pod	Memory	CPU	Disk	Replication Factor
Portal	8GB	4	NA
Maria DB	8GB	4	12
Data Server	8GB	2	NA	min=3 max=20
Discovery - Driver	16GB	8	32
Discovery - Executor	4GB	4	NA
PolicySync	32GB	8	32
Solr	32GB	8	64	3
Zookeeper	4GB	2	32	3
Ranger KMS	4GB	4	12	NA
Ranger UserSync	8GB	4	12	NA
Grafana	8GB	4	1
Graphite	8GB	4	32
Kafka	8GB	4	32
PEG	8GB	4	NA	min=3 max=20
pkafka	8GB	4	NA
Ranger Admin	16GB	8	NA	min=2 max=4
Audit Server	16GB	8	32
FluentD	16GB	8	32

Override variables¶

You can override the default compute sizing by setting the following variables in file that you can name as vars.sizing.yaml file:

Pod Heap Memory Min and Max

Values are show as examples of format to be used. These are numbers in MB formatted as YAML Strings.

YAML
AUDITSERVER_HEAP_MIN_MEMORY_MB: "1024"
AUDITSERVER_HEAP_MAX_MEMORY_MB: "4096"

CONNECTOR_HEAP_MIN_MEMORY_MB: "1024"
CONNECTOR_HEAP_MAX_MEMORY_MB: "4096"

DATASERVER_HEAP_MIN_MEMORY_MB: "2048"
DATASERVER_HEAP_MAX_MEMORY_MB: "8192"

DISCOVERY_CONSUMER_HEAP_MIN_MEMORY_MB: "2048"
DISCOVERY_CONSUMER_HEAP_MAX_MEMORY_MB: "8192"

DISCOVERY_DRIVER_HEAP_MIN_MEMORY_MB: "2048"
DISCOVERY_DRIVER_HEAP_MAX_MEMORY_MB: "8192"

DISCOVERY_EXECUTOR_HEAP_MIN_MEMORY_MB: "2048"
DISCOVERY_EXECUTOR_HEAP_MAX_MEMORY_MB: "8192"

KAFKA_HEAP_MIN_MEMORY_MB: "1024"
KAFKA_HEAP_MAX_MEMORY_MB: "4096"

OPS_SERVER_HEAP_MIN_MEMORY_MB: "1024"
OPS_SERVER_HEAP_MAX_MEMORY_MB: "4096"

PEG_HEAP_MIN_MEMORY_MB: "1024"
PEG_HEAP_MAX_MEMORY_MB: "4096"

PORTAL_HEAP_MIN_MEMORY_MB: "2048"
PORTAL_HEAP_MAX_MEMORY_MB: "8192"

# GDS backend in Platform  
PRIVACERA_SERVICES_HEAP_MIN_MEMORY_MB: "2048"
PRIVACERA_SERVICES_HEAP_MAX_MEMORY_MB: "8192"

RANGER_HEAP_MIN_MEMORY_MB: "2048"
RANGER_HEAP_MAX_MEMORY_MB: "8192"

SCHEME_SERVER_HEAP_MIN_MEMORY_MB: "2048"
SCHEME_SERVER_HEAP_MAX_MEMORY_MB: "8192"

SOLR_HEAP_MIN_MEMORY_MB: "2048"
SOLR_HEAP_MAX_MEMORY_MB: "8192"

USERSYNC_HEAP_MIN_MEMORY_MB: "1024"
USERSYNC_HEAP_MAX_MEMORY_MB: "4096"

ZOOKEEPER_HEAP_MIN_MEMORY_MB: "1024"
ZOOKEEPER_HEAP_MAX_MEMORY_MB: "4096"

Pod CPU Min and Max

Values are show as examples of format to be used. These are numbers in CPU units formatted as YAML Strings.

YAML
AUDITSERVER_CPU_MIN: "0.5"
AUDITSERVER_CPU_MAX: "1.0"

CONNECTOR_CPU_MIN: "1"
CONNECTOR_CPU_MAX: "4"

DATASERVER_CPU_MIN: "2"
DATASERVER_CPU_MAX: "8"

DISCOVERY_DRIVER_CPU_MIN: "2"
DISCOVERY_DRIVER_CPU_MAX: "8"

DISCOVERY_EXECUTOR_CPU_MIN: "2"
DISCOVERY_EXECUTOR_CPU_MAX: "8"

KAFKA_CPU_MIN: "1"
KAFKA_CPU_MAX: "4"

OPS_SERVER_CPU_MIN: "1"
OPS_SERVER_CPU_MAX: "4"

PEG_CPU_MIN: "1"
PEG_CPU_MAX: "4"

PORTAL_CPU_MIN: "2"
PORTAL_CPU_MAX: "8"

PRIVACERA_SERVICES_CPU_MIN: "2"
PRIVACERA_SERVICES_CPU_MAX: "8"

RANGER_CPU_MIN: "2"
RANGER_CPU_MAX: "8"

SCHEME_SERVER_CPU_MIN: "2"
SCHEME_SERVER_CPU_MAX: "8"

SOLR_CPU_MIN: "2"
SOLR_CPU_MAX: "8"

USERSYNC_CPU_MIN: "1"
USERSYNC_CPU_MAX: "4"

ZOOKEEPER_CPU_MIN: "1"
ZOOKEEPER_CPU_MAX: "4"

Pod Replicas Min and Max

Values are show as examples of format to be used. These are numbers formatted as YAML Strings. Min and Max is when the workload has a Kubernetes HPA configured otherwise it is only the replica count.

YAML
AUDIT_FLUENTD_K8S_REPLICAS: "1"
AUDITSERVER_K8S_REPLICAS: "1"

DISCOVERY_CONSUMER_K8S_REPLICAS_MIN: "1"
DISCOVERY_CONSUMER_K8S_REPLICAS_MAX: "4"

PROMETHEUS_DEPLOYMENT_REPLICAS: "1"
PROMETHEUS_STATEFULSET_REPLICAS: "1"

GRAFANA_DEPLOYMENT_REPLICAS: "1"

PORTAL_K8S_REPLICAS: "1"
DIAG_SERVER_K8S_REPLICAS: "1"
PRIVACERA_SERVICES_K8S_REPLICAS: "1"

RANGER_K8S_REPLICAS: "1"

GRAFANA_DEPLOYMENT_REPLICAS: "1"
GRAFANA_AUTOSCALING_HPA_MIN_REPLICA: "1"
GRAFANA_AUTOSCALING_HPA_MAX_REPLICA: "2"

OPS_SERVER_REPLICAS_MIN: "1"
OPS_SERVER_REPLICAS_MAX: "4"

PEG_REPLICAS_MIN: "1"
PEG_REPLICAS_MAX: "4"

SCHEME_SERVER_REPLICAS_MIN: "1"
SCHEME_SERVER_REPLICAS_MAX: "4"

Pod Storage Size for EBS

Values are show as examples of format to be used. These are numbers in MB formatted as YAML Strings when the variable ends in _MB. And for other variables the value format is Kubernetes disk units.

YAML
# The value is in Kubernetes disk format
AUDIT_FLUENTD_K8S_PVC_STORAGE_SIZE: "10Gi"

# The value is in Kubernetes disk format 
AUDITSERVER_K8S_PVC_STORAGE_SIZE: "10Gi"

CONNECTOR_K8S_PVC_STORAGE_SIZE_MB: "1024"
CONNECTOR_ROCKSDB_K8S_PVC_STORAGE_SIZE_MB: "11264"

DATASERVER_K8S_PVC_STORAGE_SIZE_MB: "1024"

DISCOVERY_K8S_MAPDB_PVC_STORAGE_SIZE_MB: "2048"

# The value is in Kubernetes disk format
KAFKA_K8S_PVC_STORAGE_SIZE: "10Gi"

DB_MARIADB_K8S_PVC_STORAGE_SIZE_MB: "2048"

OPS_SERVER_K8S_PVC_STORAGE_SIZE_MB: "1024"

PEG_K8S_PVC_STORAGE_SIZE_MB: "1024"

PORTAL_K8S_PVC_STORAGE_SIZE_MB: "1024"

DIAG_SERVER_K8S_PVC_STORAGE_SIZE_MB: "1024"

PRIVACERA_USERSYNC_K8S_PVC_STORAGE_SIZE_MB: "1024"

PRIVACERA_USERSYNC_ROCKSDB_K8S_PVC_STORAGE_SIZE_MB: "11264"

# The value is in Kubernetes disk format
SOLR_K8S_PVC_STORAGE_SIZE: "5G"

ZOOKEEPER_K8S_PVC_STORAGE_SIZE_MB: "5120"

Self Managed and PrivaceraCloud Data Plane Compute Sizing¶

Compute Sizing¶

Override variables¶

Comments