Skip to content

Scan-Failure Safeguards

These safeguards ensure that an incomplete or failed read from Collibra is never mistaken for "all tags were removed," which would otherwise wipe valid tag mappings out of Ranger or MDS. Deletes are only applied when a scan passes every safeguard.

Configurable safeguards

PolicySync Property Privacera Manager Variable Description Default
tag.sync.suspicious.delete.drop.pct CONNECTOR_TAG_SYNC_SUSPICIOUS_DELETE_DROP_PCT Abort tag-sync deletes when the number of assets seen in a scan drops by more than this percentage compared with the previous successful run. Protects against partial scans (pagination cutoff, transient errors) deleting valid mappings. Set to 0 to disable the partial-scan check. 50
collibra.tag.sync.circuit.failure.threshold CONNECTOR_COLLIBRA_TAG_SYNC_CIRCUIT_FAILURE_THRESHOLD Number of consecutive scan failures before the circuit breaker opens and tag sync is paused. 3
collibra.tag.sync.circuit.open.duration.sec CONNECTOR_COLLIBRA_TAG_SYNC_CIRCUIT_OPEN_DURATION_SEC How long the circuit stays open before a single probe scan is allowed, in seconds. 300
collibra.tag.sync.circuit.probe.timeout.sec CONNECTOR_COLLIBRA_TAG_SYNC_CIRCUIT_PROBE_TIMEOUT_SEC How long to wait for a half-open probe scan to complete before treating it as lost and allowing a new probe, in seconds. 600

Built-in guards (no configuration)

In addition to the configurable options above, the connector applies two guards automatically:

  • Zero-row guard — if a scan returns no tagged assets at all but the previous successful run found some, the scan is rejected as incomplete and no deletes are applied.
  • Vanished-split guard — if a particular connection/engine combination that had tagged assets last time returns none this time, that scan is rejected as incomplete.

The state used by these guards is persisted to disk next to the connector's local store, so the safeguards continue to work correctly across connector restarts.

Configuration

  1. SSH to the instance where Privacera Manager is installed.

  2. Run the following command to open the .yml file to be edited.

    If you have multiple connectors, then replace instance1 with the appropriate connector instance name.

    Bash
    1
    vi ~/privacera/privacera-manager/config/custom-vars/connectors/collibra/instance1/vars.connector.collibra.yml

  3. Update or add the following properties:

    YAML
    1
2
3
4
    CONNECTOR_TAG_SYNC_SUSPICIOUS_DELETE_DROP_PCT: "50"
CONNECTOR_COLLIBRA_TAG_SYNC_CIRCUIT_FAILURE_THRESHOLD: "3"
CONNECTOR_COLLIBRA_TAG_SYNC_CIRCUIT_OPEN_DURATION_SEC: "300"
CONNECTOR_COLLIBRA_TAG_SYNC_CIRCUIT_PROBE_TIMEOUT_SEC: "600"

  4. Once the properties are configured, run the following commands to update your Privacera Manager platform instance:

    Step 1 - Setup which generates the helm charts. This step usually takes few minutes.

    Bash
    1
2
    cd ~/privacera/privacera-manager
./privacera-manager.sh setup
    Step 2 - Apply the Privacera Manager helm charts.
    Bash
    1
2
    cd ~/privacera/privacera-manager
./pm_with_helm.sh upgrade
    Step 3 - (Optional) Post-installation step which generates Plugin tar ball, updates Route 53 DNS and so on. This step is not required if you are updating only connector properties.

    Bash
    1
2
    cd ~/privacera/privacera-manager
./privacera-manager.sh post-install