Send on-demand sync audits to S3, SQS, or SNS¶

Overview¶

When a PolicySync connector is configured with on-demand resource sync, it can publish one ON_DEMAND audit event per requested resource to the policysync_audit stream. Each event carries that resource's load outcome (found, added, updated, deleted, or not-found), the policies applied during the sync, and the request context.

These events flow through the same Audit Server path as every other connector audit. Once you turn the feature on for the connector, the ON_DEMAND events reach whichever destinations you have already configured for the policysync_audit stream — Apache Solr, Amazon S3 (through Audit Fluentd), Amazon SQS, and Amazon SNS (through an SNS→SQS subscription).

Because the stream is indexed into Apache Solr, the events are also visible in Privacera Portal alongside your other audit records.

The feature is opt-in and controlled by a single connector setting. When it is off, on-demand syncs still run and apply grants exactly as before — only the per-resource ON_DEMAND audit is not produced.

There are two parts to this configuration:

1. Point the policysync_audit stream at S3, SQS, or SNS using the standard destination configuration (Choose your destinations).
2. Enable the ON_DEMAND audit on the connector so the events are produced (Setup).

Prerequisites¶

On-demand resource sync is configured and working on the connector, so the connector is polling the Ops Server for RESOURCE_SYNC tasks.

Choose your destinations¶

Choose where the audits go before you turn the feature on, so the events have somewhere to land as soon as the connector starts producing them.

ON_DEMAND events travel on the policysync_audit stream, so they follow the destinations you set for that stream. There is no separate on-demand destination list — configure the outputs once, and the on-demand events are included automatically.

You also need AUDITSERVER_POLICYSYNC_AUDIT_HANDLER_V2_ENABLED: true. See Connector Audit Output Destinations.

Setup¶

Follow these steps on the host where Privacera Manager is installed.

1. Sign in to that host.
2. Open the Manager configuration directory:

   Bash
   1	cd ~/privacera/privacera-manager/config

3. Copy the connector sample variables into custom-vars if you have not already. The -n flag avoids overwriting an existing file. Replace the file name with the one for your connector (for example, vars.connector.databricks.unity.catalog.yml).

   Bash
   1	cp -n sample-vars/vars.connector.databricks.unity.catalog.yml custom-vars/

4. Edit the connector custom variables file:

   Bash
   1	vi custom-vars/vars.connector.databricks.unity.catalog.yml

5. Enable the on-demand audit by adding (or updating) the following property. This renders the connector property policysync.on-demand.audit.enable=true.

   Property	Purpose	Privacera Manager default
   CONNECTOR_ON_DEMAND_AUDIT_ENABLE	Emit one ON_DEMAND audit event per requested resource, on the policysync_audit stream	false

   Leaving this false only suppresses the audit. Resource load, grant application, and the Ops Server task-status flow behave the same either way.

1

CONNECTOR_ON_DEMAND_AUDIT_ENABLE: "true"

1. Confirm on-demand processing is enabled for the connector (CONNECTOR_ON_DEMAND_PROCESSING_ENABLE: "true"); the audit is only produced when on-demand syncs actually run.
2. Apply the configuration with Privacera Manager, then restart the connector.

1
2
3

cd ~/privacera/privacera-manager
./privacera-manager.sh setup
./pm_with_helm.sh upgrade 

Event format¶

ON_DEMAND is one more eventType on the policysync_audit stream, so it uses the same wire contract as every other PolicySync connector event: one JSON object per event, UTF-8, camelCase property names, and UPPERCASE enum-like string values. Unknown root or extraInfo keys may appear in future releases — ignore or store them unless you explicitly handle them. See Policy Tracing and Explainability for the shared contract and the other event types.

The same payload shape is indexed into Apache Solr and forwarded to Amazon S3, Amazon SQS, and Amazon SNS.

Envelope fields¶

An ON_DEMAND event fixes several envelope fields to constant values, because it is always produced by the resource loader running from an on-demand task.

A few of these fields need more than a one-line description:

• eventType — a scheduled resource load reports LOADER instead, so filtering on ON_DEMAND selects only the syncs that were explicitly requested.
• status — the outcome for this one resource, not for the whole request. One request can produce a mix of COMPLETED and FAILED events. See Outcome codes.
• msg — comes verbatim from the outcome code rather than being assembled at run time, so it is safe to show to a user. It never contains the target service's raw error; that is in errorDetails.serviceError. The wording can change between releases, so code that branches on the outcome should read errorCode instead.
• timeTakenMs — measured from the start of the resource load to the moment the event is produced.

Output: what the sync did¶

The extraInfo.output object describes what happened to this resource.

When a field is empty rather than absent:

• resources is {} when the resource was not found, when the load failed, or when the request could not be matched to a single resource.
• policies is [] when this resource did not change, because no grants were applied to it.
• aclsExecuted is absent when the resource did not change. It is present on successful events too, not only on failures.
• errorDetails is absent on a success, because there is nothing to report.

Resource counts¶

The value under output.resources.<type>. Each count is 0 or 1, because one event describes exactly one requested resource.

The policies that were applied¶

Each entry of output.policies:

ACL totals for this resource¶

The output.aclsExecuted object adds up the ACL operations for this resource across all of its policies. Read the counts from here whatever the outcome, instead of adding up policies on success and reading the failure block on failure.

Absent when the resource did not change, because no grants ran for it.

Why the outcome failed¶

The output.errorDetails object explains a failure. It never appears on a successful outcome, and it does not appear on every failed one either — a resource that never reached grant application has no cause to report. When it is present, it carries only the parts that apply.

Which of the two appear depends on the outcome — see Outcome codes. topReasons is a summary; the full per-policy breakdown stays in policies[].failedReasons. ACL counts are not here — they live in ACL totals.

Input: the original request¶

The extraInfo.input object echoes back the request this event answers, so you can match an event to its request without calling the Ops Server.

The source value is the OPS_BRIDGE_SOURCE name configured for the connector, so it names the path the request came in through — the REST API, MSK, or another configured source. See On-Demand Sync for PolicySync Connectors.

Each element of input.resources[]:

Inside input.resources[].values:

These two fields are per resource, not per request. If one request asks for several resources on behalf of different people, each event carries its own requestor.

Sample event¶

One resource (table1) added by on-demand task 6, with one policy applied:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71

{
  "eventType": "ON_DEMAND",
  "evtTime": "2026-07-28T07:14:35Z",
  "timeTakenMs": 2140,
  "status": "COMPLETED",
  "msg": "Resource synced; access grants applied",
  "connectorType": "databricks_unity_catalog",
  "connector": "databricks-unity-catalog-instance1",
  "changeLogId": 18472,
  "changeLogType": "SERVICE_RESOURCES",
  "loaderType": "LOAD_SERVICE_RESOURCES",
  "triggerFrom": "EVENT",
  "ondemandTaskId": 6,
  "extraInfo": {
    "output": {
      "resources": {
        "table": {
          "found": 1,
          "added": 1,
          "updated": 0,
          "deleted": 0
        }
      },
      "policies": [
        {
          "id": "160",
          "version": "12",
          "label": "",
          "name": "policy1",
          "aclsExecuted": {
            "total": 3,
            "applied": 3,
            "failed": 0,
            "skipped": 0,
            "timeTakenMS": 717
          },
          "failedReasons": {},
          "skippedReasons": {},
          "ownerIgnore": {
            "count": 0
          }
        }
      ],
      "aclsExecuted": {
        "total": 3,
        "applied": 3,
        "failed": 0,
        "skipped": 0
      },
      "errorCode": "SYNCED_GRANTS_APPLIED"
    },
    "input": {
      "createTime": 1785222862000,
      "syncType": "RESOURCE_SYNC",
      "source": "REST_API",
      "appType": "PS_CONNECTOR",
      "resources": [
        {
          "type": "table",
          "values": {
            "catalog": "catalog1",
            "schema": "schema1",
            "table": "table1",
            "requestor_id": "6bab473e-612f-450b-a99c-e6a1d5de2424",
            "requestor_create_time": "1785222861"
          }
        }
      ]
    }
  }
}

A request for three resources produces three events like this one, all sharing ondemandTaskId, changeLogId, and timeTakenMs.

Outcome codes¶

Every requested resource gets exactly one event, and every event carries extraInfo.output.errorCode. The code is the stable, machine-readable outcome — branch on errorCode rather than on msg, because the wording of msg can change between releases while the code does not.

errorCode is present on success as well as on failure, so a consumer can switch on one field instead of inferring the outcome from status plus the counts.

Which resources a failure affects¶

Failed use-cases¶

A failed outcome always emits status = FAILED, a readable msg from the table above, a machine-readable errorCode, and — where there is a cause to report — the underlying error in errorDetails.serviceError.

Resource not found¶

output.resources and output.policies are both empty, and there is no errorDetails block — the resource never entered grant application, so the sync's grant error is not attributed to it. input still echoes the request, so you can see exactly which resource was missing.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

{
  "eventType": "ON_DEMAND",
  "evtTime": "2026-07-28T07:15:34Z",
  "timeTakenMs": 403,
  "status": "FAILED",
  "msg": "Resource not found in source",
  "connectorType": "databricks_unity_catalog",
  "connector": "databricks-unity-catalog-instance1",
  "changeLogId": 18479,
  "changeLogType": "SERVICE_RESOURCES",
  "loaderType": "LOAD_SERVICE_RESOURCES",
  "triggerFrom": "EVENT",
  "ondemandTaskId": 8,
  "extraInfo": {
    "output": {
      "resources": {},
      "policies": [],
      "errorCode": "RESOURCE_NOT_FOUND"
    },
    "input": {
      "createTime": 1785222921000,
      "syncType": "RESOURCE_SYNC",
      "source": "REST_API",
      "appType": "PS_CONNECTOR",
      "resources": [
        {
          "type": "table",
          "values": {
            "catalog": "catalog1",
            "schema": "schema1",
            "table": "does_not_exist",
            "requestor_id": "7bfaed32-8c8d-40b8-b122-0580cae8cfcc",
            "requestor_create_time": "1785222921"
          }
        }
      ]
    }
  }
}

Resource load failed¶

The load did not complete, so every resource in the task is FAILED with the same code, and output.resources is empty on all of them. The load error itself is in errorDetails.serviceError, not in msg.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

{
  "eventType": "ON_DEMAND",
  "evtTime": "2026-07-28T07:16:10Z",
  "timeTakenMs": 512,
  "status": "FAILED",
  "msg": "Resource load failed",
  "connectorType": "databricks_unity_catalog",
  "connector": "databricks-unity-catalog-instance1",
  "changeLogId": 18485,
  "changeLogType": "SERVICE_RESOURCES",
  "loaderType": "LOAD_SERVICE_RESOURCES",
  "triggerFrom": "EVENT",
  "ondemandTaskId": 9,
  "extraInfo": {
    "output": {
      "resources": {},
      "policies": [],
      "errorCode": "RESOURCE_LOAD_FAILED",
      "errorDetails": {
        "serviceError": "resource key persist failed"
      }
    },
    "input": {
      "createTime": 1785222970000,
      "syncType": "RESOURCE_SYNC",
      "source": "REST_API",
      "appType": "PS_CONNECTOR",
      "resources": [
        {
          "type": "table",
          "values": {
            "catalog": "catalog1",
            "schema": "schema1",
            "table": "table1",
            "requestor_id": "5f2a1c9e-3b7d-4e21-9a44-1c0b2d8e6f30",
            "requestor_create_time": "1785222969"
          }
        }
      ]
    }
  }
}

Grant application partially applied¶

The resource did change, so its counts are reported and the policy list rides the FAILED event, just as it does on a COMPLETED event. Read output.aclsExecuted to see how far the grants got: here 1 of 3 ACL operations applied and 2 failed, which is a partial grant — hence GRANT_PARTIALLY_APPLIED rather than GRANT_FAILED. The target's own error text is grouped in errorDetails.topReasons and repeated per policy in policies[].failedReasons.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82

{
  "eventType": "ON_DEMAND",
  "evtTime": "2026-07-28T07:16:44Z",
  "timeTakenMs": 1633,
  "status": "FAILED",
  "msg": "Grants partially applied; some grants failed",
  "connectorType": "databricks_unity_catalog",
  "connector": "databricks-unity-catalog-instance1",
  "changeLogId": 18490,
  "changeLogType": "SERVICE_RESOURCES",
  "loaderType": "LOAD_SERVICE_RESOURCES",
  "triggerFrom": "EVENT",
  "ondemandTaskId": 10,
  "extraInfo": {
    "output": {
      "resources": {
        "table": {
          "found": 1,
          "added": 1,
          "updated": 0,
          "deleted": 0
        }
      },
      "policies": [
        {
          "id": "160",
          "version": "12",
          "label": "",
          "name": "policy1",
          "aclsExecuted": {
            "total": 3,
            "applied": 1,
            "failed": 2,
            "skipped": 0,
            "timeTakenMS": 604
          },
          "failedReasons": {
            "PERMISSION_DENIED: cannot GRANT SELECT on catalog1.schema1.table1": 2
          },
          "skippedReasons": {},
          "ownerIgnore": {
            "count": 0
          }
        }
      ],
      "aclsExecuted": {
        "total": 3,
        "applied": 1,
        "failed": 2,
        "skipped": 0
      },
      "errorCode": "GRANT_PARTIALLY_APPLIED",
      "errorDetails": {
        "topReasons": [
          {
            "reason": "PERMISSION_DENIED: cannot GRANT SELECT on catalog1.schema1.table1",
            "count": 2
          }
        ],
        "serviceError": "changelog status=FAILED"
      }
    },
    "input": {
      "createTime": 1785223004000,
      "syncType": "RESOURCE_SYNC",
      "source": "REST_API",
      "appType": "PS_CONNECTOR",
      "resources": [
        {
          "type": "table",
          "values": {
            "catalog": "catalog1",
            "schema": "schema1",
            "table": "table1",
            "requestor_id": "a71e0d64-9c52-4f18-8b30-2e6f5a1d7c93",
            "requestor_create_time": "1785223003"
          }
        }
      ]
    }
  }
}

Validate¶

In Privacera Portal¶

The audits are indexed into Apache Solr, which is what Privacera Portal reads, so you can confirm the feature is working from the Portal without querying anything directly.

1. Sign in to Privacera Portal.
2. From the application menu at the top, select Access Controls.
3. In the left navigation, select Audits.
4. Select the Policy Sync tab.
5. Adjust the date range at the top right if your sync is older than the default window, and use the search box to narrow the list.
6. The tab lists every PolicySync event, so narrow it to the rows where Event Type is ON_DEMAND.

Each row is one requested resource. Read the outcome from these columns:

To review one request end to end, note its Ondemand Task ID and look at every row sharing that value.

If a sync produced no ON_DEMAND rows at all, the audit is most likely not enabled on the connector — check CONNECTOR_ON_DEMAND_AUDIT_ENABLE.

In Prometheus¶

The connector publishes one counter for on-demand audit emission, so you can confirm events are being produced — and spot events that failed to reach the audit pipeline at all, which by definition never reach the Portal or Solr.

Useful queries:

1
2
3
4

sum by (result) (ps_core_ondemand_audit_events_total)
sum by (status) (ps_core_ondemand_audit_events_total{result="emitted"})
ps_core_ondemand_audit_events_total{result="failed"}
ps_core_ondemand_audit_events_total{path!="normal"}

Any non-zero result="failed" means an audit event was produced but never delivered — worth alerting on. A non-zero path!="normal" means something upstream failed while the audit was still emitted.

In Amazon SQS¶

Use the built-in test script inside the Audit Server diagnostics container to confirm ON_DEMAND records are arriving in your queue.

1. Open a shell inside the Audit Server diagnostics container.
2. Navigate to the SQS test directory:

   Bash
   1	cd /workdir/manual_test/read_sqs

3. Set the required environment variables. Replace <account-id> and <queue-name> with your AWS account ID and queue name.

   Bash
   1 2	export SQS_QUEUE_URL=https://sqs.us-east-1.amazonaws.com/<account-id>/<queue-name> export DELETE_AFTER_PROCESSING="false"

4. Run the script:

   Bash
   1	./run_read_sqs.sh

The script prints the messages it reads. Look for records with "eventType":"ON_DEMAND" to confirm on-demand audits are flowing to the queue.

Guidelines¶

• The ON_DEMAND audit is per resource — an on-demand request for N resources produces N audit records. Keep this in mind when sizing a large batch, and watch the SQS/SNS 256 KB per-message limit for very large, policy-heavy syncs.
• Build consumers against extraInfo.output.errorCode, not msg. New codes may be added across releases.
• Enabling CONNECTOR_ON_DEMAND_AUDIT_ENABLE does not change the Ops Server / task-status flow; that continues to work whether the flag is on or off.
• If you filter an Amazon SNS subscription by event type, allow eventType = ON_DEMAND — otherwise the on-demand events are dropped before they reach the subscribed queue even though the connector emitted them.