Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .changeset/31797-dataset-query-audit-row.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
'@objectstack/plugin-audit': patch
'@objectstack/rest': patch
---

feat(plugin-audit, rest): a query of an opted-in saved dataset writes one `dataset_query` row in `sys_audit_log` (#31797)

Clause-②: yes (widening) — a new `sys_audit_log.action` value, `dataset_query`, with its writer; a new `AuditPlugin` option, `datasetQueryAudit` (`datasets`, `maxBatchSize`, `flushIntervalMs`); a new `recordDatasetQuery` member on the `audit` service slot; and new exports from `@objectstack/plugin-audit` (`createDatasetQueryAuditWriter`, `buildDatasetQueryAuditRow`, `DATASET_QUERY_AUDIT_ACTION` and their types). `createReadAuditBatcher` takes an optional event type parameter and an optional `overflowMessage`, both defaulting to what it did before. No door answers differently: with no dataset named, nothing is recorded or buffered.
Metadata-protocol: none
Skills-docs: none — no file under `skills/**` teaches the audit ledger's actions or `AuditPlugin`'s options; `content/docs/permissions/record-view-auditing.mdx` gains a "Dataset queries" section.

**What a deployment gets.** Name saved datasets in `new AuditPlugin({ datasetQueryAudit: { datasets: [...] } })` and each query of one through `POST /analytics/dataset/query`, or through each query of its batch door, writes one run-level row: the caller (`user_id`, `actor`, `tenant_id`), the dataset's base object in `object_name`, and in `metadata` the dataset name, the package holding it (`dataset_package`, `null` for the environment's own definition; a name in `datasets` covers every holder of that name), `preview: true` when a draft definition was read, the dimensions and measures selected, the filtered field names and operators (never the values), the row count and the outcome. A report or dashboard is composed in the browser, so the row names the dataset, not a report.

**The rules.** Off by default and opted in per saved dataset; an inline definition (the Studio preview) writes no row. A request the door refuses before the read writes no row; a query that fails after the read started writes one row with outcome `failed`, the HTTP status and the error's machine code, never the error's text. Rows are written in batches off the request path, stamped with the query's instant, and a lost batch is reported at `error` once per cause.
38 changes: 38 additions & 0 deletions content/docs/permissions/record-view-auditing.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -272,6 +272,44 @@ datasource whenever one is registered, and `os dev` provisions one by default as
a sibling SQLite file.
</Callout>

## Dataset queries

A report or a dashboard reads its data through the analytics dataset door,
`POST /analytics/dataset/query` (and its batch door), one query per block or
widget. The same plugin can record each of those queries as a `dataset_query`
row: **who** ran it, **which dataset** (its base object in `object_name`), the
**dimensions and measures** selected, the **filtered field names and
operators** (never the values), the **row count** and the **outcome**. It is
opted in per saved dataset and off by default:

```typescript
new AuditPlugin({
datasetQueryAudit: {
datasets: ['sales_pipeline', 'support_backlog'],
// maxBatchSize: 50, // flush once this many rows are buffered
// flushIntervalMs: 2000, // flush this long after a batch's first row
},
});
```

The rows are written in batches off the request path, stamped with the query's
own instant, by the same batcher as record views. The edges:

- **Saved datasets only.** A query that posts an inline definition (the Studio
preview) writes no row: the opt-in names a saved dataset, and an inline
definition's name is the caller's own text.
- **A refused query writes no row.** A request the door refuses before the
read (sign-in, a malformed body or selection, a dataset it cannot find or
parse) is not a read. A query that fails after the read started writes one
row with outcome `failed`, the HTTP status and the error's machine code.
- **A name covers every holder.** Two packages can each hold a dataset of the
same name. A name in `datasets` audits every one of them, and each row names
the package it read in `metadata.dataset_package` (`null` for the
environment's own definition). A query that read a draft definition carries
`preview: true`.
- **No report name.** A report is composed in the browser, so the server sees
its queries, not the report: one opened report can write one row per block.

## Not this

- **[`services.audit`](/docs/kernel/runtime-services/audit-service)** is the
Expand Down
34 changes: 17 additions & 17 deletions content/docs/permissions/tenant-audit-census.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,7 @@ are reported as `undecidable` rather than assumed either way.

The same holds twice over for the context. An options argument spelled as a
literal can be read; one spelled `options`, `{ ...opts }`, or handed through a
forwarding shim cannot, and **59 of the 314 sites are spelled that way**. A
forwarding shim cannot, and **59 of the 315 sites are spelled that way**. A
context resolved from an inline literal or a local `const` can be tested for
`isSystem`; one arriving from a helper call cannot.

Expand Down Expand Up @@ -197,10 +197,10 @@ reproduce them. Where it disagrees, it disagrees on the page:

| carried figure | where it survives | this census |
| :--- | :--- | ---: |
| 175 write call sites | quoted in the merged changeset | **314** |
| 175 write call sites | quoted in the merged changeset | **315** |
| 24 carrying no tenant context | quoted in the merged changeset | **3** provable and tenancy-enabled; **32** more whose options argument is unreadable |
| 127 of 175 statically decidable, 48 runtime-parameter-name sites | restated on the `isSystem`-scoping card | **219 of 314** decidable, **95** undecidable |
| 135 (77%) silenced by the `isSystem` guard before the posture gate | the lost issue body — **no surviving corroboration** | **not reproduced**: 177 decidably elevated, 0 decidably not, 125 undecidable |
| 127 of 175 statically decidable, 48 runtime-parameter-name sites | restated on the `isSystem`-scoping card | **220 of 315** decidable, **95** undecidable |
| 135 (77%) silenced by the `isSystem` guard before the posture gate | the lost issue body — **no surviving corroboration** | **not reproduced**: 178 decidably elevated, 0 decidably not, 125 undecidable |
| 141 and 132, two independent re-derivations | the card that filed this work | — |

**The differences are not reconciled, and deliberately so.** The old census's
Expand All @@ -217,11 +217,11 @@ would report a smaller number and would not say so.

The fourth row is the one worth flagging to anyone citing it. **The 135 / 77%
figure has no surviving corroboration anywhere in the tree.** This census reads
177 of 314 (56%) as decidably elevated, with 125 more whose elevation is a
178 of 315 (57%) as decidably elevated, with 125 more whose elevation is a
run-time fact — so the claim is neither confirmed nor refuted, and the honest
answer is that a static reading cannot settle it.

⇒ **Cite `3 / 314`, and say what it is**: the sites whose options argument was
⇒ **Cite `3 / 315`, and say what it is**: the sites whose options argument was
READ and holds no tenant context, against a decidably tenancy-enabled object.
That is the control's provable yield surface. ⛔ Do not cite it as "the sites
without tenant context" — **32 further sites** have an options argument this
Expand All @@ -233,28 +233,28 @@ cannot read, and they are neither in nor out.

| what | count |
| :--- | ---: |
| write call sites on the application surface | **314** |
| …whose object name is statically decidable | 219 |
| write call sites on the application surface | **315** |
| …whose object name is statically decidable | 220 |
| …whose object name is chosen at run time | 95 |
| …against an object with tenancy ENABLED | 212 |
| …against an object with tenancy ENABLED | 213 |
| …against an object that declares tenancy off | 7 |
| threading a tenant context | 243 |
| threading a tenant context | 244 |
| PROVABLY carrying none (options read, no context key) | **12** |
| …of those, against a decidably tenancy-enabled object | **3** |
| options argument UNREADABLE — may or may not carry one | 59 |
| …of those, against a decidably tenancy-enabled object | 32 |
| threading a decidably ELEVATED (`isSystem`) context | 177 |
| threading a decidably ELEVATED (`isSystem`) context | 178 |
| threading a context that is decidably NOT elevated | 0 |
| threading a context whose elevation is a run-time fact | 125 |

| how the instrument reached the site | count |
| :--- | ---: |
| receiver carried a readable engine type | 252 |
| receiver carried a readable engine type | 253 |
| receiver erased, placed by the object NAME | 52 |
| receiver erased, placed by an `object: string` PARAMETER | 5 |
| receiver erased, placed by an `UNTYPED_RECEIVERS` row | 5 |

| object name spelled inline | 133 |
| object name spelled inline | 134 |
| object name spelled through a `const` | 86 |
| object name is an `object: string` parameter | 10 |
| object name is some other run-time expression | 85 |
Expand Down Expand Up @@ -307,13 +307,13 @@ holds still. They are required to be HERE and to say WHEN they were true;
their values are not compared. The reasoning, and the measurement behind it,
are in `scripts/check-tenant-audit-census.mjs`.

Measured on 2026-11-14 at `aad3132fa`.
Measured on 2026-11-15 at `c1014aa8e`.

| corpus scale (not enforced) | count |
| :--- | ---: |
| tracked non-test sources scanned | 1160 |
| engine-shaped types recognised | 138 |
| tracked non-test sources scanned | 1170 |
| engine-shaped types recognised | 139 |
| declared objects in the registry | 137 |
| same-named calls subtracted as non-engine | 298 |
| same-named calls subtracted as non-engine | 309 |

{/* END GENERATED: tenant-audit-census */}
19 changes: 10 additions & 9 deletions docs/audits/2026-08-tenant-audit-write-call-sites.counts.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,17 +33,17 @@ silent, and `node scripts/tenant-audit-census.mjs --write` is the resolution.

| Measure | Value |
|---|---:|
| Write call sites | 314 |
| Object name statically decidable | 219 |
| Write call sites | 315 |
| Object name statically decidable | 220 |
| Object name chosen at run time | 95 |
| Against a tenancy-enabled object | 212 |
| Against a tenancy-enabled object | 213 |
| Against an object declaring tenancy off | 7 |
| Threading a tenant context | 243 |
| Threading a tenant context | 244 |
| Provably carrying none | 12 |
| …and decidably tenancy-enabled | 3 |
| Options argument unreadable | 59 |
| …and decidably tenancy-enabled | 32 |
| Threading a decidably elevated context | 177 |
| Threading a decidably elevated context | 178 |
| Threading a decidably non-elevated context | 0 |
| Threading a context of undecidable elevation | 125 |

Expand Down Expand Up @@ -90,14 +90,14 @@ holds still. They are required to be HERE and to say WHEN they were true;
their values are not compared. The reasoning, and the measurement behind it,
are in `scripts/check-tenant-audit-census.mjs`.

Measured on 2026-11-14 at `aad3132fa`.
Measured on 2026-11-15 at `c1014aa8e`.

| corpus scale (not enforced) | count |
| :--- | ---: |
| tracked non-test sources scanned | 1160 |
| engine-shaped types recognised | 138 |
| tracked non-test sources scanned | 1170 |
| engine-shaped types recognised | 139 |
| declared objects in the registry | 137 |
| same-named calls subtracted as non-engine | 298 |
| same-named calls subtracted as non-engine | 309 |

## Every site

Expand Down Expand Up @@ -129,6 +129,7 @@ Measured on 2026-11-14 at `aad3132fa`.
| `packages/plugins/plugin-audit/src/comment-parent-pair-backfill.ts` | `update` | `sys_comment` | enabled | elevated | 1 |
| `packages/plugins/plugin-audit/src/credential-use-audit.ts` | `insert` | `sys_audit_log` | enabled | elevated | 1 |
| `packages/plugins/plugin-audit/src/data-export-audit.ts` | `insert` | `sys_audit_log` | enabled | elevated | 1 |
| `packages/plugins/plugin-audit/src/dataset-query-audit.ts` | `insert` | `sys_audit_log` | enabled | elevated | 1 |
| `packages/plugins/plugin-audit/src/mail-log-body-migration.ts` | `update` | `object` | undecidable | elevated | 1 |
| `packages/plugins/plugin-audit/src/mail-log-body-migration.ts` | `update` | `object` | undecidable | options unreadable | 1 |
| `packages/plugins/plugin-audit/src/package-lifecycle-audit.ts` | `insert` | `sys_audit_log` | enabled | elevated | 1 |
Expand Down
51 changes: 51 additions & 0 deletions packages/plugins/plugin-audit/src/audit-plugin.ts
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ import { installReadAuditWriter, type ReadAuditWriterHandle } from './read-audit
import { createAuthEventAuditSink } from './auth-event-audit.js';
import { createActionInvocationAuditWriter } from './action-invocation-audit.js';
import { createDataExportAuditWriter } from './data-export-audit.js';
import { createDatasetQueryAuditWriter, type DatasetQueryAuditWriter } from './dataset-query-audit.js';
import { createPackageLifecycleAuditWriter } from './package-lifecycle-audit.js';
import {
createCredentialUseAuditWriter,
Expand Down Expand Up @@ -99,10 +100,34 @@ export interface AuditPluginReadAuditOptions {
flushIntervalMs?: number;
}

/**
* Dataset-query auditing configuration — the per-dataset opt-in, closed, with
* the record-view writer's shape and for its reasons
* ({@link AuditPluginReadAuditOptions}).
*/
export interface AuditPluginDatasetQueryAuditOptions {
/**
* Saved datasets whose queries through the analytics dataset door are
* recorded as `dataset_query` rows in `sys_audit_log`: who, which dataset,
* the dimensions and measures selected, the filtered field names (never the
* values), the row count and the outcome. Absent or empty records nothing
* and buffers nothing. A name covers every package that holds a dataset of
* that name (ADR-0150 D3); each row records the holder it read in
* `metadata.dataset_package`.
*/
datasets?: readonly string[];
/** Flush once this many rows are buffered. Default 50. */
maxBatchSize?: number;
/** Flush this long after the first row of a batch. Default 2000ms. */
flushIntervalMs?: number;
}

/** Constructor options for {@link AuditPlugin}. */
export interface AuditPluginOptions {
/** [#8992] Record-view auditing. Off unless objects are named. */
readAudit?: AuditPluginReadAuditOptions;
/** Dataset-query auditing (`dataset_query` rows). Off unless datasets are named. */
datasetQueryAudit?: AuditPluginDatasetQueryAuditOptions;
/**
* Host locale resolver: the language this plugin writes the title of the
* @mention notification it emits in. Called with the write's organization
Expand Down Expand Up @@ -311,6 +336,7 @@ export class AuditPlugin implements Plugin {
* succeeded.
*/
private readAuditWriter: ReadAuditWriterHandle | null = null;
private datasetQueryWriter: DatasetQueryAuditWriter | null = null;
/** [ADR-0154 D3] The sealer, started at `kernel:ready` and stopped by `destroy()`. */
private sealer: AuditSealer | null = null;

Expand Down Expand Up @@ -435,13 +461,35 @@ export class AuditPlugin implements Plugin {
// this slot, from plugin-auth's OAuth stamp point
// (`credential-use-audit.ts`).
this.credentialUseWriter = createCredentialUseAuditWriter({ getEngine: getLedgerEngine, logger: ctx.logger });
//
// And the dataset-query writer (`recordDatasetQuery`): the analytics
// dataset door hands it one event per saved-dataset query that reached
// the read; it records only the datasets named in
// `datasetQueryAudit.datasets`, batched off the request path
// (`dataset-query-audit.ts`). Named nothing, it records nothing.
const datasetQueryAudit = this.options.datasetQueryAudit;
const datasetQueryWriter = createDatasetQueryAuditWriter({
getEngine: getLedgerEngine,
logger: ctx.logger,
datasets: datasetQueryAudit?.datasets ?? [],
...(datasetQueryAudit?.maxBatchSize !== undefined ? { maxBatchSize: datasetQueryAudit.maxBatchSize } : {}),
...(datasetQueryAudit?.flushIntervalMs !== undefined ? { flushIntervalMs: datasetQueryAudit.flushIntervalMs } : {}),
});
this.datasetQueryWriter = datasetQueryWriter;
if (datasetQueryWriter.auditedDatasets.length > 0) {
ctx.logger.info(
`AuditPlugin: dataset-query auditing on ${datasetQueryWriter.auditedDatasets.length} dataset(s) — `
+ `${datasetQueryWriter.auditedDatasets.join(', ')}`,
);
}
ctx.registerService('audit', {
...createAuthEventAuditSink({ getEngine: getLedgerEngine, logger: ctx.logger }),
...createActionInvocationAuditWriter({ getEngine: getLedgerEngine, logger: ctx.logger }),
...createDataExportAuditWriter({ getEngine: getLedgerEngine, logger: ctx.logger }),
...createPackageLifecycleAuditWriter({ getEngine: getLedgerEngine, logger: ctx.logger }),
...this.credentialUseWriter,
...createScimRefusalAuditWriter({ getEngine: getLedgerEngine, logger: ctx.logger }),
recordDatasetQuery: datasetQueryWriter.recordDatasetQuery,
// The activity door's writer: one timeline entry an app adds through
// `ctx.activity.append` (`app-activity-door.ts`).
...createAppActivityWriter({ getEngine: getLedgerEngine, getActorSnapshot: () => this.actorSnapshotOf }),
Expand Down Expand Up @@ -1009,6 +1057,9 @@ export class AuditPlugin implements Plugin {
const writer = this.readAuditWriter;
this.readAuditWriter = null;
if (writer) await writer.stop();
const datasetQueryWriter = this.datasetQueryWriter;
this.datasetQueryWriter = null;
if (datasetQueryWriter) await datasetQueryWriter.stop();
}
}

Expand Down
7 changes: 6 additions & 1 deletion packages/plugins/plugin-audit/src/data-export-audit.ts
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,12 @@ export interface ExportFilterSummary {
omitted: number;
}

function recordedField(path: string): string {
/**
* A field path as the ledger records it: the path when it is a machine-name
* path, else `?`. Shared with the `dataset_query` writer, which records the
* selection's dimension and measure names under the same rule.
*/
export function recordedField(path: string): string {
return FIELD_PATH.test(path) ? path : '?';
}

Expand Down
Loading
Loading