Skip to content

feat(plugin-audit, rest): a query of an opted-in saved dataset writes one dataset_query audit row - #33559

Draft
objectstack-fleet[bot] wants to merge 11 commits into
mainfrom
claude/area-A2-31797
Draft

objectstack-fleet[bot] wants to merge 11 commits into
mainfrom
claude/area-A2-31797

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Requested by Elon · project thread

domain:server#5 · os-elon-musk · session_016WV4cz2MxDkQYCC4U8indM

Fixes #31797
Clause-②: yes (widening)
Metadata-protocol: none
Skills-docs: none — no file under skills/** teaches the audit ledger's actions or AuditPlugin's options; content/docs/permissions/record-view-auditing.mdx gains a "Dataset queries" section.

Maintainer quick read

  • What changed: a deployment can name saved datasets in new AuditPlugin({ datasetQueryAudit: { datasets: [...] } }). Each query of one of them through the analytics dataset door then leaves one dataset_query row in sys_audit_log: who ran it, which dataset, the dimensions and measures selected, the filtered field names and operators (never the values), the row count, and whether it succeeded.
  • Why: ruling C on this card. An enterprise SIEM must be able to see who read which data under which filter. Reports and dashboards are put together in the browser, so the server only ever sees dataset queries; this records those, and it does not name a report.
  • Risk and rollback: off by default. With no dataset named, nothing is recorded or buffered, and every door answers as before. Rows are written in batches off the request path. Rollback is reverting this PR.
  • Look first: (1) packages/rest/src/analytics-dataset-query-audit.ts runAuditedDatasetQuery: only the read is wrapped, so a refusal before it writes nothing; (2) packages/plugins/plugin-audit/src/dataset-query-audit.ts buildDatasetQueryAuditRow: what the row holds and that the filter goes through summarizeExportFilter; (3) the queryDataset hunk in packages/rest/src/analytics-routes.ts.
  • Two-minute check: pnpm --filter @objectstack/plugin-audit exec vitest run src/dataset-query-audit.test.ts. Expect 10 passing cases, among them "a dataset that is not opted in writes no row" and "records no filtered value".
  • What you need to do: nothing; the seat lands it.

What changes

plugin-audit. A new writer, dataset-query-audit.ts, emits action: 'dataset_query'. Each row is a run-level row: record_id null, the dataset's base object in object_name, user_id / actor / tenant_id from the caller's envelope. metadata holds dataset, dimensions, measures, time_dimensions (names only, when selected), filter (field and operator terms from the export writer's summarizeExportFilter, every comparand dropped unread), row_count and outcome. A failed query adds status and error_code. The delegated-call pair and impersonated_by are written as the export row writes them. The writer follows the scope pins read-audit.ts set for #8992:

  • Per-dataset opt-in, closed, off by default. AuditPlugin's new datasetQueryAudit option (datasets, maxBatchSize, flushIntervalMs) has the same shape as readAudit. No spec change: it is plugin configuration, not authorable metadata, so Metadata-protocol: none. A dataset that is not named is dropped before anything is buffered. An empty list builds no batcher.
  • Async batched writes off the request path. recordDatasetQuery on the audit slot builds the row when it is called (names and counts only, so the buffer never holds the caller's filter), enqueues it and returns. Rows are written by createReadAuditBatcher, the record-view writer's batcher, which now takes an event type parameter and an optional overflow sentence. The query instant is kept through preserveAudit, as on record views. A lost batch is reported at error once per cause. The named callee persistDatasetQueryAuditRows is registered in check:durability-log-level and in the swallow census.
  • The sys_audit_log.action options gain Dataset Query / dataset_query, with the generated translation leaves for the 11 bundles. The enum ledger (sys-audit-log-retired-actions.test.ts) and the label pin list it with its writer. Its shipped surfaces are the unfiltered recent and all_events views, as for scim_refused.

rest. answerDatasetQuery wraps only the queryDataset call in runAuditedDatasetQuery (new analytics-dataset-query-audit.ts). The event shape is declared structurally, as the export door declares its own. The rules are #31000's, as data-export-audit.ts implements them:

  • a request the door refuses before the read (identity, the body and selection doors, the dataset's resolution, its parse) hands no event;
  • a read that throws hands one event with outcome failed, plus the status and machine code resolveThrownHttpError reads, never the error's text;
  • the batch door calls the same judgment, so each of its queries that reaches the read hands one event.

AnalyticsRouteHost gains resolveAuditService, and rest-server.ts hands it over.

Overlap with open PRs. #33491 edits analytics-routes.ts (the import of admitRefusedDatasetHolders, the refusal-arm hunks near the resolution). This PR's three hunks in that file are elsewhere: one import after stripReadDecorations, one member at the end of AnalyticsRouteHost, and the queryDataset call. git merge-tree of this head with each of #33491 and #33512 reports no conflict.

Tests

The pins were committed first (4cdb3b2df) and run red before the fix:

  • plugin-audit src/dataset-query-audit.test.ts: Error: Cannot find module './dataset-query-audit.js'. No writer existed.
  • rest src/analytics-dataset-query-audit.test.ts: Tests 3 failed | 3 passed (6). Red: one event per answered query, one failed event after the read started, one event per batch query that reached the read. The three green cases (no event for a refused request, none for an inline definition, a throwing sink does not change the answer) are pins on the absence of an event, so they were green before the door handed any event.

After the fix, with module-level evidence in Verification:

  • one row per query of an opted-in dataset (writer over the real ledger, and through AuditPlugin and the audit slot);
  • no row for a dataset that is not opted in, nor with no option at all;
  • no event for a query refused before the read (selection without measures 400, unknown dataset 404, unknown body key 400; queryDataset never called);
  • one failed row and event after the read started (503 with SERVICE_UNAVAILABLE), the error's text not carried;
  • no filtered value anywhere in the row (a sentinel value in $in, $contains, a nested relation path and an implicit equality);
  • the query instant, not the flush instant, on created_at; the enqueue returning before the write; a lost batch reported once per cause.

Acceptance notes

  • Declared narrowing: inline definitions write no row. The opt-in names saved datasets, and an inline definition's name is text the caller chose. Recording inline queries would need a different opt-in key (the base object, say), which this ruling does not decide. The docs section states this edge.
  • A refusal thrown inside queryDataset is recorded as a failed read. The analytics service judges object exposure and dataset fields inside queryDataset, and the door cannot tell that throw from a driver fault without reading the service's internals. It follows the export door's precedent, where a refusal from findData after the export began is a failed row with its status and code: the row says outcome: failed with the status and machine code the door answered (a 403 for an exposure refusal). No case pins a 403 row; the 503 case pins the mechanism.
  • The door resolves the audit slot for every saved-dataset query. The opt-in list lives in the writer (one input, the record-view writer's rule), so the door cannot skip the lookup. The cost is a service lookup; the write itself is batched.
  • The same dataset name under two packages is one opt-in entry; the row records the name as the door resolved it, not the holding package.
  • No ADR-0087 entry is needed: migrations/entries/semantic/17.audit-log-action-enum-retired.ts retires export / permission_change / restore only, packages/spec has no mirror of the action options, and no other ledger lists them (git grep -n scim_refused reaches only plugin-audit and plugin-auth).

Verification

Package suites and typecheck

Run on the branch before the last two commits (3d982b0ac, then 0f3569f23 for the rest re-run). The last commit 1f741879a only drops two as any casts on the writer's insert and refreshes the tenant audit census; after it, src/dataset-query-audit.test.ts + src/read-audit.test.ts were run again (Tests 43 passed (43)) and plugin-audit tsc --noEmit exited 0.

  • pnpm --filter @objectstack/plugin-audit test: Test Files 94 passed (94), Tests 1505 passed (1505). pnpm --filter @objectstack/plugin-audit typecheck: exit 0 (lock VERDICT command-exit 0).
  • pnpm --filter @objectstack/rest test: Test Files 412 passed (412), Tests 8399 passed | 332 skipped (8731). pnpm --filter @objectstack/rest typecheck: exit 0 (lock VERDICT command-exit 0). After moving the import, analytics-dataset-query-audit.test.ts + analytics-routes.test.ts ran again (Tests 83 passed (83)) and rest tsc --noEmit exited 0.
  • eslint --no-inline-config on every changed .ts / .mjs file: exit 0.
  • node scripts/check-test-source-alias.mjs: exit 0 after the new rest pin gained its module-top @objectstack/objectql import, which the gate asked for.
  • node scripts/check-nul-bytes.mjs: exit 0.
  • pnpm --filter @objectstack/metadata-protocol exec vitest run src/sys-metadata-repository.draft-drain.test.ts, which reads the edited durability gate script: Tests 11 passed (11).
Derived gate families

node scripts/pm/dispatch-gates.mjs --commands derived 127 families for this diff, and all 127 were run (--ran: 127 derived, 127 run, 0 unrun). Each exit code was captured before any pipe. 121 exited 0 on the first pass. The other six:

  • check-tenant-audit-census and its --self-test: exit 1. This diff adds one write call site (dataset-query-audit.ts, insert into sys_audit_log, elevated), so the census moved from 314 to 315. Fixed by node scripts/tenant-audit-census.mjs --write plus the six hand-written figures on content/docs/permissions/tenant-audit-census.mdx; both re-run exit 0 (315 write call sites certified, self-test 27 cases pass).
  • pnpm check:i18n: exit 3, a prerequisite (CLI not built). I built the closure the gate names and re-ran it: exit 0, all bundles in sync.
  • NOT MEASURED: check-plugin-teardown-shape --self-test (exit 3, shallow clone cannot reach its pinned positive control), check:dual-build-cjs-loads (exit 3, needs a whole-repo build), and spec check:skill-examples (exit 3, needs the client SDK dist/). None of them reads a file this diff changes in a way it depends on; CI runs all three on a full checkout.

The families ran against the tree before 1f741879a. That commit changes the writer's insert options and the census page, and only the census gate was re-run on it.

Human Minimum

  • 1. Schemas start as Zod; packages/spec holds no runtime logic. (No packages/spec change.)
  • 2. Every name I added uses the casing for its kind.
  • 3. No formatter ran; pnpm lint passes. (eslint on the changed files exit 0; the repo-wide run is CI's.)
  • 4. One issue, one branch off an up-to-date main, one pull request.
  • 5. A change that publishes carries a changeset, and both declaration lines are filled in here and in the changeset.
  • 6. packages/spec touched: not touched.
  • 7. No hand edits to content/docs/references/, packages/*/CHANGELOG.md or content/docs/releases/.
  • 8. Title, body and comments are in English.
  • The Before Submitting list in CONTRIBUTING.md is done.

🤖 Generated with Claude Code

https://claude.ai/code/session_016WV4cz2MxDkQYCC4U8indM

…et query, none when refused before the read

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016WV4cz2MxDkQYCC4U8indM
… opted-in saved dataset

The analytics dataset door hands the audit slot one event per saved-dataset
query that reached the read, answered or failed; a request refused before the
read hands none. plugin-audit records only the datasets named in
AuditPlugin's datasetQueryAudit.datasets (off by default), batched off the
request path with the record-view writer's batcher, and keeps the filter's
field names and operators, never its values.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016WV4cz2MxDkQYCC4U8indM
…xt readably; the tenant audit census counts the new site

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016WV4cz2MxDkQYCC4U8indM
@objectstack-fleet objectstack-fleet Bot added domain:server Domain lane — seat card indexed by label:pm:seat lease:A2 labels Nov 15, 2026
@github-actions github-actions Bot added size/xl documentation Improvements or additions to documentation tests tooling Fix lands in tooling, not a product package — enters pm:queue only with Unblocks: or a named surface labels Nov 15, 2026
@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 2 package(s): @objectstack/plugin-audit, @objectstack/rest, touching 73 documentable anchor(s). ⚠️ 1 changed file(s) yielded no anchor (packages/plugins/plugin-audit/src/index.ts), so the pages documenting them are NOT COVERED by this run — this is not a clean bill of health for those files.

57 hand-written doc(s) name something this change touched — list omitted above 15 rows. Re-derive on the tree named below: node scripts/docs-audit/affected-docs.mjs --json 4bba8b8f02fdbdf30e9ad41fc5d2a6fdd3ccaf2a.

⛔ 18 release-owned page(s) also affected — read-only, see AGENTS.md Documentation Guardrails.

What this run could not see
  • 1 changed file(s) yielded no anchor (packages/plugins/plugin-audit/src/index.ts) — pages documenting those are invisible to this run
  • 1 cross-cutting symbol(s) contributed no route anchor: environmentId (44 routes)
  • 1 anchor(s) matched too much of the corpus to be a work list: objectName (symbol, 58 pages)
  • 18 name(s) were too generic to anchor anything (single lowercase words)
  • the SDK route bridge reached 65 of 247 client-bound route-ledger rows — the other 182 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 182: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 59 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 123 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.
  • a key NAME is not a key, so the hand re-read the line above prescribes can land on the wrong schema. The same spelling is authorable on one governed type and a [REMOVED] tombstone on another for each of active, aria, joins, objects, template, tools and version (censused on [finding] tools is a key on BOTH AgentSchema (tombstoned, dead) and SkillSchema (live, cloud-attested), so a name-based search attributes skill examples to the agent key — it produced a false stop-the-line alarm on PR #19059 #19093 over the liveness ledger's governed types, top-level keys); nothing in a search result distinguishes the two, so a grep hit on a LIVE example reads as evidence about the DEAD key. Measured on fix(spec): the agent.tools liveness row says dead — it claimed live on a key the schema tombstoned #19059: content/docs/ai/agents.mdx was reported as contradicting the agent.tools tombstone over its tools: example at :161, which is inside the defineSkill({ block opened at :155 — the page was already correct. Settle ownership by PARSING the value against both schemas, never by the name: that literal PASSES SkillSchema, and as an AgentSchema it FAILS at tools with the tombstone prescription. ⛔ These names are not the whole class — a key retired through a .strict() guidance map leaves no tombstone in the walked shape and none of them here (tool.category, live as AIToolDefinition.category).

Coarse fallback — 27 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 4bba8b8f02fdbdf30e9ad41fc5d2a6fdd3ccaf2a → packageMentionDocs.

Which tree this was computed on

This run read content/docs from e5d8edd608d9bdfe5221b0eafc4aeee334e5e6df — the merge of head be4a48ae3044fd52058048c8508b3d5fc877c57c into base 4bba8b8f02fdbdf30e9ad41fc5d2a6fdd3ccaf2a, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin e5d8edd608d9bdfe5221b0eafc4aeee334e5e6df && git checkout e5d8edd608d9bdfe5221b0eafc4aeee334e5e6df
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 4bba8b8f02fdbdf30e9ad41fc5d2a6fdd3ccaf2a be4a48ae3044fd52058048c8508b3d5fc877c57c && git checkout -B drift-repro 4bba8b8f02fdbdf30e9ad41fc5d2a6fdd3ccaf2a && git merge --no-ff be4a48ae3044fd52058048c8508b3d5fc877c57c

node scripts/docs-audit/affected-docs.mjs --json 4bba8b8f02fdbdf30e9ad41fc5d2a6fdd3ccaf2a

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Advisory only, and a precision-first one (#9192): a page is listed because it names a
symbol, wire route or SDK method this diff touched — not because it mentions a changed
package. Each row says which anchor put it there, so a wrong row is reportable rather than
merely annoying. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs 4bba8b8f02fdbdf30e9ad41fc5d2a6fdd3ccaf2a → pass the list as
args.docs, on the commit named under Which tree this was computed on.

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

Contract-review-claim: session_016WV4d1PGptBW3QRQNmy5e6 · head 1f74187 · 2026-11-15T01:44Z

review#3.

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

Contract review

Served-tier: TIER_DEFAULT
Head-sha: 1f741879a98e944a648aad37e953a2c9674dbffc
Local-runs: none

The PR's own commits (4cdb3b2d…1f741879, on top of e3f0cfd4) were read. The rest of the range against today's main is main.

① Derived judgments

  • Your question: "inline definitions are not audited" is scope, not a narrowing arm. Before this PR no dataset query wrote a row. After it, an opted-in saved-dataset query writes one, and every door answers exactly as before. Nothing a caller could send or read before is refused or reshaped. Clause-② item 5 asks about the accept set and the public surface, and leaving the inline path out shrinks neither. The widening declaration is right: a new action value, a new plugin option, a new recordDatasetQuery slot member, new exports, and two optional parameters on createReadAuditBatcher whose defaults keep the old behaviour.
  • The event boundary is right.
    • runAuditedDatasetQuery wraps only svc.queryDataset, so every earlier refusal returns before it and hands no event: identity, body, selection, resolution, the holder judgment and the parse. The pins measure this.
    • A thrown read hands one failed event with resolveThrownHttpError's status and code, never the message, and is rethrown unchanged.
    • The batch door goes through the same answerDatasetQuery, so each entry gets one event (pinned).
    • The sink is resolved per request, and its promise is not awaited, so the write stays off the request path. A throwing sink never changes the answer (pinned).
  • The row is right. The principal, tenant, actor (presence-probed) and impersonation follow the CRUD mirror's rules. metadata keeps the dataset, dimension, measure and time-dimension names (bounded at 32 each, through recordedField), the filter as summarizeExportFilter terms with no comparands (pinned), row_count, outcome, and on a failure status/error_code. The row is stamped with the query instant (pinned) and written with preserveAudit. The writer is null when nothing is opted in, so nothing is buffered by default (pinned). Lost batches are reported at error once per cause, and overflow is reported once.
  • must-fix M1: the row does not say WHICH dataset when two packages hold the name.
    • Two holders are real. dataset is a type stored per (package, name). This door resolves ?package=, ?callerPackage= and the bare name per ADR-0150 D3, and since fix(rest): the dataset door judges a holder whose own address a row no one module owns contends, so the name refuses 409, not 503 (ADR-0150 D4) #33491 a contended name can be answered by one holder judged on its own item (admitRefusedDatasetHolders).
    • The row loses the holder. The event carries only datasetName and the dataset's object. The opt-in is keyed on the bare name, so an opted-in name audits every holder's queries under one indistinguishable dataset value, and an auditor cannot tell package A's dataset from package B's.
    • The fix. Pass the resolved holder from the item the door read (read.item's package stamp, or null for the environment's own) into the event, and record it in metadata (for example dataset_package). Say in the option's doc and in record-view-auditing.mdx that a name in datasets covers every holder of that name. Add one pin at the rest layer: two holders of one name, a query at ?package=A and one at ?package=B, two rows that differ in dataset_package.
  • nit N1. previewDrafts reads a draft definition under the same name. If the door knows it served a draft, recording preview: true would keep a draft read distinguishable from a published one. Optional.

② Semver level

@objectstack/plugin-audit: patch and @objectstack/rest: patch, Clause-②: yes (widening), Metadata-protocol: none. That is correct for a non-protocol widening. M1 adds a metadata key to a new row, which stays inside this widening.

③ Boundary flags

Implemented-by: claude/area-A2-31797
Reviewed-by: session_016WV4d1PGptBW3QRQNmy5e6

VERDICT: FAIL

… the dataset_query row

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016WV4cz2MxDkQYCC4U8indM
…ng the dataset it read, and a served draft

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016WV4cz2MxDkQYCC4U8indM
…ert order within one batch

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016WV4cz2MxDkQYCC4U8indM
@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

Contract-review-claim: session_016WV4d1PGptBW3QRQNmy5e6 · head be4a48a · 2026-11-15T02:15Z

review#3 (delta).

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

Contract review

Served-tier: TIER_DEFAULT
Head-sha: be4a48ae3044fd52058048c8508b3d5fc877c57c
Local-runs: none

This reviews the delta on my FAIL at 1f741879a98e944a648aad37e953a2c9674dbffc. The PR-side commits are 9a177815 (red pins), 37cead9a (fix) and 99e4217b (pin order). After them come a merge of main (c1014aa8) and a census re-measure (be4a48ae).

① Derived judgments

  • M1 closed.
    • The door. It now keeps the item the by-name read served (served = read?.item, captured before translation and strip) and hands datasetPackage: holderPackageOf(served). That is the item's _packageId, or null for the environment's own definition (sys_metadata or unstamped).
    • The row. The writer records it as metadata.dataset_package, bounded to a package-id shape (? otherwise).
    • The pin. A rest pin holds two holders of sales. It queries ?package=com.acme.crm and ?package=com.acme.ops and gets two events that differ in datasetPackage. A plugin-audit pin orders the two rows by row count.
    • The docs. The option doc, the changeset and record-view-auditing.mdx say that a name covers every holder and that each row names the holder it read.
  • N1 taken. A served draft (_draft) records preview: true (pinned together with the environment's own definition recording dataset_package: null).
  • Merge and census. The merge brings in main. The census re-measure on the merged main changes only the counts doc and the census mdx. No PR-side source changed in either commit.
  • Note. The holder is read from the served item's _packageId stamp, the same marker other doors read a holder from. An item served unstamped would be recorded as the environment's (null). This is consistent with how the metadata read serves package items today.

② Semver level

Unchanged and correct: @objectstack/plugin-audit: patch, @objectstack/rest: patch, Clause-②: yes (widening), Metadata-protocol: none. The two new metadata keys sit inside the new row.

③ Boundary flags

The ACCEPT re-reads every check at the landing head.

Implemented-by: claude/area-A2-31797
Reviewed-by: session_016WV4d1PGptBW3QRQNmy5e6

VERDICT: PASS

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation domain:server Domain lane — seat card indexed by label:pm:seat lease:A2 size/xl tests tooling Fix lands in tooling, not a product package — enters pm:queue only with Unblocks: or a named surface

Projects

None yet

Development

Successfully merging this pull request may close these issues.

audit: running a report writes no sys_audit_log row

2 participants