Skip to content

test(formula): judge each distinct sweep refusal once, and give the validateExpression sweep a measured budget - #30919

Merged
objectstack-fleet[bot] merged 5 commits into
mainfrom
claude/issue-30901-test-core-timeouts
Nov 6, 2026
Merged

objectstack-fleet[bot] merged 5 commits into
mainfrom
claude/issue-30901-test-core-timeouts

Conversation

@objectstack-fleet

@objectstack-fleet objectstack-fleet Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Requested by 田騏 · project thread

Fixes #30901

Clause-②: no
Metadata-protocol: none

维护者速读

  • 改了什么:formula 包里那条「validateExpression 扫描」用例,重复的拒绝结果只判一次(3,865 条里只有 199 条不同),并给它一个按实测定的 16 秒超时,注释里写明读数和算法;生产代码、其它用例、分片计划都没动。
  • 为什么改:这条用例在 main 推送的 Test Core (4/8) 上跑了 5034 ms,撞上 vitest 默认 5 秒超时变红,下一次推送又绿;本地空闲只要 0.8 秒,是分片负载把它推过了线。
  • 风险与回滚:只改一个测试文件,不发布任何东西;断言一条没少,变异实验证明它照样会红。回滚就是 revert 这一个 PR。
  • 先看:packages/formula/src/expression-refusal-codes.test.ts 里的 refusalKey 与 VALIDATE_SWEEP_BUDGET_MS 注释块,以及扫描用例里 judged 那几行。
  • 两分钟验证:pnpm --filter @objectstack/formula exec vitest run src/expression-refusal-codes.test.ts --reporter=verbose,应看到 42 条全过,扫描用例约 0.5–0.7 秒(改前约 0.8 秒)。
  • 需要你做的:无,席位落地。

What changes

packages/formula/src/expression-refusal-codes.test.ts, the case every refusal validateExpression reports on the sweep carries a code from its set, with params the message interpolates (line 870 on origin/main):

  1. Cut: each distinct refusal is judged once. The sweep makes 5,760 validateExpression calls (3 roles x 128 inputs x 15 schemas) and gets 3,865 refusals, but only 199 distinct ones. The case ran its five assertions again on every duplicate, about 25,000 expect calls. Every assertion reads only the refusal itself (code, params, message, source, the four fields refusal() in validate.ts builds), so a refusal equal to one already judged would get the same verdict again. It is now keyed with inspect, not JSON.stringify, so an undefined or NaN param is still told apart, and judged once. refusals still counts every refusal (the > 100 control is unchanged), and provoked still has to reach every code.
  2. Budget: the case takes an explicit VALIDATE_SWEEP_BUDGET_MS = 16_000. It uses the rule from packages/cli/test/hook-timeout-override-refusal.test.ts: 3x the slowest reading, rounded up to the next second. The slowest reading is the CI failure's 5034 ms, so 3 x 5034 = 15102, rounded up to 16000. The readings and the rule are in the comment above the constant.

What was not cut, and why:

  • The sweep's breadth. About 70% of the case's CPU goes to buildTypedEnv (cel-engine.ts), which builds a fresh typed CEL environment for each call. That is production code, outside this card's file surface. The inputs are the claim: the docblock says "a new arm reached by an old input is judged", so dropping inputs would weaken the case.
  • No beforeAll and no module-scope precompute. The two sweeps share nothing. The collectCelRootIdentifiers sweep (line 895) walks 40 sources through a different producer and measured 5 to 37 ms everywhere, so it needs neither a cut nor a budget. Moving the validateExpression sweep into a hook would not remove its work. The work would run under the hook's 10 s default, which is a timeout raise by another name.

No production file, no shard plan file, no other test file is touched. There is no global testTimeout, no skip and no retry.

Tests

The formula case: before / after, local and CI
Reading Before (origin/main 98f640d) After (this branch)
Local, 4 vCPU, this file alone, 3 runs 786 / 830 / 757 ms 592 / 534 / 645 ms
Local, pinned to one CPU shared with two busy loops, 3 runs 3063 / 2542 / 2603 ms 2035 / 2137 / 1961 ms
CI Test Core jobs (main push / merge group, 2026-11-03..06, a sample of 7 that ran formula) 429, 1161, 2122, 2177, 2968, 3065, 5034 (timed out) not yet measured (this PR's own run)

The CI failure was job 127808107135 (main push run 42192379976 on d55acb7828, Test Core (4/8)): Test timed out in 5000ms, case duration 5034 ms. The case is synchronous, so vitest judges it only after the body returns, and 5034 ms is the full run, not a cut-off. In the same run the second sweep took 37 ms.

Profile (--cpu-prof, the sweep x3): validateExpression 1189 ms inclusive, of which buildTypedEnv 830 ms (typeSoundnessIssue, firstTypeMismatch, firstDateTextVerdict). No dynamic import or module load happens inside the clocked window.

Mutation: the deduplicated sweep still goes red

Done after the fix was committed (7a6ff9240d), with node scripts/ablation-replace.mjs. In packages/formula/src/validate.ts the line return { source, message, code, params } as ExprValidationError; was changed to add a param ghost: 'never-in-the-message' to unbound-root refusals only. unbound-root is a code the sweep reaches from only a few of its 5,760 calls.

  • anchor x1 -> x0, blob 8931039644aa -> afa0bba73789 (mutation landed)
  • vitest run src/expression-refusal-codes.test.ts -t "validateExpression reports on the sweep": 1 failed, AssertionError: expected 'unbound root \pge` — beyond the recor…' to contain 'never-in-the-message'`
  • restored: blob == HEAD (8931039644aa), git diff HEAD empty

No build was needed: the test imports ./validate from src.

Package suite, typecheck, lint
  • pnpm --filter @objectstack/formula test: Test Files 65 passed (65), Tests 1703 passed (1703)
  • pnpm --filter @objectstack/formula typecheck: exit 0, check:test-typecheck: OK — 3 file(s) / 7 error(s) / 4 pinned signature(s) held (the ledger is unchanged)
  • pnpm exec eslint packages/formula/src/expression-refusal-codes.test.ts: exit 0
  • Publishes nothing: files is dist, README.md, CHANGELOG.md. After pnpm --filter @objectstack/formula build, grep -rl 'VALIDATE_SWEEP_BUDGET_MS|refusalKey' dist finds 0 files, and the positive control validateExpression finds 4. So skip-changeset.

Acceptance notes

The measurement this card owns. It is a sample, and the queries are named so it can be repeated.

Per-shard wall time, `Test Core (N/8)`, 2026-11-03 10:00Z .. 2026-11-06 10:30Z

Source: GET /repos/objectstack-ai/objectstack/actions/workflows/224644133/runs?event=push|merge_group&created=>=2026-11-03T10:00:00Z (980 runs: 320 main push, 660 merge group), then GET /actions/runs/{id}/jobs for each completed run. Wall = completed_at - started_at of each successful shard job, setup and build included. The population pools path-filtered main push runs with full-plan merge-group runs (sampled push run 42181191624 planned 37 of 84 items, and its shard 1/8 got one item), so a push run's shard reads low. Restrict to full-plan runs before re-deriving the cli slicing from these numbers.

Shard n median s p90 s max s
1/8 749 948 1160 1604
2/8 722 734 1051 1542
3/8 658 859 1060 1599
4/8 620 1189 1504 2098
5/8 626 612 822 1979
6/8 617 624 886 1606
7/8 616 622 898 1431
8/8 598 627 785 1972

Shard 4 runs at 1.53x the mean of the eight medians (776 s). The plan predicts all eight bins equal (1553/1556/1551/1552/1552/1553/1556/1554s in job 127808107135). The gap comes from the CLI. Shards 1 and 4 each carry one @objectstack/cli slice, and the dataset weighs the whole package at 1432.04 s (716 s a slice). In 6 sampled push runs the two slices measured 607-959 s (1/2) and 627-1070 s (2/2), so 1234-2002 s for the package, median 1782 s. Shard 4 also runs plugin-approvals (289 s), verify (340 s) and driver-memory (183 s) beside the slice. Its formula run is the contended one.

The dataset was refreshed 2026-11-02 by shard-timings-refresh.yml (scheduled, weekly). Its generator needs turbo run summaries that only the scheduled runs keep, so this PR does not touch scripts/test-shard-timings.json, and a hand edit is forbidden. The next scheduled refresh is the place the CLI weight moves.

Every 5 s timeout in Test Core over the same window

All 29 failed Test Core (N/8) jobs were read (GET /actions/jobs/{id}/logs). Ten failed with Test timed out in 5000ms:

The other 19 are not timeouts. 18 are @objectstack/spec assertion failures in scripts/liveness/check-liveness.test.ts on shards 1 and 2. The 19th is shard-4 job 127654182380, whose log stops in the middle of the verify suite, so its cause was not measured. #30873's rest case is not among these 29 jobs (its red was on a run outside this window's failed set).

The population: cases on the 5 s default that run 2-5 s in CI (sample)

Sample: all 44 successful shard jobs of 6 main push runs spread over the window (41615880546, 41887304051, 41996918981, 42086692390, 42136924584, 42181191624), plus the failing job above. vitest prints every case over 300 ms, which gives 3,162 (package, file, case) rows. 402 of them peaked at 2-5 s. Each was classified from its source with the TypeScript AST (the it/test call's own timeout argument, its enclosing describe's, or the package config's testTimeout): 157 carry a per-case timeout, 16 inherit one from a describe, 180 sit in a package with a raised testTimeout (spec 60 s, plugin-auth 10 s, ...), 18 are it.each rows that all carry one. 31 are on the 5 s default. The ones that run closest to the line, beyond #30785's and #30873's:

Case CI ms (n, median, max) Local idle
packages/plugins/plugin-hono-server/src/adapter-tls-listener.test.ts › drains on close exactly as the plain listener does n=2, 4228, 4239 4209 (twice)
packages/plugins/plugin-hono-server/src/adapter-drain.test.ts › lets an in-flight request complete instead of aborting it n=2, 3303, 3321 3230 (twice)
packages/client/src/auth-rotated-session-token.test.ts › login → enable → verifyTotp → disable → deleteUser, no hand-repaired credential n=5, 1424, 3427 —
packages/cli/src/utils/unmanaged-tables.integration.test.ts › the boot this sweep reads from really did compose the deployment n=5, 2892, 3295 —
packages/metadata-protocol/src/reference-sites.derivation.test.ts › two derivations of the same schemas agree exactly n=2, 2548, 3210 —
packages/plugins/plugin-security/src/org-admin-assignment-boundary.posture.test.ts › posture 'single' / 'isolated' / 'group': a column the caller's field rules hide is never named in the refusal… n=3, 1989-2228, 2554-2949 —
packages/cli/test/released-port-production-spawn.pin.test.ts › the census reaches the live tree: every way of spawning the CLI, and the draw, are found n=6, 2301, 2629 —
packages/plugins/plugin-sharing/src/share-link-password.test.ts › a refused upgrade write does not block the read… n=4, 2078, 2605 —
packages/drivers/driver-sql/src/logger-receiver-detach.test.ts › sql-driver.ts and the modules split out of it contain none of them n=2, 2470, 2499 —

The two plugin-hono-server drain cases are not load-driven. They take the same 3.2 s and 4.2 s on an idle box, every run. close() calls closeIdleConnections() while the request is still in flight, so the socket that goes idle after it waits for the client's keep-alive to end. That is a hypothesis from reading adapter.ts, not measured. None is fixed here, because each one sits in another package and outside this claim's file surface. The seat filed them as one class card, #30920, led by the two drain cases. The shard-4/8 reading is on the on-hold shard card #25870 (with the pooling caveat above), for its re-derivation after the weekly refresh.

Verification

Gates derived by node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack for the final diff, run at ``4588caea1c(the merge oforigin/main`)`:

48 commands, each exit code taken before any pipe
node scripts/check-closing-keyword-parity.mjs :: exit 0
node scripts/check-closing-keyword-parity.mjs --self-test :: exit 0
node scripts/check-comment-mask-adoption.mjs :: exit 0
node scripts/check-comment-mask-adoption.mjs --self-test :: exit 0
node scripts/check-comment-mask-corpus.mjs :: exit 0
node scripts/check-dts-emitted.mjs --self-test :: exit 0
node scripts/check-issue-citations.mjs :: exit 0
node scripts/check-plugin-teardown-shape.mjs :: exit 0
node scripts/check-plugin-teardown-shape.mjs --self-test :: exit 0
node scripts/ci/assert-turbo-remote-cache.mjs --self-test :: exit 0
node scripts/ci/plan-test-core-shards.mjs --self-test :: exit 0
node scripts/ci/prune-turbo-cache.mjs --self-test :: exit 0
node scripts/docs-audit/check-affected-docs.mjs :: exit 0
node scripts/docs-audit/check-drift-comment.mjs :: exit 0
pnpm --filter @objectstack/spec run check:duration-unit-keys :: exit 0
pnpm check:ci-filter-parity :: exit 0
pnpm check:closed-ceremony-text :: exit 0
pnpm check:cross-package-test-inputs :: exit 0
pnpm check:doc-authoring :: exit 0
pnpm check:driver-memory-census :: exit 0
pnpm check:dts-closure :: exit 0
pnpm check:dual-build-cjs-loads :: exit 0   (first pass: exit 3, no dist/; rerun after a full build)
pnpm check:engine-double-contract :: exit 0
pnpm check:gitlink-declared :: exit 0
pnpm check:issue-citations :: exit 0
pnpm check:keyed-text-bounds :: exit 0
pnpm check:lean-entry-closure :: exit 0   (first pass: exit 3, no dist/; rerun after a full build)
pnpm check:logger-receiver-detach :: exit 0
pnpm check:nul-bytes :: exit 0
pnpm check:objectql-double-limit :: exit 0
pnpm check:org-identifier :: exit 0
pnpm check:page-declaration-shape :: exit 0
pnpm check:platform-object-tenancy-census :: exit 0
pnpm check:published-files :: exit 0
pnpm check:query-options-erasure :: exit 0
pnpm check:refd-timer-probe :: exit 0
pnpm check:registry-log-declared :: exit 0
pnpm check:rest-log-spy-declared :: exit 0
pnpm check:slot-lookup :: exit 0
pnpm check:sourcemap-no-sources-content :: exit 0
pnpm check:system-context-census :: exit 0
pnpm check:test-source-alias :: exit 0
pnpm check:tier-file-adoption :: exit 0
pnpm check:type-check-coverage :: exit 0
pnpm check:type-check-debt :: exit 0
pnpm check:undeclared-dep-imports :: exit 0
pnpm check:watch-hint-literal :: exit 0
pnpm check:where-matcher :: exit 0

--ran reconciliation: ✓ dispatch-gates --ran: 48 derived famil(ies) accounted for — 48 run, 0 NOT-MEASURED (a DERIVED zero — all 48 recorded an exit code and none of them is 3).

CI: this PR's check-runs are the evidence for every family CI runs. Test Core reruns the formula suite.

🤖 Generated with Claude Code

…alidateExpression sweep a measured budget

The validateExpression sweep in expression-refusal-codes.test.ts provokes
3,865 refusals, 199 of them distinct, and re-ran the same assertions on
every duplicate. It now judges each distinct refusal once (keyed on the
whole refusal), keeps counting every refusal, and still requires every
code to be provoked.

The breadth of the sweep is its claim and stays. The case is synchronous
and took 5034 ms on a contended Test Core (4/8) shard, so it carries an
explicit budget of 3x that reading, with the readings in its comment.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ihL2ebbZkr2QAEG1x7UC8
@objectstack-fleet objectstack-fleet Bot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Nov 6, 2026
@github-actions github-actions Bot added the size/s label Nov 6, 2026
@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs.

What this run could not see

Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 1697d5ca9ca45205583bf7d381a91b95bf2271f8 → packageMentionDocs.

The two counts in the validateExpression sweep's comments now say where
they were read: the 3,865 / 199 refusal count at 98f640d, and the
429-3065 ms band as seven sampled Test Core jobs.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ihL2ebbZkr2QAEG1x7UC8
@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

domain:devx#2 · marchtian · session_017ihL2ebbZkr2QAEG1x7UC8

Contract review

Served-tier: CONTRACT_REVIEW_TIER
Head-sha: 4588caea1c1cbaa7a2d4d8b8c5589aa7749582e6
Local-runs: none
Scope: packages/formula/src/expression-refusal-codes.test.ts + packages/formula/src/validate.ts, packages/formula/src/cel-engine.ts, packages/formula/src/expression-refusal.ts, packages/cli/test/hook-timeout-override-refusal.test.ts, packages/plugins/plugin-sharing/src/share-link-password-webcontainer.test.ts, packages/formula/package.json, packages/formula/vitest.config.ts

Inputs read: card #30901 body and both comments (seat claim 6682594819, dev report 6683692239, read as claims); PR #30919 body, labels (size/s, tests, skip-changeset), file list and the net diff origin/main...4588caea1c fetched into a ref of my own (1 file, +43 −1, three commits on top of origin/main 6ff27b3); the test file at main and at the head; the producers it calls; the Actions API read-only; the head's check-runs at 2026-11-06T11:33:57Z. The dispatch order and the dev's and seat's conclusions were not inputs to the verdict.

① Derived judgments

(1) No weakened assertion — holds. The key is inspect(said, { depth: Infinity, breakLength: Infinity, maxArrayLength: Infinity, maxStringLength: Infinity }) over the whole refusal. ExprValidationError (validate.ts line 298) is { message, source } & ExpressionRefusal = { code, params }, and refusal() at line 316 is the single constructor, building exactly those four fields. The five assertions in the case read only said.code, said.params (typeof, not null, Object.entries values flattened by paramStrings), said.message and said.source (expectLocatorsRead reads said.source.length). All four are in the key; inspect at infinite depth tells undefined from absent, NaN from null, -0 from 0, and escapes strings injectively, so two refusals that differ in anything an assertion reads cannot share a key (key order differences split, never merge — the safe direction). refusals++ runs before the continue, so every refusal is still counted and the toBeGreaterThan(100) control is unchanged; a duplicate's code equals the judged one's, so provoked still has to reach every code in VALIDATE_CODES. The three loops, SWEEP_INPUTS (40 sources x 3 spellings + 8 = 128, counted in the file) and SWEEP_SCHEMAS (15, counted) are untouched, so the case judges exactly the same 5,760 calls. The one thing the cut changes is diagnostic, not evidential: on a red, the ${said.code} from ${JSON.stringify(input)} message names the first input that provoked that refusal rather than every one. Not a weakening.

(2) The timeout — holds. VALIDATE_SWEEP_BUDGET_MS = 16_000 is passed as the it() third argument on the one case, not a testTimeout in packages/formula/vitest.config.ts (which carries none), and the diff adds no .skip, no .retry, no retry: option. Sizing follows the repo's per-case rule as share-link-password-webcontainer.test.ts lines 90-93 state it and hook-timeout-override-refusal.test.ts lines 121-123 adopt it — 3x the slowest reading, rounded up to the next second: 3 x 5034 = 15102, so 16000. The reading is named with its run and job (main push run 42192379976, job 127808107135) and I read it in that job's log: every refusal validateExpression reports on the sweep ... 5034ms, Error: Test timed out in 5000ms. at expression-refusal-codes.test.ts:870:3, the second sweep at 37 ms, the file at 5438 ms. The synchronous-body note is right: vitest judged the case after it returned, so 5034 is the whole run. The file-level 10 s rule (AGENTS.md, ruling record 6181973363) is not engaged — the file read 5.4 s in the failing job and the readings put it well under on an idle box. The 5034 reading is of the pre-cut case, so the budget is conservative for the post-cut body; the comment says to re-measure when inputs or schemas grow.

(3) The measurement — holds on spot-check, with one honesty gap. Spot-checked read-only against the Actions API:

  • Job 127808107135: Test Core (4/8), main push on d55acb7, conclusion failure, the case at 5034 ms as above; the shard plan line in the same log reads all bins: 1553/1556/1551/1552/1552/1553/1556/1554s, as the body quotes.
  • Shard walls on two cited push runs: 42181191624 — 1/8 1082 s, 2/8 407 s, 3/8 711 s, 4/8 1034 s, 5/8 400 s, 6/8 567 s, 7/8 318 s, 8/8 726 s; 42136924584 — 1/8 1120 s, 2/8 464 s, 3/8 605 s, 4/8 1467 s, 5/8 677 s, 6/8 512 s, 7/8 564 s, 8/8 733 s. Shard 4 slowest in both, shards 5-8 the fastest, consistent with the body's medians (4/8 1189 s vs 612-948 s).
  • The failed-job census: the workflow is 224644133 (ci.yml); the window 2026-11-03T10:00Z to 2026-11-06T10:30Z now holds 324 push and 665 merge-group runs (the body's 320/660 were read an hour earlier; the drift is the window's live tail), 63 failed runs, exactly 29 failed Test Core (N/8) jobs. Grepping each of the 29 logs: 10 carry Test timed out in 5000ms — 9 in packages/cli/test/validate-build-gate-parity.test.ts (all 4/8) and 1 the formula case; 18 are scripts/liveness/check-liveness.test.ts assertion failures on 1/8 and 2/8; job 127654182380 (4/8) matches nothing — its log ends inside verify, as the body says. The body's numbers hold exactly.
  • The near-5 s list: cli unmanaged-tables.integration.test.ts › the boot this sweep reads from really did compose the deployment read 3295 ms in run 42181191624, the table's max. The two plugin-hono-server rows were not in the run I read (that run did not execute hono-server) and stay the dev's reading, unverified here.
  • Sample honesty: the body says "a sample of 7" for the CI case readings, "sample" for the 44-job case census, and names every query. One caveat it does not state (should-fix, below): run 42181191624, one of its six sampled push runs, planned 37 of 84 items (shard 1/8 got 1 item), so the per-shard medians pool path-filtered push runs with full merge-group runs. The ranking survives — shard 4 is slowest in both runs I read — but the absolute medians are a mixed population.

(4) Fixes honesty — stands on the card's own ownership clause, with two acts owed by the seat. The card's Wanted says "rebalance the shards, cut the heavy work out of these cases, and/or set explicit per-case timeouts sized from the measurement, each with its reason", then "This card owns the measurement and the formula case." Both owned halves are in the tree and the PR body: the formula case carries a cut and a measured per-case budget with its reason, and the measurement (per-shard walls, the 29-job census, the 31-case default-5 s population) is in the body's acceptance notes with its queries. The shard plan is correctly not edited: scripts/test-shard-timings.json opens GENERATED by scripts/measure-test-shard-timings.mjs -- do not hand-edit, shard-timings-refresh.yml runs on cron: '30 5 * * 1', and the splitter's dataset was last carried on 2026-11-03 (4b71a82). What Fixes #30901 does not close: the measured shard-4/8 overload (1.53x the mean median) and the nine near-5 s cases — the dev reports them under out_of_scope_findings with carrier: seat to file and filed_issues: []. Under Prime Directive #10 the two deterministic plugin-hono-server drain cases (4209 ms and 3230 ms on an idle box, every run, 84% and 65% of the default) are a reproducible defect, not a note. For Fixes to stand at close the seat files (a) the shard-4/8 readings onto #25870 (open, pm:on-hold, the card that owns re-deriving the cli slicing) or a fresh card if #25870 stays on hold, and (b) one class card for the default-5 s population led by the hono-server cases, and the PR body's acceptance notes name the carriers by number. Should-fix, not a FAIL: the acts are the seat's at ACCEPT and sit outside the tree, and the body already names them with a carrier.

(5) The landing — holds. skip-changeset is right: @objectstack/formula publishes files: ["dist", "README.md", "CHANGELOG.md"], the diff touches one src/*.test.ts, and Check Changeset is green. Clause-②: no and Metadata-protocol: none are line-initial in the body and match a test-only diff. The body matches the tree: one file, +43 −1, inspect import, refusalKey, VALIDATE_SWEEP_BUDGET_MS with its comment, judged set, the budget as third argument; "no global testTimeout, no skip, no retry" is true of the diff and of the formula vitest config. The diff stays out of packages/rest and packages/cli/test/validate-build-gate-parity.test.ts. No governed path, no packages/spec, head repo is the base repo (not a fork), PR is draft and bot-authored for the owning seat to land. node:util is a builtin and two other formula tests already import node: modules.

Check-runs on the head at 2026-11-06T11:33:57Z — the gate verdicts, some still running: success — Lint & Repo Gates, Build Core, Governed Surface Queue Guard, Build & E2E UI, Live E2E UI, Check Changeset, Check PR Size, Plan Test Core shards, Lint · PM dispatch-gates, every completed Lint · and Type Check · lane; skipped — Lint UI (path filter); in progress — Test Core (1/8) through (8/8), Test UI (1/16) through (16/16), Type Check UI, Type Check · workspace. Test Core reruns the formula suite and is the one that re-measures this case; the PASS below is on the tree and does not stand in for its conclusion.

② Semver level

None. A test-only diff in packages/formula/src; nothing the package publishes moves. skip-changeset holds; Clause-②: no, Metadata-protocol: none.

③ Boundary flags

④ Design judgment

The change holds against the code it cites. refusalKey is sound because the refusal is a plain four-field value object built in exactly one place and every assertion is a pure function of that object; the dedup is a memo of a deterministic verdict, not a filter on what is judged. The budget is sized and documented to the repo's own per-case rule and pinned to a named CI reading, placed on the one case, and the second sweep (5-37 ms) is rightly left on the default. The two readings that are counts without a tree (should-fix 1) are the only drift from the house standard "a reading is a count plus the tree it was taken against".

Must-fix: none.

Should-fix:

  1. expression-refusal-codes.test.ts — the in-case comment "The sweep's 3,865 refusals are 199 distinct ones" and the budget comment's "Other Test Core jobs of the same three days: 429 to 3065" are readings with no tree and no sample size: name the commit the counts were taken on (the body's 98f640d581) and say the 429-3065 band is seven jobs, or drop the numbers. The structural counts (5,760 = 3 x 128 x 15) are fine — they are the file's own arrays.
  2. PR body, acceptance notes — state that the per-shard medians pool path-filtered main push runs (run 42181191624 planned 37 of 84 items) with full merge-group runs, or restrict the table to full-plan runs, before ci(test-shards): Test Core's two slowest shards set every core PR's wall clock at ~28 min (5.6 min dependency build + ~20 min tests) while the other six finish in 7–17 — re-derive the cli slicing and shard balance on post-merge readings #25870 re-derives the cli slicing from it.
  3. For Fixes #30901 to stand at close, the seat files the two carriers the dev report leaves to it — the shard-4/8 overload onto ci(test-shards): Test Core's two slowest shards set every core PR's wall clock at ~28 min (5.6 min dependency build + ~20 min tests) while the other six finish in 7–17 — re-derive the cli slicing and shard balance on post-merge readings #25870 (or a fresh card while it is on hold) and one class card for the default-5 s population led by the two deterministic plugin-hono-server drain cases (a PD chore: version packages #10 reproducible defect, noted but unfiled) — and the body names them by number.

Implemented-by: claude/issue-30901-test-core-timeouts
Reviewed-by: session_017ihL2ebbZkr2QAEG1x7UC8

VERDICT: PASS

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

domain:devx#2 · marchtian · session_017ihL2ebbZkr2QAEG1x7UC8

Contract review

Served-tier: CONTRACT_REVIEW_TIER
Head-sha: 649a0ab7df90d6e2edafbca3e32bbba30f7ed37f
Local-runs: none
Scope: packages/formula/src/expression-refusal-codes.test.ts + none

Delta review on the first record (comment 6683972205 on this PR, head 4588cae, PASS with three should-fixes). Read: git diff 4588caea1c..649a0ab7df (one commit, 649a0ab test(formula): name the sample behind the sweep budget's readings, 1 file, +6 −5, comments only); the live PR body; #25870 comments 6683757685 and 6683959694; card #30920; the check-runs on 649a0ab at 2026-11-06T11:41:41Z. Everything the first record judged on the tree is unchanged by this commit: no code line moves, so ① (the key, the counting, the inputs), ② (the per-case budget) and ⑤ (skip-changeset, Clause-②: no, Metadata-protocol: none, one file, no governed path) stand as written there and are not re-argued.

① Derived judgments

S1 — the comment readings now name a tree and a sample size; one of the two counts is wrong.

  • The in-case comment now reads "At 98f640d the sweep's 3,865 refusals are 199 distinct ones". 98f640d is the origin/main the PR body's before-column names and the commit the dev measured against; the tree is named. Holds.
  • The budget comment now reads "Seven other Test Core jobs sampled from the same three days: 429 to 3065." The PR body's own row (unchanged) is "a sample of 7 that ran formula" with the values 429, 1161, 2122, 2177, 2968, 3065, 5034 — seven jobs including the 5034 one, which the line above it already names on its own. The other jobs are six: 429, 1161, 2122, 2177, 2968, 3065. The dev report's ci_sample is the same seven values. So "Seven other" overstates the sample by one. The band 429 to 3065 is right. Must-fix: the word is "Six", or "Seven jobs in all, the other six: 429 to 3065". This is the exact line S1 asked to make honest, and the budget comment is the authority the next re-sizing reads, so a false count there is a reading that is not one (AGENTS.md: a reading is a count plus its tree, or it is not a reading).

S2 — done. The body's Source line now continues: "The population pools path-filtered main push runs with full-plan merge-group runs (sampled push run 42181191624 planned 37 of 84 items, and its shard 1/8 got one item), so a push run's shard reads low. Restrict to full-plan runs before re-deriving the cli slicing from these numbers." That is the caveat I measured, stated where the numbers are. #25870 comment 6683959694 (11:39:35Z) carries the same caveat with the two spot-checked push runs (1034 s and 1467 s on 4/8 against 318-733 s on 5-8), marked "not a claim; the card stays pm:on-hold". Holds.

S3 — done; Fixes #30901 now stands.

Check-runs on 649a0ab at 11:41:41Z — early: success so far on Governed Surface Queue Guard, Check PR Size, Dogfood Regression Gate, PR-body guards, Lint · PM dispatch-gates, Lint · scoped · tooling self-tests, Auto Label, filter; in progress — Build Core, every other Lint · and Type Check · lane, Plan Test Core shards, Live E2E UI; Test Core, Test UI, Type Check UI and Check Changeset not yet started. On the superseded head 4588cae, Test Core 2/8, 5/8, 7/8 and 8/8 completed green and 1/8, 3/8, 4/8 and 6/8 were cancelled by this push, so the formula case still has no CI reading after the cut; this head's Test Core is where that lands.

② Semver level

None — unchanged from the first record; comment-only delta.

③ Boundary flags

None new. One test file; no governed path; packages/rest, the cli parity test and scripts/test-shard-timings.json untouched.

④ Design judgment

The dedup key and the budget are as judged on 4588cae. The delta is the honesty of the comment that documents the budget: it now carries its tree and its sample size, and the sample size is off by one in the direction that overstates the evidence.

Must-fix:

  1. packages/formula/src/expression-refusal-codes.test.ts, budget comment: "Seven other Test Core jobs sampled from the same three days: 429 to 3065" — the sample is seven jobs including the 5034 one named on the line above; the others are six (429, 1161, 2122, 2177, 2968, 3065). Write "Six other ..." (or "seven in all; the other six: 429 to 3065").

Should-fix: none remaining. S2 and S3 are closed as stated above.

Implemented-by: claude/issue-30901-test-core-timeouts
Reviewed-by: session_017ihL2ebbZkr2QAEG1x7UC8

VERDICT: FAIL

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

domain:devx#2 · marchtian · session_017ihL2ebbZkr2QAEG1x7UC8

Contract review

Served-tier: CONTRACT_REVIEW_TIER
Head-sha: 3f0ddde822903f6400d32ff754357c7e6c7595ee
Local-runs: none
Scope: packages/formula/src/expression-refusal-codes.test.ts + none

Second delta review, on the FAIL record for 649a0ab (comment 6684040945 on this PR) and the PASS record for 4588cae (comment 6683972205). Read: git diff 649a0ab7df..3f0ddde822 (one commit, 3f0ddde test(formula): the budget comment's sample is six other jobs, not seven, 1 file, +2 −2, one comment line and its rewrap); the net diff origin/main...3f0ddde822 (still 1 file, +44 −1, the same test file); the PR state (draft, head 3f0ddde, labels size/s, tests, skip-changeset); the check-runs on 3f0ddde at 2026-11-06T11:44:46Z. No code line moves in this commit, so the first record's ① (the inspect key, every refusal counted, the same 5,760 calls), ② (the per-case 16_000 budget by the 3x rule, no global raise, no skip, no retry), ③, ④ and ⑤ (skip-changeset, Clause-②: no, Metadata-protocol: none, no governed path, not a fork) stand as written and are not re-argued; the delta record's S2 and S3 closures (the pooling caveat in the body and on #25870, #30920 filed, both carriers named by number) are untouched by this commit and stand.

① Derived judgments

The must-fix is closed, and the count is now true. The budget comment reads: "The six other Test Core jobs sampled from the same three days: 429 to 3065." The sample the PR body names ("a sample of 7 that ran formula") is 429, 1161, 2122, 2177, 2968, 3065 and the 5034 job named on the line above — six others, with 429 and 3065 as their band. The dev report's ci_sample is the same seven values. Count, tree (run 42192379976, job 127808107135 for the 5034 reading; "the same three days" with the body's window) and band now agree with the readings I spot-checked. The in-case comment "At 98f640d the sweep's 3,865 refusals are 199 distinct ones" is unchanged from 649a0ab and holds.

Nothing else changed. The diff is the one comment line and its rewrap; VALIDATE_SWEEP_BUDGET_MS = 16_000, refusalKey, the judged set, the refusals++ before the continue, the provoked control and the third-argument budget are byte-identical to 4588cae. The net diff against origin/main is the single test file, +44 −1.

Fixes #30901 stands, as the delta record set out: the card's own ownership clause (the measurement and the formula case) is delivered in the tree and the body; the shard-4/8 reading is on #25870 (comments 6683757685 and 6683959694) and the nine near-5 s cases are #30920; the body names both.

Check-runs on 3f0ddde at 2026-11-06T11:44:46Z — the run has just started: success — Governed Surface Queue Guard, Check PR Size, PR-body guards, Auto Label; in progress — Flag docs affected by code changes; not yet reported — Lint & Repo Gates, TypeScript Type Check, Test Core (1/8) to (8/8), Build Core, Check Changeset, Lint UI, Type Check UI, Test UI, Build & E2E UI, Live E2E UI. On the superseded head 649a0ab, Test Core 2/8 and 5/8 were still running and 1/8, 3/8, 4/8, 6/8, 7/8 and 8/8 were cancelled by this push; on 4588cae before it, 2/5/7/8 had completed green and 1/3/4/6 were cancelled. So the formula case still has no Test Core reading after the cut; this head's Test Core is where it lands, and the landing waits on every check being green as the directive requires. This PASS is on the tree and does not stand in for those conclusions.

② Semver level

None — comment-only delta on a test-only diff; skip-changeset holds.

③ Boundary flags

None new. One test file; no governed path; packages/rest, packages/cli/test/validate-build-gate-parity.test.ts and scripts/test-shard-timings.json untouched.

④ Design judgment

As judged on 4588cae: the dedup key is a complete memo of a deterministic verdict over a four-field value object built in one place, and the budget is per-case, measured, and documented to the repo's own rule. The documentation now carries its tree and a correct sample size.

Must-fix: none.

Should-fix: none.

Implemented-by: claude/issue-30901-test-core-timeouts
Reviewed-by: session_017ihL2ebbZkr2QAEG1x7UC8

VERDICT: PASS

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

domain:devx#2 · marchtian · session_017ihL2ebbZkr2QAEG1x7UC8

席位核对 — ACCEPT · domain:devx#2 · Head-sha 3f0ddde822903f6400d32ff754357c7e6c7595ee · 2026-11-06T12:12Z

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/s skip-changeset PR has no user-facing published change; bypasses the changeset gate tests

Projects

None yet

1 participant