Repository navigation
fix(scenarios): the run grader reads a hook's declared refusal off the door's declaredCode - #33556
objectstack-fleet[bot] wants to merge 1 commit into
Conversation
…e door's declaredCode The MCP door demotes a hook's own refusal code to `declaredCode` and serves the status's catalogue code on `code`. The runner kept only `code` and `status`, so the grader read the hook's ASSET_NOT_IN_STOCK at 409 as a platform RESOURCE_CONFLICT and failed a declared VALIDATION_FAILED. A refused ObservedAnswer now carries `declaredCode` when served, and refusalMeets reads it: a declared code equal to it meets exactly, and a declared VALIDATION_FAILED is met by one at any 4xx. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FEkQ5nw3MDkRymEzv6FDJs
📓 Docs Drift CheckThis PR changes 1 package(s): 3 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:
⛔ 2 release-owned page(s) also name something this change touched. These are read-only:
What this run could not see
Coarse fallback — 1 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 9b77b1aa9caeffc7922f9dd71182dea61bea613a && git checkout 9b77b1aa9caeffc7922f9dd71182dea61bea613a
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin e3f0cfd422d69100448ccd4f6a329c90a25715f7 06330bd9bf55a195473553a36c959f3b21a58265 && git checkout -B drift-repro e3f0cfd422d69100448ccd4f6a329c90a25715f7 && git merge --no-ff 06330bd9bf55a195473553a36c959f3b21a58265
node scripts/docs-audit/affected-docs.mjs --json e3f0cfd422d69100448ccd4f6a329c90a25715f7
|
|
domain:devx#4 · os-project-manager · session_01FEkQ5ooRyyS95kdyC2dPEX 席位核对 · ACCEPT · head
|
Requested by Leehom · project thread
domain:devx#4 · os-project-manager · session_01FEkQ5nw3MDkRymEzv6FDJs
Fixes #33497
Clause-②: no
Metadata-protocol: none
维护者速读
declaredCode(hook 自己声明的拒绝码,被门降级到code旁边)保留到ObservedAnswer上;评分器refusalMeets读它:声明码等于declaredCode时精确命中,声明VALIDATION_FAILED时 4xx 上带declaredCode即算 app 自己的拒绝。RESOURCE_CONFLICT409 +declaredCode: ASSET_NOT_IN_STOCK,runner 只留了code,评分器看到的是平台冲突码,于是 app 正确拒绝的check_out_asset_in_repair被判失败;原测试把 hook 码直接塞进code,门从来不这么返回。declaredCode的目录码不命中别的声明码,5xx 或无 status 的declaredCode不命中。回滚即还原本 PR。packages/qa/scenarios/src/score/scenario-operation-grade.ts的refusalMeets(L225)和refusalText(L237);packages/qa/scenarios/src/runner/observe.ts的declaredCodeOf(L218)、refusedBy(L242)。pnpm --filter @objectstack/scenarios exec vitest run test/runner/mid-tier-run-readings.test.ts,预期 49 条全绿。What changes
packages/qa/scenarios/src/runner/observe.ts.refusedBy(L242) now takes the tool output and returns the refused answer withcode,statusanddeclaredCode, each only when served.declaredCodeOf(L218) readsdeclaredCodeoff the same enveloperefusalOfreads (the same two JSON readings; the nestederrorobject when it carries a stringcode, else the flat body), and answersundefinedfor an absent or empty one: no default is invented. The three refused-answer paths go through it: a write call's refusal (refusalAnswerOf), the last read on the object when no write was made (lastReadRefusalOutput, L147, read at L270), and the runner's probe (observeProbebuilds a one-call turn and reads it throughobservedAnswer).scenario-operation-grade.tsL94.ObservedAnswer's refused member gains the optionaldeclaredCode?: string.refusalMeets(L225). (a) A declared code equal to the answer'sdeclaredCodemeets exactly, likecode === declared. (b) For a declaredVALIDATION_FAILED, a presentdeclaredCodeat a 4xx status meets, whatever the cataloguecodebeside it. A catalogue code with nodeclaredCodestill never meets a different declared code, and adeclaredCodeat 5xx or with no status does not meet.refusalText(L237) names what met:refused RESOURCE_CONFLICT (409), the hook's declared refusal ASSET_NOT_IN_STOCK.refusalMeetsdocblock andpackages/qa/scenarios/README.mdL70–L75 now describe the wire shape:codeis the status's catalogue code,declaredCodeis the hook's own.The wire codes the tests pin are the door's (
HttpStatusErrorCodeMap,packages/spec/src/api/errors.zod.tsL183, with 4xx otherwiseVALIDATION_ERRORthroughstandardErrorCodeForHttpStatus): 400 and 422 →VALIDATION_ERROR, 403 →PERMISSION_DENIED, 404 →RESOURCE_NOT_FOUND, 409 →RESOURCE_CONFLICT, 500 →INTERNAL_ERROR.Tests
All in
packages/qa/scenarios/test/runner/mid-tier-run-readings.test.ts.OVER_RECEIPTat 400, 403, 404, 409 and 422 is served as the status's catalogue code withdeclaredCode: 'OVER_RECEIPT', and meetsVALIDATION_FAILED. Negatives: each catalogue code with nodeclaredCode(VALIDATION_ERROR400,PERMISSION_DENIED403,RESOURCE_NOT_FOUND404,RESOURCE_CONFLICT409,INTERNAL_ERROR500) does not meet;declaredCodeat 500 does not meet;declaredCodewith no status does not meet; a hook'sNOT_REQUESTERserved at 400 does not meetPERMISSION_DENIED.RESOURCE_CONFLICT409 +declaredCode: 'ASSET_NOT_IN_STOCK'meets a declaredASSET_NOT_IN_STOCK(andRESOURCE_CONFLICT); the same conflict with nodeclaredCodedoes not meetASSET_NOT_IN_STOCK; a different declared code (OVER_RECEIPT) does not meet.VALIDATION_ERROR400 +declaredCode, with the detailrefused VALIDATION_ERROR (400), the hook's declared refusal ….check_out_asset_in_repaironRESOURCE_CONFLICT409 +declaredCode: 'ASSET_NOT_IN_STOCK', both halves passing with the new detail;RESOURCE_CONFLICT409 with nodeclaredCodefails.VALIDATION_ERROR400 +declaredCode: 'NOT_APPROVER'.asset_lifecycle check_out_asset_in_repair:declaredCodereaches theObservedAnswerthrough acreate_recordcall and through the probe, in the nested ({ error: { code, message, status, declaredCode } }) and the flat ({ error, code, status, declaredCode }) shape;toStrictEqual);gradeCardOperation: the served shape passes both halves; the same conflict with nodeclaredCodefailsagent:refused:VALIDATION_FAILED-or-withheldandplatform:refused:VALIDATION_FAILED.Ablation (head
06330bd9,scripts/ablation-replace.mjs, anchor...(declaredCode !== undefined ? { declaredCode } : {}),inobserve.tsdeleted, anchor 1 → 0, blobd6a5337a5748→d9534323aac4; the tests importsrc/, so no dist is in the path):Ablated: 3 failed | 46 passed (49); restored: 49 passed
The "absent stays absent" case stays green under the ablation, as it should.
Acceptance notes
code. The/datafaces of the MCP record tools answer throughrefusalAsTheDoorServesIt(packages/mcp/src/mcp-http-tools.tsL990), which serves the door's cataloguecodewithdeclaredCodewhenever the door demoted (dataDoorRefusalEnvelope, L954); the resource body does the same (packages/mcp/src/mcp-server-runtime.tsL2179–L2187). The door's demotion isresolveThrownHttpError's code row (packages/types/src/thrown-http-error.tsL253: a registered code stays, anything else is derived from the status) withdemotedDeclaredCode(L526).run_actiondemotes the same way (packages/types/src/actions-door-answer.tsL58, L69–L70). So the legacycode: 'OVER_RECEIPT'andcode: 'ASSET_NOT_IN_STOCK'cases are dropped.refusalMeetsstill lets acodeoutside theErrorCodecatalogue at 4xx meet a declaredVALIDATION_FAILED(the branch before this change). It is kept so that no reading narrows beyond the order's rulings, and it is now unpinned because no door was found serving that shape. class: observation · reach: none · carrier: none. The seat may retire it on a later card.code: 'PERMISSION_DENIED'with itsdeclaredCode. It meets a declaredPERMISSION_DENIEDthroughcode === declared, as before this change, and now also a declaredVALIDATION_FAILEDthrough rule (b) (ruling A: any 4xx). class: observation · reach: none (no card declaresPERMISSION_DENIEDfor a hook's own refusal) · carrier: none.refusalOfleft as it is. It lives inpackages/qa/scenarios/src/score/scenario-run-grade.ts, outside the ordered surface, sodeclaredCodeis read inobserve.tsby a private reader that mirrors its envelope selection, rather than by wideningrefusalOf'sRefusal.ObservedAnswer.declaredCodeis an additive optional member of a published package type, ordered underClause-②: no.Verification
On head
06330bd9(branchclaude/project-thread-rl158j, merge baseorigin/maine3f0cfd4), each exit code captured before any pipe:pnpm --filter @objectstack/scenarios test: exit 0, Test Files 32 passed (32), Tests 712 passed (712).pnpm --filter @objectstack/scenarios typecheck(tsc --noEmit, which compiles the tests): exit 0.pnpm --filter @objectstack/scenarios build: exit 0.node --stack-size=4000 node_modules/eslint/bin/eslint.js packages/qa/scenarios --no-inline-config: exit 0 (the package has nolintscript).node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack, 59 families): 58 exit 0;pnpm check:dual-build-cjs-loadsexit 3 PREREQUISITE NOT MET (needs every package'sdist/), NOT MEASURED, CI runs it.--ran: "59 derived famil(ies) accounted for — 58 run, 1 NOT-MEASURED".dispatch-gates --tier: no path-derived mandate. 239 changed lines, under the 3000 threshold. No governed surface..changeset/33497-scenario-grader-declared-code.md(file name unique to this card).🤖 Generated with Claude Code
https://claude.ai/code/session_01FEkQ5nw3MDkRymEzv6FDJs