Skip to content

test: verify research readers never debit quota - #8764

Merged
cocolord merged 1 commit into
mainfrom
codex/r1-reader-ledger-20261114t1218
Nov 14, 2026
Merged

cocolord merged 1 commit into
mainfrom
codex/r1-reader-ledger-20261114t1218

Conversation

@cocolord

@cocolord cocolord commented Nov 14, 2026 •

Copy link
Copy Markdown
Member

The research-reader test could pass after a debit reached the actual quota ledger because its snapshots omitted runs/index.jsonl and the spend artifact. This closes the verified coverage gap from #8746 by comparing durable index bytes and the shipped quota reader after each reader, asserting one authorized spend, and preserving the ledger on replay.

Only tests/capabilities/test_research_composition_gate.py changes (+22/-0). It reuses load_index_snapshot and goal_quota_with_spend_ledger; no new product owner, protocol, activation, permission or execution pipeline.

Validation at a307e43 on main 45cfd0c:

  • Complete three-file consumers: 113 passed, 11 conditional real-native-Host skips.
  • Current C1 aab22d9 plus this correction: 122 passed, 14 conditional native/PowerShell skips; patch applies without conflict.
  • Real-debit mutation matrix: 8 passed. For File/SQLite and review/writeback, the old assertion accepts an event independently counted as one slot; the correction rejects it.
  • Premerge: five direct checks and six canaries pass. Ruff and diff/bilingual checks pass. All fixtures use isolated temporary roots outside Git.

研究 reader 的无副作用检查已补实际账本:每次读取后比对原 run index 字节与现有 quota projection,合法扣额恰一次,重放不改变账本。主干/C1 完整消费者与8项真实扣额故障注入验证通过,条件跳过保留。原生 Host/context、完整 R1、完整成本和原目标回归继续开放。

Review context: #8746 (review)

Signed-off-by: lusendong.6789 <lusendong.6789@bytedance.com>

@cocolord cocolord left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent; model=GPT-6; provider=OpenAI; declaration_source=self_reported (runtime observation unavailable: session_not_bound).

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)

动机 / Motivation

维护者需要判断研究任务的读取阶段是否真的不扣额度。 研究 reader 返回正常结果,原测试却可能在实际账本已多记一笔扣额后仍通过;补丁读取真实账本,两个读取边界都要求不变,合法结算只计一笔且重放不变。 File/SQLite 的研究 reader 现在有能检测真实入账的独立观察,并继续验证同一义务的后继、结果和结算。 本次只补既有测试的账本断言,不修改生产配额、权限、配置、协议、UI 或 C1 分支。 完整默认闭包、原生 Host/context 采用、渐进控制、可比完整成本和原目标回归仍未验收。

改动思路 / Approach

在既有真实研究 CLI 用例内补独立观察。run index 与其 quota artifact 才记录实际扣额,Registry、状态和 rollout 日志不足以替代它们。局部 quota_ledger helper 复用 load_index_snapshot 与 goal_quota_with_spend_ledger:每次 reader 后比较索引字节与真实 quota projection。第一次合法 spend 后要求 spent_slots=1、spend_event_count=1;重复同一命令后要求 ledger 不变,CLI 成功本身由已有 fixture 校验。

继续采用原回执、研究前沿、后继、结果和结算 owner,没有新增 schema、权限、激活策略或执行 pipeline。与 C1 的关系是增强同一现有消费者的观察,生产决策与状态语义保持原样。

具体改动 / Changes

完整 head a307e43:仅 tests/capabilities/test_research_composition_gate.py 增加22行。规范 docs/development/good-smokes.md,spec_revision 45cfd0c。Prefer Semantic Pressure:implemented,8个实际入账 mutation 用例证实旧断言放行、修复后拒绝。Consolidate Without Losing Coverage:implemented,沿既有测试与已交付 reader 补断言,未新增公共 smoke 或产品机制。

局部 quota_ledger 读取真实索引和 artifact;test_real_guard_and_successor_share_the_current_gap 在 review/writeback 后各比较一次,并在原义务合法结算与 replay 后比较计数及字节。原非法输入、失效前沿、阻塞、结果、archive/replay 断言继续执行。修复直接回应 #8746 的有效 P2 评审:https://github.com/loopx-project/loopx/pull/8746#pullrequestreview-5903956635。

对主干的风险 / Risks

未发现阻断性缺陷。主要风险是新增的观察仍看错账本或只检测形式变化;私有 fault matrix 使用已有 quota ledger-integrity fixture 的真实 artifact/index 形状,独立验证真实 reader 计入1 slot。File/SQLite × review/writeback × 旧/新断言共8例全部通过。旧测试的漏检被保留,注入仅发生于隔离 fixture。

验证范围 结果
当前完整三文件消费者 PASS:113 passed / 11 skipped
当前 C1 aab22d9 叠加修复 PASS:122 passed / 14 skipped
实际扣额 mutation matrix PASS:8例,旧断言漏检、新断言拒绝
premerge / lint PASS:5 direct checks、6 canaries、Ruff

主干跳过原因:{'Requires an explicit real native Host binary; static local provider only': 6, 'Requires a real native Host': 5};C1 跳过原因:{'Requires PowerShell 7': 3, 'Requires an explicit real native Host binary; static local provider only': 6, 'Requires a real native Host': 5}。这些环境未资格化。初次正例曾错误要求 spend/replay 响应具有相同 settlement_progress;保留该失败,最终采用 CLI 成功与真实账本不变的语义断言。历史 ENOSPC、旧 C1 完整性拒绝和未定位的旧归档仍独立保留,不被这次通过改写。远端 CI 和合入就绪独立读回。

我的整体评价 / Verdict

APPROVE,justified_increment。long_horizon=preserved:同一原义务和结算继续被执行,累计额外扣额有了独立可证伪的防线。user_experience=preserved:只改测试,无 UI、自动加载指令或用户交互变化。最小方案已复用现有账本 owner,22行增量与已证实缺口相称;本次无新增共享语义词汇。R1 的原生 Host/context、完整闭包、渐进控制、可比成本和原目标回归仍开放,不以测试通过宣告父目标完成。模型署名为 self_reported,不认证运行权重或授予权限。

English verdict: APPROVE — exact head a307e43; no blocking finding. Actual-ledger mutations now fail the corrected reader assertions, authorized spend is counted once and replay preserves the ledger. Full consumer/C1 and required premerge validation pass; conditional native gaps and GitHub readiness remain separate.

@cocolord
cocolord merged commit 8d7d234 into main Nov 14, 2026
6 of 9 checks passed
@cocolord
cocolord deleted the codex/r1-reader-ledger-20261114t1218 branch November 14, 2026 12:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant