Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
e75e0a4
Use model coverage in the PR review queue
loopx-agent Nov 18, 2026
1b7e3e7
Merge commit '07722b169a4edea313f6a17d12dd73819e3f8843' into codex/re…
loopx-agent Nov 18, 2026
9d85f67
Recognize dot-separated reviewer declarations in the queue
loopx-agent Nov 18, 2026
6f779ed
Merge remote-tracking branch 'origin/main' into codex/review-queue-cr…
loopx-agent Nov 18, 2026
672fa85
Recognize bare reasoning effort in review history
loopx-agent Nov 18, 2026
5da2fc3
Share reviewer declaration parsing across publication and the queue
loopx-agent Nov 18, 2026
8654c9b
Merge commit '27eaeb127777378d1e95d98e478ccfa779d08873' into codex/re…
loopx-agent Nov 19, 2026
8ba5248
Keep model coverage for shared independent review accounts
loopx-agent Nov 19, 2026
b8408bb
Merge commit '62784ecc532585d4f277dd94fe9a169fb95b5050' into codex/re…
loopx-agent Nov 19, 2026
43ae372
Merge commit 'ff905437669054465a4d7aa8c70ba3396aaf9182' into codex/re…
loopx-agent Nov 19, 2026
b5bb9ed
Require qualified conclusions for latest model coverage
loopx-agent Nov 19, 2026
e7deb67
Merge commit '6a0f8bc17b439f1e5fa1d389ad5a87faf7d73b25' into codex/re…
loopx-agent Nov 19, 2026
5dae2b3
Merge commit '45c015ffd9c9065b3af533e0abf103170627d15f' into codex/re…
loopx-agent Nov 19, 2026
7105188
Keep qualified review coverage after follow-ups
loopx-agent Nov 19, 2026
f84e25c
Merge remote-tracking branch 'origin/main' into codex/review-queue-cr…
loopx-agent Nov 19, 2026
33f4679
Preserve review coverage for each model family on shared accounts
loopx-agent Nov 19, 2026
f0ef7e9
Merge remote-tracking branch 'origin/main' into codex/review-queue-cr…
loopx-agent Nov 19, 2026
bbbf348
Merge commit '477def5c176ab4df92ac19ea1dd616828b325e87' into codex/re…
loopx-agent Nov 19, 2026
2e61725
Retain model coverage across unattributed review activity
loopx-agent Nov 19, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 26 additions & 5 deletions loopx/capabilities/pr_review_queue/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -632,15 +632,16 @@ evidence and do not pad. Invoking review uses the installed capability policy.

### Reviewer provenance and specification basis

Policy revision 14 makes two publication requirements mandatory for every new
result.
Policy revision 14 requires reviewer attribution and a specification basis for
every new result.

- `result.reviewer` (`reviewer_declaration`) names `actor_kind`
(`model_agent` or `human_operator`) and `declaration_source`
(`runtime_reported` or `self_reported`); an agent adds `declared_model` and
`declared_provider` in product-family wording. The body repeats them on
exactly one visible `Reviewer:` line, each value as a whole token. This is
provenance, not a credential: it authenticates nothing and adds no weight.
`declared_provider` in product-family wording. Use one visible line such as
`Reviewer: model_agent · MODEL · PROVIDER · self_reported`. Publication and
queue readback share the declaration parser. Model coverage requires an
explicit source. The declaration adds no weight or account authority.
- `problem_context.spec_basis` (`spec_basis_assessment`) is `mapped`,
`no_spec` or `not_yet_proven`. `mapped` gives text `spec_ref`,
`spec_revision` (a full commit id for `accepted_rfc` and
Expand All @@ -653,6 +654,26 @@ result.
reference as an error, and a spec-basis gap as an approval blocker. Regenerate
results made under revision 13 rather than adding these fields to them.

In a Goal-bound GitHub queue, a recognized model family from the invoking
runtime can reopen an open, non-draft exact head with a declared review from
another family. The queue uses complete provider history. An independent
review from the same family suppresses repeat work even when another account
submitted a later review. If different models share an independent GitHub
account, each valid, attributable formal review on the current head supplies
coverage while that account maintains the same formal opinion. The latest
qualified `COMMENTED` review from each known family in that history can also
supply coverage. A comment with unknown model provenance can record account
activity and preserves existing family coverage. A dismissal, a different head
or a changed formal opinion resets the account's coverage history.
A qualifying review published under the shared author
login with a matching runtime declaration also suppresses repeat work for that
model. GitHub independence still requires independent account evidence.

Missing model identity or incomplete history retains the existing queue
behavior. A review with unknown provenance supplies no model-family coverage.
The review action leaves the formal conclusion, dissent, CI policy, approval
and merge-readiness rules in their existing owners.

### Semantic alignment and CI constraint recovery

The semantic triage introduced in policy revision 5 replaces universal detailed
Expand Down
2 changes: 1 addition & 1 deletion loopx/capabilities/pr_review_queue/catalog_entry.py
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,7 @@
"A queue is observed only when result_completeness.complete=true; partial or failed reads are not_observed and never count as unchanged.",
"Fingerprints cover exact head, formal conclusion, next action, check state, draft state, and mergeability for every open PR.",
"Direction resolves from a command override, registered Agent, Goal, then machine. Forward visits other authors first, oldest first within tiers; reverse inverts the complete actionable queue before limiting. Packet and autonomous selection share that order. Inactive heads remain inventory-only. Legacy other-developers-first/owner-first input maps to forward/reverse, including reversed time ordering.",
"Only rows with a non-null review_action_kind enter review_sequence and carry review plans, templates, or evidence commands; valid exact-head conclusions remain artifact-free inventory-only rows, and only --fresh-audit-exact-head NUMBER@HEAD_OID can explicitly reopen one.",
"Only rows with a non-null review_action_kind enter review_sequence and carry review plans, templates, or evidence commands; valid exact-head conclusions retain their verdicts. A bound model-family coverage gap can request a full review of an open ready head. Explicit audits of otherwise concluded heads use --fresh-audit-exact-head NUMBER@HEAD_OID.",
"Todo prose, monitor notes, and one-off author filters are not scheduling authority.",
"A complete exact-head conclusion requires the five Chinese sections, a state-aligned English verdict, and formal state or the verdict-specific titled author-owned fallback.",
"Every merge must rerun the Goal-scoped merge-readiness gate for the reviewed exact head; the resulting compact observation suppresses only unchanged qualification work, and any head, base, review, CI, thread, draft, merge-state, or PR-state change reopens it.",
Expand Down
18 changes: 15 additions & 3 deletions loopx/capabilities/pr_review_queue/github_source.py
Original file line number Diff line number Diff line change
Expand Up @@ -244,12 +244,18 @@ def fetch_complete_pr_reviews(*, repository: str, number: str, cwd: Path | None,

def _attach_review_provenance(row: dict[str, Any], details: dict[str, Any], *,
repository: str, number: str, cwd: Path | None,
run_gh_json: GitHubJsonRunner) -> bool:
run_gh_json: GitHubJsonRunner,
model_review_coverage: bool = False) -> bool:
author = row.get("author")
author = author.get("login") if isinstance(author, dict) else None
# This only avoids unnecessary provider reads. Eligibility belongs to TS;
# no association, state or actor-type policy is duplicated here.
if not author or not any(isinstance(review, dict)
if not author or not details["reviews"]:
return True
model_history_needed = (
model_review_coverage and row.get("state") == "OPEN" and row.get("isDraft") is not True
)
if not model_history_needed and not any(isinstance(review, dict)
and isinstance(review.get("author"), dict) and review["author"].get("login")
and review["author"]["login"].casefold() != author.casefold()
for review in details["reviews"]):
Expand Down Expand Up @@ -292,6 +298,7 @@ def attach_pr_review_details(
cwd: Path | None = None,
run_gh_json: GitHubJsonRunner = run_gh_json,
wait_for_ci: bool = True,
model_review_coverage: bool = False,
) -> bool:
"""Attach complete per-PR details after the lightweight list scan."""

Expand Down Expand Up @@ -348,7 +355,8 @@ def attach_pr_review_details(
return False
details["files"] = detail_files
if not _attach_review_provenance(row, details, repository=repository, number=number,
cwd=cwd, run_gh_json=run_gh_json):
cwd=cwd, run_gh_json=run_gh_json,
model_review_coverage=model_review_coverage):
return False
for key in detail_fields:
row[key] = details[key]
Expand All @@ -362,6 +370,7 @@ def scan_github_pull_request_targets(
cwd: Path | None = None,
run_gh_json: GitHubJsonRunner = run_gh_json,
wait_for_ci: bool = True,
model_review_coverage: bool = False,
) -> dict[str, Any]:
"""Read only explicitly requested exact heads, without scanning a queue."""

Expand Down Expand Up @@ -402,6 +411,7 @@ def scan_github_pull_request_targets(
repository=repository,
cwd=cwd,
**({"wait_for_ci": False} if not wait_for_ci else {}),
**({"model_review_coverage": True} if model_review_coverage else {}),
run_gh_json=run_gh_json,
)
if not details_ok:
Expand Down Expand Up @@ -440,6 +450,7 @@ def attach_pr_review_details_concurrently(
attach: Callable[..., bool] = attach_pr_review_details,
run_gh_json: GitHubJsonRunner = run_gh_json,
wait_for_ci: bool = True,
model_review_coverage: bool = False,
) -> list[bool]:
"""Read per-PR details concurrently while preserving queue order."""

Expand All @@ -454,6 +465,7 @@ def read(row: dict[str, Any]) -> bool:
cwd=cwd,
run_gh_json=run_gh_json,
**({"wait_for_ci": False} if not wait_for_ci else {}),
**({"model_review_coverage": True} if model_review_coverage else {}),
)

with ThreadPoolExecutor(max_workers=worker_count) as executor:
Expand Down
90 changes: 90 additions & 0 deletions loopx/capabilities/pr_review_queue/model_coverage.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
"""Read model declarations for queue coverage without changing review authority."""

from __future__ import annotations

import re
from collections.abc import Mapping, Sequence
from typing import Any

from .review_body import parse_model_reviewer_line, reviewer_declaration_lines


# A product alias is evidence only when paired with its declared provider.
_PROVIDERS = {
"gpt": {"openai"},
"gemini": {"google", "google deepmind"},
"claude": {"anthropic"},
"deepseek": {"deepseek"},
"qwen": {"alibaba", "alibaba cloud", "qwen"},
}
_MODEL_NAME = re.compile(r"([a-z][a-z0-9]*)(?:[ .-][a-z0-9]+)*")


def _model_family(model: object, provider: object) -> str | None:
if not isinstance(model, str) or not isinstance(provider, str):
return None
name = _MODEL_NAME.fullmatch(model.strip().casefold())
if name and provider.strip().casefold() in _PROVIDERS.get(name[1], ()):
return name[1]
return None


def current_review_model_family(execution: Mapping[str, Any] | None) -> str | None:
if not execution or execution.get("status") != "runtime_reported":
return None
return _model_family(execution.get("model"), execution.get("provider"))


def published_review_model(body: object) -> tuple[str, str] | None:
if not isinstance(body, str):
return None
lines = reviewer_declaration_lines(body)
if len(lines) != 1:
return None
declaration = parse_model_reviewer_line(lines[0])
if declaration is None or declaration.source is None:
return None
family = _model_family(declaration.model, declaration.provider)
return (family, declaration.source) if family else None


def needs_cross_model_review(
*, family: str, history: dict[str, Any],
independent_coverage_ids: Sequence[int], independent_activity_ids: Sequence[int],
model_reports: Sequence[tuple[str, str] | None],
) -> bool:
"""Detect missing model coverage from one complete exact-head snapshot.

The history owner supplies current account activity and qualified
conclusions. Another family's submission can trigger a second opinion.
Suppressing it requires qualified coverage from the invoking family.
"""
author = history["author"].casefold()
head = history["head"].casefold()
coverage_ids = set(independent_coverage_ids)
activity_ids = set(independent_activity_ids)
other_family_review = False
for row, declared in zip(history["reviews"], model_reports, strict=True):
if (row["head"].casefold() != head or row["state"] not in
{"APPROVED", "CHANGES_REQUESTED", "COMMENTED"}):
continue
review_id = row["review_id"]
if type(review_id) is not int or review_id <= 0 or declared is None:
continue
reviewed_family, source = declared
if row["login"].casefold() == author:
if row["state"] == "COMMENTED" and row["conclusion"]["valid"]:
if reviewed_family != family:
other_family_review = True
elif source == "runtime_reported":
# A shared author login cannot establish GitHub independence.
# A complete review already published by this model family
# still prevents repeat assignment to the same model.
return False
continue
if reviewed_family == family:
if review_id in coverage_ids:
return False
elif review_id in coverage_ids or review_id in activity_ids:
other_family_review = True
return other_family_review
58 changes: 33 additions & 25 deletions loopx/capabilities/pr_review_queue/result_check.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
author_owned_review_title_matches,
check_review_body,
normalized_review_prose,
parse_model_reviewer_line,
review_section_text,
reviewer_declaration_lines,
visible_review_text,
Expand Down Expand Up @@ -251,7 +252,9 @@ def _published_as_token(reference: str, text: str) -> bool:
return re.search(pattern, text) is not None


def _reviewer_errors(value: object, body: str) -> list[str]:
def _reviewer_errors(
value: object, body: str, *, execution_present: bool = False, runtime_reported: bool = False,
) -> list[str]:
key = "reviewer"
contract = REVIEWER_DECLARATION
if not isinstance(value, Mapping):
Expand All @@ -260,26 +263,41 @@ def _reviewer_errors(value: object, body: str) -> list[str]:
if actor_kind not in contract["actor_kinds"]:
return [f"{key}:invalid_actor_kind"]
errors: list[str] = []
if value.get("declaration_source") not in contract["declaration_sources"]:
source = value.get("declaration_source")
if source not in contract["declaration_sources"]:
errors.append(f"{key}:invalid_declaration_source")
declared: list[str] = [str(actor_kind)]
if actor_kind == "model_agent":
for field in contract["model_agent_fields"]:
text = value.get(field)
if not isinstance(text, str) or not text.strip():
errors.append(f"{key}:missing_field:{field}")
elif _DECLARED_NAME_FORBIDDEN.search(text):
errors.append(f"{key}:not_a_product_family_name:{field}")
else:
declared.append(text.strip())
lines = reviewer_declaration_lines(body)
if len(lines) != 1:
errors.append(f"{key}:body_line_missing_or_ambiguous")
elif _PUBLISHED_LINE_FORBIDDEN.search(lines[0]):
errors.append(f"{key}:body_line_carries_runtime_detail")
elif any(not _published_as_token(token.casefold(), lines[0].casefold())
for token in declared):
return [*errors, f"{key}:body_line_missing_or_ambiguous"]
if _PUBLISHED_LINE_FORBIDDEN.search(lines[0]):
return [*errors, f"{key}:body_line_carries_runtime_detail"]
if actor_kind == "human_operator":
if not _published_as_token("human_operator", lines[0].casefold()):
errors.append(f"{key}:body_line_disagrees_with_declaration")
return errors

published = parse_model_reviewer_line(lines[0])
model, provider = value.get("declared_model"), value.get("declared_provider")
if (published is None or not published.has_model_actor
or isinstance(model, str) and published.model.casefold() != model.strip().casefold()
or isinstance(provider, str) and published.provider.casefold() != provider.strip().casefold()
or published.source is not None and published.source != source
):
errors.append(f"{key}:body_line_disagrees_with_declaration")
if execution_present or runtime_reported or source == "runtime_reported":
expected_effort = value.get("declared_reasoning_effort") or "effort_unavailable"
if (published is None or published.source != source or
runtime_reported and isinstance(expected_effort, str)
and published.reasoning_effort != expected_effort.casefold()
):
errors.append(f"{key}:runtime_basis_not_published")
return errors


Expand Down Expand Up @@ -612,11 +630,15 @@ def check_review_result(
behavior_bearing=applicability.get("behavior_bearing_change") is True)
errors.extend(f"review_body:{reason}" for reason in body["invalid_reasons"])
body_text = str(result.get("review_body") or "")
errors.extend(_reviewer_errors(result.get("reviewer"), body_text))
from ...control_plane.effect_runtime import effect_runtime_result

execution = packet.get("reviewer_execution")
declaration = result.get("reviewer")
observed = current_execution or execution
runtime_reported = isinstance(observed, Mapping) and observed.get("status") == "runtime_reported"
errors.extend(_reviewer_errors(
declaration, body_text, execution_present=execution is not None, runtime_reported=runtime_reported,
))
if execution is not None or current_execution is not None or (
isinstance(declaration, Mapping) and declaration.get("declaration_source") == "runtime_reported"
):
Expand All @@ -625,20 +647,6 @@ def check_review_result(
params["current"] = current_execution
matched = effect_runtime_result("runtime.execution_identity.match", params)
errors.extend(f"reviewer:{reason}" for reason in matched["errors"])
if isinstance(declaration, Mapping) and declaration.get("actor_kind") == "model_agent" and (
execution is not None or declaration.get("declaration_source") == "runtime_reported"
or current_execution is not None and current_execution.get("status") == "runtime_reported"
):
lines = reviewer_declaration_lines(body_text)
public_fields = [declaration.get("declaration_source")]
observed = current_execution or execution
if isinstance(observed, Mapping) and observed.get("status") == "runtime_reported":
public_fields.append(declaration.get("declared_reasoning_effort") or "effort_unavailable")
if len(lines) == 1 and any(
isinstance(value, str) and not _published_as_token(value.casefold(), lines[0].casefold())
for value in public_fields
):
errors.append("reviewer:runtime_basis_not_published")
problem_context = evidence.get("problem_context")
impact = problem_context.get("outcome_impact") if isinstance(problem_context, Mapping) else None
experience = impact.get("user_experience") if isinstance(impact, Mapping) else None
Expand Down
Loading
Loading