Skip to content

Fix duplicate references request on hosted rescans (#1606, #1607) - #1695

Merged
Mikola Lysenko (mikolalysenko) merged 3 commits into
mainfrom
agent/fix-rescan-duplicate-references-fetch
Oct 23, 2026
Merged

Mikola Lysenko (mikolalysenko) merged 3 commits into
mainfrom
agent/fix-rescan-duplicate-references-fetch

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

LLM Description written by Claude Code:claude-opus-5-5

Refs #1606
Refs #1607

Hosted rescans no longer send the same reference request twice. This is one slice of #1606/#1607, so both use Refs and stay open for the rest (see Not fixed here).

Root cause

A hosted scan asked the patch API's reference endpoint about the same patch uuids twice:

  1. The withdrawn-pin sweep (Restore withdrawn hosted pins upstream during scan #1592, scan/withdrawn.rs) asks about every uuid the lockfiles pin.
  2. The redirect (run_redirect_selected_untracked in scan/hosted.rs) then asks about every selected uuid.

On a rescan, the pins and the selection are the same patches, so the second request repeated the first. That cost one extra network round trip on every hosted rescan, in every ecosystem, including the bun and yarn-berry rescan scenarios in these issues. #1693 found this while profiling them.

Changes

  • The sweep keeps the answers it fetched as KnownReferences. It keeps only settled answers (granted, reused, withdrawn, not_found, forbidden), and only for uuids it actually asked about.
  • Scan hands those answers to the redirect through rollout::Gate::with_references, in both the --json arm and the human arm. The redirect asks only about the selected uuids that have no known answer. When the sweep covered all of them, it sends no request.
  • Some uuids are asked about again:
    • pending_build or an unknown status, because the answer may change while the run goes on (a build that finishes mid-run still gets pinned)
    • a uuid the response left out (a 404 chunk answers nothing)
  • get --mode hosted runs no sweep and passes no gate, so it is unchanged.

Test evidence

Red → green: cargo test -p socket-patch-cli --all-features --test in_process_scan_withdrawn

  • a_json_rescan_asks_about_a_pinned_uuid_once and a_human_rescan_asks_about_a_pinned_uuid_once failed on main with two identical requests ([[UUID], [UUID]] vs [[UUID]]). Both pass with the fix.
  • New behavior-pinning tests, which pass:
    • the_redirect_asks_only_about_uuids_the_sweep_did_not: a replacement uuid is still looked up, and only it
    • a_pending_build_answer_is_asked_again
    • unit test withdrawn::tests::only_settled_answers_are_known
  • 13/13 in the suite pass.

Local checks on 4fd3d54:

  • cargo fmt --all -- --check: clean
  • cargo clippy --workspace --all-features -- -D warnings: clean
  • cargo test --workspace --all-features --no-fail-fast: 14,867 passed, 11 failed. All 11 are environmental, and none touch the changed code:
    • 9 assert that a write to a 0o555 directory fails, which can't happen in the sandbox because it runs as root. They are the same set Speed up bun and yarn-berry rescans (#1606, #1607) #1693 lists: copy_tree, vlt_heal, pypi_poetry, pypi_requirements, in_process_redirect, repair.
    • 2 are in e2e_bun_lockb: binary_shared_bundled_record_hosted_pin_is_managed (the sandbox proxy refuses the real patch host) and workspace_text_migration_heals_on_rerun, which fails the same way with main's sources.
  • I did not run a bench: the change removes one request per hosted rescan, and the tests count requests exactly.

Per-issue checklist

Not fixed here (why the issues stay open)

/code-review high

The review reported 8 findings. I fixed 6:

  • stale pending_build answers
  • treating a 404 chunk as an answer
  • a doc that contradicted itself
  • clones in answers_for, now a consuming into_answers
  • references taken out of the sweep two different ways, now done once
  • a duplicated test helper

I skipped 2:

  • Capped-rollout predicate. On a failed lookup it still checks every selected row. This is the same behavior as before. Narrowing it to the fetched rows would drop the candidates for rows that did get answers.
  • Move the cache into ApiClient. The only duplicate is scan's sweep→redirect handoff, and get runs no sweep. A client-wide memo would also have to handle when an answer goes stale, for every caller.

Generated by Claude Code

Assisted-by: Claude Code:claude-opus-5-5
A hosted scan asked the patch API's reference endpoint about the same
patches twice: once in the withdrawn-pin sweep, and again in the
redirect. On a rescan those are the same uuids, so every hosted rescan
paid for one extra network round trip.

The sweep now keeps the answers it fetched and hands them to the
redirect through the rollout gate. The redirect asks only about the
selected uuids the sweep did not, and sends no request at all when
the sweep covered them. A uuid the sweep asked about but the endpoint
left out keeps reading as not found, as a second request would say.

Refs #1606, #1607

Assisted-by: Claude Code:claude-opus-5-5
Review fixes for the reused reference answers. The redirect now reuses
only settled answers (granted, reused, withdrawn, not_found, forbidden)
for uuids the sweep actually asked about. A pending build, an unknown
status, or a uuid the response left out (a 404 chunk) is asked again,
so a patch that finishes building mid-run is still pinned.

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

BugBot review


Generated by Claude Code

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 4fd3d54. Configure here.

Merged via the queue into main with commit 2f7c3f9 Oct 23, 2026
86 checks passed
@mikolalysenko
Mikola Lysenko (mikolalysenko) deleted the agent/fix-rescan-duplicate-references-fetch branch October 23, 2026 17:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants