Skip to content

feat: port the colby v1.6.2 graph semantics, framework resolvers and synthesis - #322

Draft
sunerpy wants to merge 204 commits into
mainfrom
feat/v0.55-upstream-v1.6.2
Draft

sunerpy wants to merge 204 commits into
mainfrom
feat/v0.55-upstream-v1.6.2

Conversation

@sunerpy

@sunerpy sunerpy commented Oct 10, 2026

Copy link
Copy Markdown
Owner

Summary

Draft: the codegraph-rs v0.55.0 program. One pull request ports the colby
range f4ddf508..b635dd46 (v1.6.1 to main of 2026-10-07; v1.6.2 is its first
180 commits), fixes the Rust-only defects that audit found, and clears every
carry-forward DEFER. The row-by-row ledger is
docs/upstream-sync/V1_6_2_AUDIT_2026-10-10.md. The pull request is squashed
into one commit and released by release-please as v0.55.0.

The branch lands in waves, and CI must pass at the end of each wave:

  • W0 (this push): the audit ledger, golden re-extraction in the test suite
    (golden_reextract, scripts/regen-goldens.sh), a real-pipeline probe
    harness, a cross-language sync-versus-fresh matrix, 13 new golden corpora
    (C#, Java, Swift, Objective-C, PHP, Vue, Svelte, Python bases, CommonJS,
    routers, servers, mobile, synthesis), bench --ab and bench --graph-diff,
    and bounded steps in the viewer live-events test.
  • W1-00 (this push): schema 9 (synthesis_inputs,
    idx_edges_synthesis_site), and the extraction version moves once, 21 → 22.
    A v0.54.0 index reads as outdated and is rebuilt once by codegraph sync,
    a daemon's catch-up or the first MCP access. upgrade_from_v054 covers both
    paths with a real v0.54.0 index.
  • W1 to W10 (later pushes): Rust-only defects, extraction parity, resolution
    seams and per-language families, sync convergence, MCP/CLI/viewer, framework
    resolvers and synthesis, VB.NET (trial), codegraph context, Lua loaders,
    performance strategies, docs.

Owner decisions of 2026-10-10: squash merge; COBOL OWNER-EXCLUDED; VB.NET
through a revertible trial; refusing a newer schema split into a follow-up
pull request.

Verification (this push)

  • cargo test -p codegraph-rs --locked --test golden_reextract --test sync_convergence --test probes --test upgrade_from_v054
    and cargo test -p codegraph-bench --locked --test equivalence: pass
    (33 corpora re-extract byte-identically; the matrix runs 49 cases and keeps 7
    known gaps ignored until W5).
  • cargo test --workspace --locked --no-fail-fast: pass.
  • The pre-push hook (make pre-ci) passed on this head.

equivalence.rs only proved that each committed colby.db agrees with its own
JSON; nothing in the suite ran the extractor, so a change that moved
extraction output passed CI until someone ran the out-of-repo regeneration
helper by hand.

- tests/golden_reextract.rs indexes every re-indexable corpus from
  crates/codegraph-bench/fixtures/ with the binary under test, dumps it
  through the same canonicalizer as `bench --gen-golden`, and compares the
  five artifacts byte for byte with reference/golden/. A failure names the
  changed artifacts, the canonical row diff and the regeneration command.
  A guard test keeps the corpus list in step with reference/golden/.
- scripts/regen-goldens.sh runs the shared recipe: --check (writes
  nothing), --write <corpus...> (only the named corpora) and
  --mini-transplant (mini's schema rebuild). Scratch directories come from
  mktemp and are removed on exit.
- equivalence.rs generates each corpus's two byte-drift tests from one
  macro, keeping the existing test names, and fails when a committed corpus
  has no test pair.
- docs/equivalence.md describes the recipe once, adds the regeneration log
  the v0.55.0 program records changes in, and each corpus section now says
  what it guards plus the one regeneration command.

Every corpus re-extracts byte-identically on this tree; a tampered
go/nodes.json makes the gate fail with a Tier-1 node diff.
…s and CommonJS corpora

The goldens had no corpus for C#, Java, Swift, Objective-C, PHP, Vue or
Svelte, and the Python corpora had no class with bases, so whole families of
extraction never reached the equivalence oracle.

Each new corpus is a small source tree under crates/codegraph-bench/fixtures/
with its golden under reference/golden/, generated by
`scripts/regen-goldens.sh --write`. They capture today's output on purpose,
misses included, so the corrections that follow show up as reviewable diffs
attributed to one change each. Both golden suites now cover them, and
docs/equivalence.md says what each one exercises and logs the addition.
Four multi-project corpora give the framework resolvers and synthesis
passes a baseline before they land: routers (React Router, Next.js, Expo
Router, Vue Router, Nuxt, SvelteKit, Angular, TanStack), servers (Express,
NestJS, Flask, Django, FastAPI, Spring, Laravel, Rails, ASP.NET, Vapor,
axum, actix-web, Gin, Play, Drupal), mobile (React Native and Expo native
modules, a Swift/Objective-C bridge) and synthesis (emitters, handler
tables, re-renders, stores, Go embedding, C function pointers, Kotlin
expect/actual, MyBatis, Celery, Spring events).

Each project carries the manifest detection reads, and projects share
symbol names on purpose, so the cross-project misses in today's output (an
Angular @component bound to a React file's Component) are in the golden and
their later removal shows up as a diff.
…n version to 22

Schema 9 adds the `synthesis_inputs` table and the partial
`idx_edges_synthesis_site` index the synthesis stage uses to decide when a
sync must refresh synthesized edges. The extraction version moves once,
from 21 to 22, ahead of every graph-semantic change in this release, so an
index written by v0.54.0 reads as outdated and is rebuilt exactly once.

Every golden corpus regenerates only `schema.sql` and `colby.db`; the
canonical JSON is unchanged (golden_reextract proves it), and mini is
transplanted with `scripts/regen-goldens.sh --mini-transplant`.

`upgrade_from_v054` plants a real v0.54.0 index (fixture built by the
released binary, provenance in its README) and upgrades it through both
`codegraph sync` and the daemon catch-up sync, asserting schema 9, the
extraction stamp 22 and a canonical graph equal to a fresh init.
Add the `probes` test binary (crates/codegraph-cli/tests/probes). A probe
writes a small tree into a temporary directory, runs the built
`codegraph init` with CODEGRAPH_NO_DAEMON=1 and CODEGRAPH_NO_WATCH=1 so
scanning, extraction, framework detection and resolution run as they do
for a user, and reads the published index back through an immutable=1
SQLite connection, which leaves no sidecar for a later sync to refuse.

`support.rs` provides the Project builder, edit/sync helpers that stamp
explicit mtimes instead of sleeping, row queries over nodes, edges,
unresolved references and files, a selector notation shared with the
failure output, and the assert_edge!, assert_no_edge! and
assert_unresolved! macros, whose failures list the project's whole
non-contains edge list. Smoke probes pin a TypeScript import call, an
unresolved external import, a Python method call and a sync re-read.
Add the `sync_convergence` test binary. Each case indexes a `before`
tree with the built `codegraph init`, applies one edit (add, modify,
delete or rename files), and syncs it through two drivers: the CLI
`codegraph sync`, and the watcher path, `sync_changed_paths` with
exactly the changed paths (or the full reconcile the real watcher runs
for a control file). A fresh `init` of the final tree in a separate
directory is the oracle; both drivers must match it under
`canonicalize_db` + `diff_canonical`, by full natural keys. Cases name
the edges their before and fresh indexes must hold, the on-disk trees
must equal the oracle's, and failures print a readable row diff.

The cases cover every language with an extractor and every change kind:
an added import target, a body-only edit (#2288 same-named methods and
Java overloads), a rename, a deleted module, a heritage edit, a
same-named competitor, control files (codegraph.json, config.toml,
.gitignore, tsconfig.json, package.json) and a C header that turns into
C++. Gates fail when a language with an extractor or a change kind has
no running case.

Known gaps are ignored with their reason and fail under --ignored:
W5-01 #2392, W5-02 #2403 and W5-03 #2450, plus four unscheduled gaps
this matrix found: an Erlang remote call and a MyBatis cross-mapper
include are not retried when their target appears, and a tsconfig.json
`paths` or package.json framework change does not re-resolve unchanged
files.

The probe `file:` selector now matches a file node by path, since the
embedded extractors key theirs by a hash.
The live-events test once held a Windows CI job for 40 minutes: the
sync by another writer and the final search had no time bound, and the
#[tokio::test] runtime's drop waits forever for a stuck spawn_blocking
task, which the viewer uses for every request.

Each test now runs on its own runtime through `run`, which bounds the
test body, propagates a panic from it, and shuts the runtime down on a
helper thread with a limit of its own. Every step that can block has a
named limit: indexing the fixture, starting the viewer, connecting to
/api/events, each awaited event, each request, the sync by another
writer, closing the hub, dropping the last client and the teardown.
Blocking steps run on the blocking pool so a stuck one never stalls the
runtime. A hang fails with the step's name (or, for the backstop, the
last step started). No assertion changed.
`bench --ab` times a cold `init` of a baseline and a candidate codegraph
binary on pinned corpora. Every run copies the corpus into a fresh project
without `.git` or an index directory, sets RAYON_NUM_THREADS (the CLI has no
thread flag), CODEGRAPH_NO_DAEMON and CODEGRAPH_NO_WATCH, removes
CODEGRAPH_DIR, and records the wall time, the wait4 peak RSS and the
canonical fingerprint of the database. Runs alternate between the binaries
and the first run of each cell is a discarded warm-up. The harness stops
without a report when a run fails, is killed or times out, leaves no
database or one that fails PRAGMA quick_check or canonicalization, or when
one binary produces two different graphs at any thread count. The JSON
report records the CPU, kernel, rustc, both binaries' SHA-256, every run,
median and MAD, candidate/baseline ratios and the regression-budget verdict;
--report-md renders it as a table.

`bench --graph-diff` compares two canonical graphs, each a database or a
golden directory, and prints the removed and added rows grouped by surface,
language, kind and resolvedBy. It exits 0 when the graphs are identical, 1
when they differ and 2 when an input cannot be loaded. A database is read
from a private copy, so the input never gains SQLite sidecars.

The corpus registry gains express-medium and the release-pinned leveldb,
gson, Newtonsoft.Json, Alamofire, redis and kong corpora, a tag field, and
--corpora-root for --list-corpora and --fetch-corpora. sha2, already in the
lock graph through codegraph-core, hashes the binaries and graphs.
docs/benchmark.md documents both commands, the report fields, the regression
budget and every pinned corpus.
V1_6_2_AUDIT_2026-10-10.md records the 268 upstream commits from colby
v1.6.1 to main of 2026-10-07 row by row: disposition, Rust target, the
v0.55.0 program commit that carries each row and its status, together with
the Rust-only defects and pre-existing extraction gaps the audit found and
the owner decisions of 2026-10-10. COBOL is OWNER-EXCLUDED, and refusing a
newer schema is recorded as PLANNED for its own follow-up pull request.

The ledger's discovery boundary moves to b635dd46, and docs-check follows
it; the tracked release stays v1.6.1 until the ported rows ship.
…egistry

`index --force` lists every registered stdio MCP server on stderr before
it rebuilds. The pre-heal ordering test drains the child's pipes only after
both lease barriers, so on a machine with many registered servers, and
pipes capped at two pages once the user's pipe budget is spent, the list
filled stderr and parked the command before its second lease. The test now
gives the command an empty registry of its own.
The single extraction-version move to 22 must be exactly one step past the version the preceding release wrote, which is 21 for v0.54.0.
@codecov

codecov Bot commented Oct 10, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.15059% with 188 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
crates/codegraph-cli/src/main.rs 84.48% 74 Missing ⚠️
...degraph-extract/src/embedded/vue_template_calls.rs 90.33% 66 Missing ⚠️
.../codegraph-extract/src/embedded/vue_options_api.rs 92.47% 21 Missing ⚠️
crates/codegraph-extract/src/embedded/vue.rs 97.02% 5 Missing ⚠️
crates/codegraph-core/src/file_class.rs 98.11% 3 Missing ⚠️
crates/codegraph-daemon/src/lease_keeper.rs 98.59% 3 Missing ⚠️
crates/codegraph-daemon/src/project_service.rs 90.62% 3 Missing ⚠️
crates/codegraph-daemon/src/session.rs 83.33% 3 Missing ⚠️
crates/codegraph-extract/src/commonjs_requires.rs 97.45% 3 Missing ⚠️
crates/codegraph-core/src/generated_header.rs 98.16% 2 Missing ⚠️
... and 4 more

❌ Your patch check has failed because the patch coverage (94.15%) is below the target coverage (95.00%). You can increase the patch coverage or adjust the target coverage.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #322      +/-   ##
==========================================
+ Coverage   95.06%   95.60%   +0.54%     
==========================================
  Files         198      280      +82     
  Lines      109293   166177   +56884     
==========================================
+ Hits       103895   158868   +54973     
- Misses       5398     7309    +1911     
Files with missing lines Coverage Δ
crates/codegraph-core/src/types.rs 100.00% <ø> (ø)
crates/codegraph-daemon/src/control.rs 97.04% <100.00%> (+0.37%) ⬆️
crates/codegraph-daemon/src/install.rs 100.00% <100.00%> (ø)
crates/codegraph-daemon/src/lib.rs 93.08% <100.00%> (+0.31%) ⬆️
crates/codegraph-daemon/src/spawn.rs 96.72% <ø> (ø)
crates/codegraph-extract/src/embedded/astro.rs 98.83% <100.00%> (+1.00%) ⬆️
crates/codegraph-extract/src/embedded/mod.rs 100.00% <100.00%> (ø)
crates/codegraph-extract/src/embedded/shopify.rs 100.00% <100.00%> (ø)
crates/codegraph-extract/src/embedded/svelte.rs 93.68% <100.00%> (+0.66%) ⬆️
crates/codegraph-extract/src/ext_config.rs 98.29% <100.00%> (+0.27%) ⬆️
... and 72 more

... and 88 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

The upgrade test built its immutable URI from the raw path, which on Windows keeps the \\?\ verbatim prefix and backslashes that SQLite cannot open. It now uses the same file: URI form as the probe harness.
dump_golden wrote its log beside the output directory, so --write left reference/golden/<corpus>.gen.log next to the named corpus and removed that path on exit. Both logs now live in scratch directories of their own.
…s W0-06 requires

The corpora lacked three shapes the plan asks them to pin at v0.54.0
semantics: a PHP `static::` call, a CommonJS `require().member`, and a
Flutter app whose `pubspec.yaml` triggers detection. Each lives in a new
file, so the regenerated goldens only add rows: `bench --graph-diff`
against the previous goldens reports 0 removed rows for all three.
The UI-family, extraction-gap, carry-forward, planning-finding and Rust-only-defect tables now carry the disposition and Rust target columns the ledger format requires, and the not-needed section is a row ledger.
--mini-transplant still logged to a path beside its scratch directory and removed it on exit; the log now lives in a scratch directory of its own, like the other two.
…ture

The script-tests loop ignored a failing script, so ci-gate.test.sh had silently stopped at its stale needs anchor since the UI job joined CI Success (#298): mutations C to K never ran. The anchor now carries the ui job and the loop stops on the first failing script; all five harnesses pass.
…tarts

A cold `serve --mcp` answers its handshake direct and spawns the shared
daemon fire-and-forget, but never connected to it. With no client the
daemon idle-exited after CODEGRAPH_DAEMON_IDLE_TIMEOUT_MS (300 s by
default) and the session's index silently stopped syncing.

The session now keeps a passive lease on that daemon from a background
thread, the same client connection an explicit projectPath service holds,
so the handshake still never waits on the daemon. When the lease stops
being live (the daemon died, was replaced, or the index was re-created) it
starts or re-attaches a daemon with backoff: CODEGRAPH_DAEMON_RETRY_MS
(default 5000, 0 disables) doubling up to CODEGRAPH_DAEMON_RETRY_MAX_MS
(default 300000), as upstream #2293 does for its degraded proxy. It never
starts a daemon for a project whose index is gone.
A settled burst used to sync exactly the paths its events named, however
many, and the notify callback forwarded only the event kind and paths, so a
backend's sign of dropped events was lost and the index could stay stale
after a mass rename or a branch switch (#269).

The watcher now runs one full project reconcile instead when the burst
named more than 500 paths (upstream #1397's SCOPED_SYNC_MAX_PENDING), when
an event carries notify's Rescan flag (an inotify queue overflow, an
FSEvents drop) or names no path (upstream #2025's filename == null), and on
Windows, whose backend can lose events silently, when an idle drift
sentinel (CODEGRAPH_WATCH_SENTINEL_MS, default 60000, 0 disables) finds
changes no event reported; it then also registers the root watch again.
Until a lost-event reconcile commits, the watcher reports RECOVERING with
the cause, and the RECOVERING banner now names its reason instead of
assuming lock contention.

As promised on #269, the MCP instructions, the codegraph_status
description and the skill now say what happens after a large burst and
that renames never need a rebuild; the initialize golden follows the
instructions.
A daemon keeps running the code it started with. After an upgrade the
project's daemon could be an older build than the session that found it:
the proxy saw a version mismatch, the session fell back to serving reads,
and the old daemon kept the writer slot for good (upstream #2343, #2346).

A session that meets a daemon of another version now reads its version from
the hello and replaces it only when it is an older plain X.Y.Z release: it
asks it to drain over the project-bound control channel, so nothing is
signalled by pid, then starts a daemon from its own install and proxies to
it, or attaches through the cold-start lease keeper if the new daemon is
slower than the usual poll. A newer release, a prerelease, an unknown
version, or a daemon that does not acknowledge the drain is never stopped;
the session then serves reads and says so on stderr.

Every daemon also records its executable at start and drains and exits once
the file is replaced or removed (CODEGRAPH_DAEMON_INSTALL_CHECK_MS, default
30000, 0 disables), and a cold-start session whose own install changed stops
starting daemons. A test-hooks feature lets test builds of the daemon claim
another version and leave control frames unanswered.
A content-hash comparison calls a file up to date even when its stored row
holds none of its symbols, so `status` said "Index is up to date" while a
file contributed nothing to the graph (upstream #2336, the status half of
#2346).

`status --json` adds `index.filesNeedingReindex` (rows stored with no nodes
and no recorded reason, which a hash-based sync never revisits) and
`index.filesWithParseErrors` (a recorded parse failure and no symbols from
it); files over the size limit and file-level-only languages are empty on
purpose and count in neither. Human output names up to three files of each
group and no longer calls such an index up to date. `files --json` carries
each file's recorded errors. Every existing field is unchanged.
`serve --mcp` launched in an unindexed directory with `.git` searches
downward for indexed sub-projects. The walk had a depth and candidate cap
but no limit on how much it read: launched in /tmp (~40K entries) it never
answered `initialize` within 30 s.

The scan now stops after 10,000 directory entries (counted, so where it
stops is deterministic) or 500 ms, whichever comes first, and a scan that
stopped early adopts nothing, since it cannot know the child it found is
the only one; the tool guidance and the stderr notice say it stopped. The
MCP servers no longer run the scan in their constructors, before
`initialize` is answered: their first `tools/list` or tool call runs it,
and reports the candidates it finds once.
The daemon runs detached, with no console of its own, so on Windows every
console program it starts without CREATE_NO_WINDOW opens a visible console
that flashes once per call. The git pending-status path already set the
flag; `git_command` (repository and hook discovery) and the viewer's
`git config user.name` did not (upstream #2280).

`codegraph_watch::hide_console_window` now sets the flag in one place, and
every git call, and the viewer's browser opener, goes through it. Because
std cannot report a command's creation flags back, a static source guard
(like upstream's) checks every `Command::new(` in the crates the daemon runs:
each must hide its console or say in a `console-window-guard:` comment why
it needs not.
KNOWN_DIFFS listed C++ receiver-type inference, Java/Kotlin field-receiver
inference, the batched and conformance passes and the per-language import
refinements as deferred, though all of them are ported. equivalence.md said
`ns::compute()` resolves through the qualified-name matcher, while the cpp
golden records it as an exact-name match, and called the missing Rust
struct-literal instantiation upstream's own shape, which it is not.

A dated UPSTREAM.md entry records the corrections, the KEEP-RUST status of
#226's stricter receiver-word gate, and the #1728 disposition; the entries
it corrects stay as written.
- Read <root>/.git/info/exclude ahead of the root .gitignore in
  RootGitignore, so the scan and the watch policy prune what git's
  exclude-standard prunes (upstream #1728). A .gitignore rule wins a
  conflict, as in git. Only a real .git directory at the project root is
  read, through the bounded stat-first reader; a .git file, a link, an
  ancestor repository and core.excludesFile are never followed.
- The watcher treats the exclude file as a control file: per-directory
  backends watch .git/info, and an edit reloads the scope and runs one
  full reconcile.
- The pending-status git record's scope fingerprint includes the exclude
  file, so a record made before an edit is not trusted after it.
- Sync matrix: a .git/info/exclude control-file case. The root .gitignore
  case now hides legacy/ rather than vendor/, which the default ignores
  already drop, so it moves an edge instead of passing vacuously.
- An import node is a statement, not a definition (upstream #965,
  0a91d0f5). Exact name matching no longer offers one, the file's own or
  another file's, and one named like a definition no longer makes that
  definition look ambiguous. The fuzzy fallback already considers
  callables only. The qualified-name matcher is unchanged here: its
  imports-to-own-statement edges belong to the #2392 target gate (W3-01).
- Goldens: csharp, erlang, java, mobile, php, python, python_bases,
  routers, ruby, rust, servers, swift, synthesis, typescript, vue lose
  every imports edge exact matching bound to an import node, and the refs
  stay unresolved. PHP `use App\Field;` now binds namespace App\Field;
  Ruby `require_relative "logger"` falls through to the fuzzy match
  class Logger. Logged in docs/equivalence.md.
- Probe pr0965_vue_import_never_binds_a_svelte_import_node, unit tests for
  the exact and fuzzy paths, and a PHP AddCompetitor sync-matrix case.
- tier2_false_edges: the phantom node:path import now binds nothing and
  stays parked as unresolved (renamed to
  phantom_import_binds_nothing_and_stays_unresolved); the Rust supertype
  fixture keeps only its in-repo `use` edge, its `crate`/`std` self-edges
  gone.
- The out-of-repo import gate (imports and supertypes) no longer counts a
  JS specifier as local because it starts with a path alias's prefix: a
  catch-all "*" alias has an empty prefix that every package starts with
  (upstream #2437, 2c807f5c). An alias now makes the import local only
  when resolve_import_path maps it to a file the index holds, so an alias
  landing in node_modules or on a directory does not either. Resolving
  an import still honors the prefix, so "*": ["src/*"] aliases resolve.
- ResolutionContext gains is_indexed_file, answered from the store or
  the snapshot's known files, never from the filesystem.
- Probes pr2437_catch_all_alias_keeps_package_imports_external and
  pr2437_alias_into_node_modules_keeps_the_import_external, unit tests,
  and a Tsx AddCompetitor sync-matrix case; the matrix gains
  no_edge_after for an edge a fresh index must not hold.
- No golden changes (scripts/regen-goldens.sh --check: 32 identical).
- detect_language_for_source (and detect_language_of for a bounded read)
  give a file's language as extraction parses it: a .h header holding C++
  or Objective-C is that language. index, sync, migration and framework
  extraction record and use it, so a header's file row agrees with its
  nodes (G15). Framework extraction reads a C header whenever a resolver
  applies to C, C++ or Objective-C.
- Goldens: cpp plain_derived.h and ue_actor.h become cpp, objc
  Sources/Base.h becomes objc; files.json only. Logged in
  docs/equivalence.md.
- c_family_correctness pins the sniffed language for headers and the
  agreement of every node; the c_header_flips_to_cpp sync-matrix case
  now requires the fresh index to record shape.h as cpp, via the new
  file_language_after expectation.
resolve_lua_require scanned the whole file list for every require, and for
every file built a `/suffix` string to compare. lua_scope::lua_require_matches
now keeps, per resolution pass, each suffix's matching files in the file
list's order, so every require of a module reads the same list once; the
choice by the longest shared prefix with the requiring file is unchanged.
A context test compares the memo with the scan for each suffix, twice.
retryable_failed_imports read every parked import and path reference as a
whole row before choosing the few a sync retries. It now reads only each
candidate's id, name and kind, borrowed from the row, buckets them under the
same three keys and per-key ceiling, and then reads the chosen rows whole by
id, in id order: the same rows in the same order.
After a sync, cmd_sync opened the index read-only twice, once for the
file_complete diagnostic events and once for the extraction warnings, and
each read-only open copies the whole database into memory. SyncOutcome now
carries each changed path's file row as the sync left it (None for a removed
file), read by the writer just before it returns, in an incremental sync and
in a migration rebuild alike, and cmd_sync reports from those rows. The
events' fields and order and the warnings are unchanged; an outcome without
rows still falls back to opening the index.
…without re-extracting them

A one-line edit to a C# file that many files reference re-extracted all of
them in memory, only to recover the references at the sites that pointed
into it. Schema 10 keeps those references instead:

- `resolved_refs` has the columns of `unresolved_refs`, the id it had there,
  a cascade on the source node, and indexes on the site and the file path.
  Migration 10 creates it empty.
- Resolution moves each row it takes out of `unresolved_refs` there, in the
  same transaction (an id listed twice moves once); re-extraction, site and
  file replacements, and node deletion clear both tables alike. So a file's
  rows in the two tables are every reference of its last extraction.
- `resolved_refs_version` (`1:` and a digest of the extension overrides,
  max_file_size, the Godot DSL config, the extraction version and the
  detected frameworks) vouches for that. Only a full index writes it; a sync
  reads it before its first change, withdraws it at that change and writes it
  back only when it held at the start and the digest did not move, so a
  migrated index, a sync that failed partway and changed inputs fall back to
  extraction until the next full index.
- While it holds, a sync re-resolves an unchanged dependent from its stored
  rows, read only at the refreshed sites unless the file goes whole; a site
  with no stored row sends the file whole, as before. `project.godot`, whose
  extraction reads every `.tscn` and `.gd.uid`, is always extracted.
- A Current writer open runs pending migrations under its exclusive lease,
  so a schema-9 index of the current extraction version gains the table on
  its first incremental sync.

Every golden corpus regenerates `schema.sql` and `colby.db` (and
`reference/golden/colby.schema.sql`); the canonical JSON is unchanged. A
schema-9 fixture built by a release build of 3878d52 drives the upgrade
tests through `codegraph sync` and the watcher's sync.
… read-only

A read-only open without sidecars read the whole database into a Vec and
then copied it into the sqlite3_malloc64 buffer it hands to
sqlite3_deserialize, two copies of the index per open. It now allocates the
SQLite buffer at the file's length and reads the file straight into it; the
buffer is freed if the read fails before SQLite owns it. A file that grows
during the read fails it ("database changed while it was read") and one that
shrinks fails read_exact, where the old read could return a torn image. The
FREEONCLOSE | READONLY deserialization and the WAL header normalization are
unchanged; mmap is not used, since a checkpoint or auto-vacuum can shorten the
file under a mapping.
oxfmt wraps the provenance command as it wraps the v0.54.0 fixture's README.
Moving each resolved reference into resolved_refs with its own INSERT ...
SELECT and DELETE made a full index measurably slower than before schema 10
(newtonsoft: 0.15 -> 0.43 s CPU in delete_resolved_unresolved_refs). The
ids now move a chunk of at most 500 at a time, one INSERT OR IGNORE ...
SELECT ... WHERE id IN (...) and one DELETE ... WHERE id IN (...) per chunk,
in the same transaction: the same rows with the same ids, and an id listed
twice, in one chunk or two, still moves once because its first move deleted
the row. A store test moves 3,000 rows across chunks with repeats and
compares both tables with the row-by-row result.
The control-input digest stood in a file's size and modification time for
its text once it passed 64 MiB, so a tsconfig.json, package.json, C# project
file or Cargo.toml that large, replaced with both kept, went unnoticed while
its reader, which has no size limit, read the new text: a fresh index used
the new aliases and the sync kept the old edges. The plan and the docs
promise every input is read whole with no size or time shortcut; the digest
now always is. A watch test replaces a 64 MiB tsconfig.json keeping its size
and mtime and checks a full sync and a watcher's sync against a fresh index.
e3839db made the control-input digest read every file whole; the W5 goal
review asked for the convergence matrix to hold that, as every sync change
here adds a case. The harness gains Op::Replace, a rewrite that keeps the
file's modification time, and the case replaces a 64 MiB + 1 byte
tsconfig.json with one of the same length aliasing another folder: both
drivers must match a fresh index. With the size and mtime shortcut restored,
both drivers diverge.
Claude Code hands a subagent's report back to the parent session as a
UserPromptSubmit prompt wrapped in <agent-message from="...">...</agent-message>
(upstream #2184, #2281). The hook took it for a user question and ran a full
explore over the whole report. A prompt that is entirely that envelope, its
opening tag bare or followed by whitespace and attributes, now exits before
any index work, by the same anchored rule as the task-notification envelope;
a question that mentions the tag, or follows the envelope, still runs. The
CLI and MCP docs now describe both envelopes as the whole prompt.
On Windows a port another program holds exclusively (SO_EXCLUSIVEADDRUSE),
or one inside a reserved range, fails the bind with a permission error, not
as in use, so `codegraph ui` stopped at the first such port and blamed the
privileged-port rule for ports like 4747 (upstream #2299, #2353). The
fallback walk now moves past a port in use on every platform and past a
refused one on Windows at any port and elsewhere from 1024 up; below 1024 off
Windows a refusal stays the privileged-port rule and the walk stops with that
message. The messages say what is true where they are read: Windows names
the exclusive hold or reserved range and the netsh command that lists the
ranges, POSIX says the system refused permission above 1024, and a walk that
ran out says "in use or reserved" or "in use or not allowed" when a port was
refused. Unit tests drive the walk through a fake binder for both platforms.
…d routes

A directed Flow's from and to went through the free-text token filter, which
drops a two-letter name, a Ruby predicate or bang method and an Objective-C
selector, so the other end was then reported as naming nothing (upstream
#2262). Directed mode now takes the two names as written.

The named walk kept whichever route reached a node first, so of two routes of
one length it followed the callee the index listed first even when the query
named the other (upstream #2453, the half the viewer's Flow uses; explore's
own Flow is W7-06). A node reached again from the same depth now takes the
route that ends on fewer unnamed hops, then the one through more named
symbols; of two equally deep named ends a seed keeps the one whose route
passes more named symbols. The expansion order and the visit cap are
unchanged. Upstream met it through an interface method's implementations,
whose dispatch edges come with the synthesis stage (W7), so the tests use
plain call routes, each asked naming one route and then the other.
Port upstream #2102 and #2270.

A folder with exactly one subfolder and no file of its own, read from
every indexed file with tests included, no longer counts as a level of
the Map: a Maven or Gradle project's src/main/java/com/acme/app/ chain
is one level, so the map opens on the packages instead of one box. The
module's wire label writes such a chain of three or more folders
first/…/last; the canvas box shows and is sized for the label, while the
id, tooltip and side panel keep the whole path. A renamed lone
(root files) bucket keeps its folder's elided label.

The default root now counts files loose in the repository root in the
whole, and when that root draws one box at every depth while the whole
repository draws more, the Map opens on the whole repository.

The SVG export writes the label too. Upstream's export still writes the
id cut to the narrower box; docs/ui.md records the difference, and the
non-production folder rule this port has not taken from upstream.
A Map box is border-box, so its 1px border (1.5px when selected) sits
inside the width, but the width allowed only the 44px and 18px insets.
A box whose label decides its width clipped the last character to an
ellipsis: `src/main/…/clinic/(root files)` needed 216px and had 214.7px.
Both lines now get the insets and a selected border.
Port the non-Steps half of upstream #2264. The viewer keys its lists by
id, and a repeated id stops the page drawing.

The routing manifest is a row per (route, edge), so a route bound to two
symbols, or an inline handler whose every call reads as a handler, came
back more than once. The routes list, and the routes among the entry
points, now fold the rows to one per route: the row naming the route's
root, else an `inline handler` row standing for a handler written at the
registration, which opens on the route itself and offers no Flow. The
root is upstream's route-roots order of evidence (a named handler, a
rendered component, the route's own calls), with ties broken by line and
id so the answer does not depend on edge order. Rows are over-fetched
four to a route so folding still fills the page, and each row carries
`inline`.

A symbol's members list each member once and never the symbol itself or
a file, the guard upstream added for components that contained their own
script symbols twice. This index records neither repeat today; the
resolvers here bind each route to one handler and mint no inline ones.
Port upstream #2405. The Dead code list showed two kinds of method a
framework calls.

A method whose decorator comes from outside the index (`@HostListener`,
`@Cron`, `@OnEvent`) has no `decorates` edge and no decorators on the
node, only the resolver's unresolved `decorates` row. That row now counts
as a decorator, the same evidence the ancestor rule reads for an
unresolved `extends` or `implements` row.

A lifecycle hook on a class that skips the `implements` clause (`ngOnInit`
on a `@Component` without `implements OnInit`) had no evidence at all. A
new `hooks` rule drops a TypeScript method when its class carries a
decorator and an interface outside the index carries the name, going by
the classes that write the interface out: every one has the method,
declared or inherited, at least two and more than half declare it, and
more than half of them carry a decorator. TypeScript only, because the
clause is erased there and the framework can only find the hook by name.

The ancestor walk moves out of override_candidates into ancestors_of,
shared by both rules, and the store gains the two unresolved-reference
lookups the rules read.
Upstream d6efd43 (#1306) made one switch honor NO_COLOR and --no-color
and go plain when stdout is piped. This port prints no color of its own
and draws progress only on a terminal stderr, so its piped output is
already plain whatever the environment says. Pin that, case for case
with upstream's test: list commands under no signal, NO_COLOR with
FORCE_COLOR, FORCE_COLOR, CI and CLICOLOR_FORCE; piped index and sync
with no carriage-return rewrites; and NO_COLOR winning over
CLICOLOR_FORCE for clap's help and errors, the one colored surface.

docs/cli.md says so, and that upstream's --color and --no-color are not
flags here.
…erver does

Claude Code 2.1.294 and later speak MCP 2026-07-28, which has no
initialize: a session opens with server/discover and names its protocol
version in every request's _meta, and a 2026-07-28 result carries
resultType "complete". The daemon proxy answered every tools/list itself
as {"tools": ...}, without resultType, and the client refused the tools.
Cold sessions serve direct, so only the second and later sessions of a
project, the ones a running daemon serves, broke.

The proxy now answers tools/list locally only in the initialize
lifecycle: after an initialize, before any server/discover, when the
request names no protocol version in _meta. Every other tools/list is
forwarded and owed in the reply ledger, and the daemon's rmcp handler
answers it as a direct server would.

The local initialize answer also pinned 2024-11-05, where the direct
server echoes a requested version that has an initialize handshake
(2024-11-05 through 2025-11-25) and falls back to 2024-11-05 for
2026-07-28 or an unknown version. It now negotiates by that rule, read
from rmcp's KNOWN_VERSIONS and pinned to rmcp's own negotiation for the
handler by a test.

A CLI test drives a proxied session (a warm serve --mcp while the
project's daemon runs) and a direct one through server/discover and
tools/list, and through initialize at 2025-06-18, 2024-11-05, 2026-07-28
and an unknown version, and compares every reply but its id; it fails on
the previous proxy.
Port the classifier half of upstream 950686de. A folder named
`integration`, `sample(s)`, `example(s)`, `fixture(s)`, `benchmark(s)` or
`demo(s)` anywhere in a path made a file non-production, so
spring-petclinic's `src/main/java/org/springframework/samples/petclinic/`
ranked as test code and the Map drew every one of its packages as a test.
Only the project layout above the first `/src/` counts now; the package
path below it never does.

The narrow test predicate is now the one the resolver's test-suite rule
reads, `codegraph_core::file_class::is_test_path`, instead of an older
copy of it. That copy lacked two things upstream's `isTestPath` has:
test-support folders (`*-test` modules, `testdata`, `testutil(s)`,
`fakes`, `mocks`, `__mocks__`, `stubs`, from 950686de) and an `e2e/` tree
(from b1f40c57).

The classification is read at query time only, by search ranking, dead
code, named-symbol flow and the viewer's test flags; the graph and the
extraction version do not change. The scoring tests now carry upstream's
`is-test-file.test.ts` case for case, and the Map and dead-code fixtures
go back to upstream's petclinic and `demo` package paths.
`serve --mcp` at `$HOME` or the filesystem root runs no daemon, watcher
or catch-up, and serves through the hand-rolled loop that can hand the
session to a daemon once the client's roots name an indexed project.
That loop knew only the `initialize` lifecycle. A 2026-07-28 client's
`server/discover` got "Method not found", its `tools/list` came back
without `resultType: "complete"`, and an `initialize` was answered with
`2024-11-05` whatever version it asked for.

A session that opens with the inline lifecycle, `server/discover` or a
request naming its protocol version in `_meta` before any `initialize`,
now ends the loop unanswered. The rest of it is served by the rmcp
handler the direct server uses, built from the state the loop already
resolved: default project, cwd, root discovery and the
explicit-`projectPath` service broker. It still starts nothing for the
broad root. The lifecycle has no roots, so such a session never adopts
the client's workspace; a call that passes `projectPath` gets that
project's daemon and watcher as before. Each `initialize` reply branch,
plain, with a `roots/list` request, or adopting the client's root, now
negotiates the version the rmcp handler would.

A CLI test drives both lifecycles through a session at a temporary
`$HOME` and a direct one and compares every reply.
A sweep of the pages against what v0.55.0 changed. The environment
variable table lists CODEGRAPH_MCP_TOOL_TIMEOUT_SECS, the per-tool-call
limit an MCP client waits on (60 s, 0 for none), which no page named;
the daemon retry, install check and Windows sentinel variables were
already there. The skill quotes the stale-index banner as the server
prints it, a heading line and one indented path per file, not the
single-line form it had. README and its Chinese mirror agree in
headings, commands and links.
Every PLANNED row of the v1.6.2 audit now records what happened to it.
A row the v0.55.0 pull request carries is LANDED with the commits that
carry it; a gate fix belongs to the row it fixed, and a row naming
upstream PRs cites the commits that port them. A row under W7, W8 or W9
is PLANNED (follow-up PR) since the owner split the program on
2026-10-16 (OD-18), and a row that lands only partly names both halves.
Where the Corrections table moved a row to another commit, its status
follows the move.

The audit also gains G16 and G17, the two C++ extraction gaps found
while porting #2446 and left OWNER-EXCLUDED on 2026-10-12; Owner
decision rows for OD-18 and OD-21 to OD-23; and Corrections rows for
schema 10, the split, and the parts of 950686de, b1f40c57 and the route
roots that moved from W7 into W6.

UPSTREAM.md's Current alignment states the new v0.55.0 scope, and a
2026-10-16 entry records the split with the follow-up pull request's
extraction-version boundary, 22 to 23 before its first graph-changing
commit. The dated entries before it stay as written.
…s fixture

The final OD-4 parity run built upstream b635dd46, indexed the mini
fixture with it and compared the result with the golden through
bench --graph-diff. Nodes, files and unresolved references are
identical, including the this.value reference the Rust regeneration
added, and every edge keeps its endpoints, kind, position, confidence
and resolvedBy. The remaining differences are recorded as KEEP-RUST:
upstream's refName/refKind stamp on resolved edges (kept here as
resolved_refs rows), the unresolved_refs status/name_tail columns and
their partial indexes, the name_segment_vocab table, and the wider
idx_nodes_kind upstream's synthesis refresh uses.

The regeneration log also gains the row the initialize MCP fixture
lacked: the server instructions sentence about lost-event rescans,
the RECOVERING banner and renames.
crates/codegraph-ui/viewer/ is `make ui` output, and `make ui-check`
byte-compares it with a fresh build, so it is never hand-edited. Its
minified script ends line 1 inside a string with a space, as v0.54.0's
bundle did, which made `git diff --check` fail on every bundle rebuild.
The `-whitespace` attribute exempts only that directory; every authored
file keeps the check.
The module comment still said the upgrade leaves a schema-9 index; the
test asserts CURRENT_SCHEMA_VERSION, which is 10 since resolved
references moved into their own table. The comment now says the
current schema.
The full-index connection set a 256 MiB page cache and a 1 GiB mmap
window. The pages a full index writes and reads back then sat in the
page cache (anonymous memory) and again as mapped file pages, so peak
RSS counted the database twice. v0.55's database is larger
(resolved_refs and more rows), so the double count grew with it.

Both bulk-index entry points (Store::set_bulk_index_pragmas and the
rebuild writer's set_bulk_index_pragmas) now set mmap_size to 0 and keep
synchronous=OFF, the page cache, deferred WAL checkpoints and the WAL
valve. Ordinary connections keep their 256 MiB window. PRAGMAs do not
change what is written, so goldens and canonical graphs are unchanged.

Official v0.54.0 musl against the musl build, peak RSS: this repository
at 32/4/1 threads 1.156x/1.361x/1.388x -> 1.035x/1.165x/1.182x,
typescript-large at 4/1 threads 1.212x/1.240x -> 0.979x/0.993x, wall
unchanged.
SnapshotResolutionContext::from_store built its per-file and per-kind
candidate lists by collecting each store query's Vec<Node> by value into
Vec<Arc<Node>>. Rust collects that in place: the list reuses the query's
buffer and keeps a node's worth of room for every 8-byte handle for the
whole resolution pass (17 MiB on this repository, 18 MiB on
Newtonsoft.Json; v0.54.0 has the same).

The lists are now collected from a borrowing iterator, so each allocates
a buffer of handles and the query result is freed at once. Elements and
order are unchanged.

Official v0.54.0 musl against the musl build with the full-index mmap
off, peak RSS on this repository at 4/1 threads: 1.165x/1.182x ->
1.083x/1.080x.
The two depth-first loops of the same-file value references (each
reader's scan and the shadowing scan) created a tree-sitter cursor for
every node they popped. A cursor is a C allocation, and tree-sitter's C
allocations go to the system allocator: on the musl release build that
is musl's malloc, which serializes them on one lock. On redis the scans
added 1.39 million allocations to 1.87 million, the summed walk stage
went from 12.9 s to 35.5 s at 32 threads, and init took 25% longer than
v0.54.0 for 3% more rows.

Each loop now creates one cursor and resets it onto every node through
named_children. Visiting order and edges are unchanged. A test counts
the scan's tree-sitter allocations through a counting allocator in a
child process: 1,935 for 30 readers before, at most two per reader now.
The static musl Linux builds allocate through mimalloc, whose v2 defaults
advise transparent huge pages for the memory it takes from the system.
On a host with THP in `madvise` mode each thread's heap then stays
resident in whole 2 MiB pages: in a redis full index at 32 threads
AnonHugePages was 98 MB of v0.54.0's 277 MB peak, and fd-small's peak at
32 threads was about 280 MB.

The musl-only mimalloc dependency now enables the crate's `no_thp`
feature (libmimalloc-sys defines MI_NO_THP), so mimalloc no longer asks
for huge pages. The process and system THP settings are not touched; on
an `always` host the kernel still decides, as for v0.54.0. glibc, macOS
and Windows builds are unaffected, and the feature does not enter
Cargo.lock.

With this and the three preceding changes, official v0.54.0 musl
against the musl build peaks at 0.40x-1.08x of v0.54.0's RSS on the
nine other registry corpora at 32 and 4 threads (fd-small 32 threads
276 -> 118 MB); wall times stay within budget.
RustSec published three advisories on 2026-10-19 and 2026-10-20 that
the Security Audit job reports for the current lock:

- RUSTSEC-2026-0382, rustls-webpki 0.103.15: certificate verification
  may accept distrusted intermediates (fixed in 0.103.16);
- RUSTSEC-2026-0381 and RUSTSEC-2026-0383, webpki-roots and
  webpki-root-certs 1.0.8: the bundled roots trust distrusted emSign
  intermediates (fixed in 1.0.10).

All three reach the binary only through self_update's reqwest/rustls
stack. The update is lockfile-only: three versions and checksums, no
dependency added or removed. cargo audit with the current advisory
database: the previous lock reports the three vulnerabilities, this
one none.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant