Skip to content

[Pi] ctx_search fails with "column index out of range" on Node 22.16–22.19 / 24.0–24.6 (node:sqlite ?NNN positional binding bug) #906

Closed

Description

@yiyi-wei

What happened?

On Pi with Node 22.19.0, ctx_search intermittently fails with:

column index out of range

(SQLITE_RANGE). It is not a data problem. The cause is a node:sqlite binding bug, nodejs/node#59340. Numbered placeholders (?1, ?2, …) are bound as named parameters, so a statement that uses ?NNN rejects positional arguments. The upstream fix is nodejs/node#59350. It shipped in v22.20.0 (listed in the v22.20.0 release notes, nodejs/node#59973), and the fix PR is referenced in the v24.7.0 release notes.

Minimal repro (no plugin code)

Node v22.19.0:

const { DatabaseSync } = require('node:sqlite');
const db = new DatabaseSync(':memory:');
db.prepare('SELECT ?1 AS v').get('x');       // Error: column index out of range
db.prepare('SELECT ?1 AS v').get({ 1: 'x' }); // { v: 'x' }
db.prepare('SELECT ? AS v').get('x');         // { v: 'x' }
db.prepare('SELECT ?1 AS a, ? AS b').get('x', 'y'); // Error (also fails on v23.11.1; OK on v22.23.5)

Where Magic Context hits it

Since v0.45.0 (4661544), with more sites added in v0.46.0 and v0.49.7, ?NNN placeholders are used in:

  • packages/plugin/src/features/magic-context/message-fts-session-filter.ts:9 (MESSAGE_FTS_SESSION_FILTER_SQL, session_id = ?1)
  • packages/plugin/src/features/magic-context/search.ts (e.g. getMessageSearchStatement: session_id = ?1 … MATCH ? … LIMIT ?)
  • packages/plugin/src/tools/ctx-expand/search-archive.ts (?1 / ?2 / ?3)
  • packages/plugin/src/features/magic-context/compartment-storage.ts

The arguments are passed positionally (stmt.all(a, b, c) / rows.iterate(sessionId, start, end)), which is the failing form. In search.ts, ?1 is only used on the session-first path (when the session's FTS rowid map is complete), so the same session fails on some searches and works on others. That explains the "intermittent" pattern.

Why I think this belongs here, not only upstream

The docs and packages/cli/package.json declare Node ^22.16.0 || >=24.0.0. That range includes 22.16–22.19 and 24.0–24.6, and on all of those builds ?NNN with positional args fails. Users inside the supported range can hit this.

Possible directions (your call)

  1. Raise the documented / checked minimum to >=22.20.0 <23 || >=24.7.0, and add a fail-fast node:sqlite preflight (e.g. SELECT ?1 with one positional arg at startup) that prints an "upgrade Node" message instead of surfacing SQLITE_RANGE from a tool call.
  2. Make the prepare() shim in packages/plugin/src/shared/sqlite.ts bind ?NNN statements as an object { 1: a, 2: b, … }. On 22.19.0 that form works. I'll follow up below with results on a fixed Node build before suggesting this as safe.

Plugin version

pi-magic-context 0.49.52

Platform / Client

macOS 27.2.1 arm64 · Node v22.19.0 (Homebrew node@22) · Pi 1.5.2

Diagnostics

doctor --issue --report shows no config parse errors and no extension conflicts. The SQLite error surfaces as a ctx_search tool error in the session, not as an error-shaped plugin log line, so the sanitized log excerpt doesn't contain it. I can attach the full report if useful.

Activity

  1. magic-alfonso commented on Nov 12, 2026

    @magic-alfonso

    Thanks for the precise report, the repro saved a lot of time. Confirmed.

    I ran your statements on stock Node builds on Linux. On 22.16.0, 22.19.0 and 24.6.0, a statement with a numbered placeholder (?1, or ?1 mixed with ?) rejects positional arguments with column index out of range. On 22.20.0 and 24.7.0 the same statements work. Plain ? works on all five, also when the same value is passed more than once.

    Magic Context uses numbered placeholders in the places you list (the session filter for message search, ctx_search's message queries, the archive search behind ctx_expand, and one summary lookup), always with positional arguments. Hosts that run on Bun are not affected, which is why our own suites did not catch it. Your explanation of the "intermittent" pattern is right too: the numbered form is only in the query on one of the two search paths.

    The fix is on our side: the supported range stays ^22.16.0 || >=24.0.0, and we are removing numbered placeholders from our SQL, so the same statements work on every Node in that range. A repository check will keep them from coming back, and the change is run once against Node 22.19.0 before it ships. It will be in the next patch release; I will post here when it is out.

    Until then, Node 22.20.0 or later on the 22 line, or 24.7.0 or later, avoids the error.

  2. yiyi-wei commented on Nov 12, 2026

    @yiyi-wei
    ContributorAuthor

    Follow-up: version matrix, and the ?1-as-object form looks safe

    I checked this across Node builds with a standalone script (no plugin code). Same 8 cases on each:

    Node ?1 positional ?1 as {1:…} ?1,?1,?2,?3 positional ?1,?1,?2,?3 as object
    22.16.0 ✗ range ✓ ✗ range ✓
    22.19.0 ✗ range ✓ ✗ range ✓
    22.20.0 ✓ ✓ ✓ ✓
    22.23.4 ✓ ✓ ✓ ✓
    24.6.0 ✗ range ✓ ✗ range ✓
    24.7.0 ✓ ✓ ✓ ✓
    24.12.0 ✓ ✓ ✓ ✓

    So the affected band really is 22.16–22.19 and 24.0–24.6 — both inside the declared ^22.16.0 || >=24.0.0. Object-form binding works on every version above, including the broken ones.

    Two details that matter for direction 2:

    • A statement mixing ?1 and a bare ? (which is what search.ts builds when sessionFirst is false — session_id = ?1 AND message_history_fts MATCH ?) exposes an ordering subtlety on the fixed builds. {1:'x', 2:'y'} is rejected with Unknown named parameter '2' — the bare ? is not a named slot — while {1:'x'} plus one extra positional arg binds correctly (a:'x', b:'y'). So a shim that rewrites ?NNN → {N: value} has to handle the mixed-arity case explicitly rather than just spreading everything into one object.
    • Since 22.20.0 a repeated ?1 is bound once and reused (SELECT ?1 AS a, ?1 AS b → x, x), which is what MESSAGE_FTS_SESSION_FILTER_SQL relies on. That behaves identically in object form on 22.16.0, so direction 2 does not regress older builds.

    Given that, direction 1 (raise the minimum + fail-fast preflight) fixes it for everyone with no SQL changes, while direction 2 also unblocks users who cannot upgrade Node. I'll do whichever you prefer; happy to send a PR against a design-approved issue if you'd rather track it that way.

  3. magic-alfonso commented on Nov 12, 2026

    @magic-alfonso

    Thanks for the matrix, it matches what I measured, and the extra builds (22.23.4, 24.12.0) are useful to have on record.

    Our replies crossed, so to be clear about the direction: we are taking neither of the two. The fix is to stop using numbered placeholders at all and pass the value again where a statement needs it twice. Plain ? with positional arguments works on every build in your table, including the broken ones, and it sidesteps the mixed case you found, where a statement with ?1 and a bare ? cannot be bound by one object. A shim would have to get that right for every statement shape; with plain ? there is nothing to get right.

    It is already being built on our side, together with a repository check against numbered placeholders and a run on Node 22.19.0, so no PR is needed for this one. I will post here when the release is out.

  4. yiyi-wei commented on Nov 12, 2026

    @yiyi-wei
    ContributorAuthor

    Thanks for the clarity — and for picking the third option. Plain ? with the value repeated is the right call: it removes the whole class of statement-shape edge cases instead of shimming around them, and it keeps the supported range intact so users on 22.16–22.19 don't have to upgrade.

    The repo-wide check against numbered placeholders is a nice bonus — this is exactly the kind of thing that comes back unnoticed.

    I'm on 22.23.4 now, so ctx_search is clean for me in the meantime. Ping me if you want the session-first search path exercised against the 22.19.0 run before it ships; otherwise I'll retest after the patch release and report back on this issue.

  5. magic-alfonso commented on Nov 12, 2026

    @magic-alfonso

    The fix is merged on master and will be in the next patch release, 0.49.53. Every search and archive statement now uses plain ? placeholders, and a repository check refuses numbered placeholders in SQL from here on.

    It was run on real Node binaries before merging: on 22.19.0 the original code failed all 31 query checks with the error you reported, and with the fix all 47 checks pass on both 22.19.0 and 22.20.0 (the count is higher because the run also covers diagnostic and fixture statements). The session-first search path you offered to exercise is among them.

    I will post here again when 0.49.53 is published. Thanks for the precise report and the version matrix.

  6. magic-alfonso commented on Nov 12, 2026

    @magic-alfonso

    Fixed in 0.49.53, now on npm. Every search and history-search query now uses plain ? placeholders, which every Node release binds correctly, and a repository check blocks numbered placeholders from coming back. Update the package and restart Pi to pick it up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions