Skip to content

fix(shared): SQLite reads keep sorts and temp tables in memory again - #39386

Open
seankd3 wants to merge 1 commit into
pingdotgg:mainfrom
seankd3:fix/reader-temp-store
Open

seankd3 wants to merge 1 commit into
pingdotgg:mainfrom
seankd3:fix/reader-temp-store

Conversation

@seankd3

@seankd3 seankd3 commented Nov 19, 2026

Copy link
Copy Markdown

Problem

#24194 set PRAGMA temp_store = MEMORY so SQLite temp files would stop blocking the server. That pragma runs only on the writer connection. The reader workers added later for concurrent reads (#27979) open their own connections and set only busy_timeout. So every read that sorts or builds a temp table still spills to a file in the system temp directory (SQLITE_TMPDIR, TMPDIR or /var/tmp).

I found it on a Linux desktop whose temp directory sat on a slower disk than the T3 database. Under load, the server's WorkerThreads, which are the SQLite reader workers, kept appearing in uninterruptible sleep in balance_dirty_pages. In 13 of 84 stuck-thread samples, a worker was in pwrite64 on a deleted /var/tmp/etilqs_* file. During that window the server's SQLite span p99 was 225 ms and the slowest span took 25.7 s. Moving /var/tmp to a fast disk cut that p99 to 86 ms, which points at these temp files.

Change

nodeSqliteClient's worker source runs PRAGMA temp_store = MEMORY right after busy_timeout when a worker opens its connection. Readers get no pragmas from callers, since callers configure the writer through SqlClient, so this is the one place a reader's connection settings are made. A writer worker also runs it at open, and caller pragmas run afterwards and still override it.

Scope

A small fix to a regression of #24194. It needs no separate issue.

Verification

  • New test, keeps a reader's sorts and temp tables in memory, not in temp files. It reads PRAGMA temp_store inside SqliteClient.readOnly. Before the fix: expected +0 to equal 2. After: it passes. All 37 tests in nodeSqliteClient.test.ts pass.
  • I checked this with the SQLite that T3 ships: Electron's Node 24.22.1 (ELECTRON_RUN_AS_NODE=1), SQLite 3.53.4, compiled with TEMP_STORE=1. A read-only connection that sorts 200,000 rows of a 22 MB WAL database, with the worker's settings (busy_timeout only), opened one $SQLITE_TMPDIR/etilqs_* file (counted with strace -e openat). With temp_store = MEMORY it opened none.
  • pnpm run typecheck in packages/shared passes. The commit hooks (vp fmt, vp lint, knip:check) pass.

Not checked: the full server under load with this build. The evidence above comes from the installed release plus the isolated probe.

Created with Claude Opus 5.5 in Claude Code.

pingdotgg#24194 set temp_store = MEMORY so SQLite temp files stop blocking the server.
That pragma runs only on the writer. The reader workers added for concurrent
reads open their own connections and set only busy_timeout, so every read that
sorts or builds a temp table spills to a file in the system temp directory.
Each reader now sets temp_store = MEMORY when it opens.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added vouch:unvouched Author is not yet trusted in the VOUCHED list. size:XS 0-9 changed lines (additions + deletions). labels Nov 19, 2026
@coderabbitai

coderabbitai Bot commented Nov 19, 2026

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The SQLite worker sets PRAGMA temp_store = MEMORY after opening its connection. A reader-client test checks that PRAGMA temp_store returns 2 within a read-only transaction.

Changes: SQLite temporary storage

Layer / File(s) Summary
Worker setting and reader test
packages/shared/src/nodeSqliteClient.ts, packages/shared/src/nodeSqliteClient.test.ts
The worker sets temporary storage to memory after opening the database connection. The reader test checks the pragma value within a read-only transaction.

Estimated code review effort: 2 (Simple) | ~8 minutes

Change: Bug fix

Suggested reviewers: juliusmarminge, t3dotgg, yash-singh1

Merge Risk: Minimal · up to c6448

This change keeps SQLite reader sorts and temporary tables in memory instead of spilling to temp files. It is a small, easily reversible setting and no merge-blocking risk was found.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title names the main change: the fix makes SQLite reads keep sorts and temp tables in memory again, which matches the added PRAGMA temp_store = MEMORY in the worker connection setup. It follows th…
Description check Passed The description follows the repository template closely. It has Problem, Change, Scope, and Verification sections. The Problem section explains the regression, with the reader workers from #27979 miss…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the main change: the fix makes SQLite reads keep sorts and temp tables in memory again, which matches the added PRAGMA temp_store = MEMORY in the worker connection setup. It follows the conventional commit format and uses plain language. The title is specific enough for a teammate scanning history to understand the primary change.

Full details: Description check

Explanation

The description follows the repository template closely. It has Problem, Change, Scope, and Verification sections. The Problem section explains the regression, with the reader workers from #27979 missing the pragma that #24194 set on the writer. The Change section explains the one-line worker fix and why callers cannot set reader pragmas. The Scope section says the fix is a small regression fix and needs no separate issue. It links the related PRs but does not link a separate issue, which the template allows for small obvious fixes when the author explains why. The Verification section gives a specific test, before and after results, a standalone probe with the shipped SQLite, and states what was not checked. The model and harness line is present. The description is mostly complete and specific.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 66b8a94 and c64484e.

Files selected for processing: 2

  • packages/shared/src/nodeSqliteClient.test.ts
  • packages/shared/src/nodeSqliteClient.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XS 0-9 changed lines (additions + deletions). vouch:unvouched Author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant