Skip to content

fix(server): a Windows Codex shadow keeps each SQLite database with its WAL - #38714

Closed
GibSmith wants to merge 27 commits into
pingdotgg:mainfrom
GibSmith:fix/codex-sqlite-wal-windows
Closed

GibSmith wants to merge 27 commits into
pingdotgg:mainfrom
GibSmith:fix/codex-sqlite-wal-windows

Conversation

@GibSmith

Copy link
Copy Markdown

Note

Submitted by claude-opus-5-5 on behalf of Gib. Code by model:

  • The commits from fix(server): keep Codex shadow SQLite state together #29574 are nkcbuilds', prepared with GPT-6 through ChatGPT using the T3/MoonDesk harness.
  • The Windows follow-up commits were written by Claude Fable 5.2, except one test fix by Claude Opus 5.5, in Claude Code running in T3 Code.
  • GPT-6 Astra reviewed each round.

Continues #29574, which stopped in automated review with one major Windows finding open. This branch is that PR's head (3001024b77, nkcbuilds' commits unchanged), a conflict-free merge of main, and 5 commits that close the finding and the gaps a re-review found after it. If you'd rather land it through #29574, its branch fast-forwards to this one.

Fixes #24486

Problem

A Codex account with a shadow CODEX_HOME shares the main home's SQLite databases. T3 used to symlink them into the shadow. Windows SQLite names the -wal and -shm files after the path it is given, so a database opened through the link gets a WAL of its own, apart from the shared home's writers. Committed state splits and goes missing. The leftover sidecars also disable the provider at the next start with Cannot create Codex shadow home entry 'goals_1.sqlite-shm' … already exists and is not a symlink.

How we found it. CodexWork, our second Codex account, kept getting disabled on Windows 11 by exactly that error, at least four times between Nov 3 and Nov 18. On native Windows the legacy layout reproduces the split. Through node:sqlite, a row written through the link is gone after reopening, and integrity_check still says ok.

Fix

From #29574: the shadow keeps CODEX_HOME for its private auth, and T3 points Codex at the shared databases with CODEX_SQLITE_HOME. T3 never links a database into a new shadow, and refuses ambiguous mixed or stranded SQLite state before changing anything. The driver, orchestration adapter, managed runtime, text generation and provider terminals all use the same routing.

The stopped review's case is a shadow with its own state_5.sqlite while the shared home has goals_1.sqlite. With no override, 3001024b77 already handles it: on native Windows it refused before linking anything and left both homes unchanged. The gap was an explicit CODEX_SQLITE_HOME. It returned before every check, so pointing it at a shadow that holds a symlinked database brought the split back. The follow-up commits change only Windows behaviour:

  1. cfd6b1bdfb: an explicit CODEX_SQLITE_HOME other than the shared home is refused if it holds a symlinked .sqlite database.
  2. 1e110ac3ca closes three more routes to a split:
    • Codex ranks sqlite_home from any config layer above CODEX_SQLITE_HOME. T3 now passes the directory it checked as a final -c sqlite_home=… to codex app-server, codex exec and codex mcp list. It refuses a root sqlite_home in the shadow's or shared home's config.toml, or in the launch args, because Codex typed into a provider terminal gets no -c from T3.
    • A relative CODEX_SQLITE_HOME is refused. Probes, sessions and text generation each start Codex in a different directory.
    • The override is compared with the shared home by real path, ignoring case. An alias of the shared home reaches Codex as the shared home's own spelling.
  3. 846e6ddb83: a provider terminal merges the host, client and instance environments by name ignoring case, keeping instance over client over host. Before, a client's codex_sqlite_home lost to the host's CODEX_SQLITE_HOME.
  4. 766ced0bf8: only Codex's root sqlite_home counts. An MCP server's sqlite_home environment variable, the text inside a multi-line string and -c mcp_servers.<name>.env.sqlite_home=… are not refused. config.toml is read by an 85-line scanner, because apps/server has no TOML parser.
  5. f7b860b87f: three CodexHomeLayout tests failed on native Windows only, which CI doesn't run. Their expectations now include the pin variable and Windows path resolution. The relative-path test no longer assumes the temp directory is on the repo's drive.

Every refusal names the files and the steps to recover, and leaves all SQLite files unchanged.

Verification

Native Windows 11 Pro 26200, Codex CLI 0.170.0, Node 26.13.1, on this tree

Focused suites, one file at a time:

Suite Passed Skipped Failed
CodexHomeLayout 20 1 0
codexSqliteHomeSetting 29 0 0
codexLaunchArgs 22 0 0
CodexAdapterV2 203 1 0
CodexTextGeneration 32 0 0
CodexProvider 7 0 0
CodexManagedErrors 6 0 0
terminal/Manager 117 1 3
CodexManagedRuntime 2 0 2
CodexDriver 3 14 1

All 6 failures fail the same way on clean main 8cf97b5f06:

CodexTextGeneration also exits 1 from two unhandled ECONNRESET errors, the same on main.

The -c pin survives Windows spawning. T3 runs codex.CMD through cmd.exe with shell: true, so the chain is cmd → codex.cmd → node → codex.exe. We drove the real CLI through both T3 paths, CodexAdapterV2.open and the CodexProvider probe. A trusted project's .codex/config.toml set sqlite_home to the shadow:

  • With no pin, all 15 SQLite files (5 databases with their -wal and -shm) landed in the shadow, from the project layer.
  • With the pin, config/read reported sqlite_home from sessionFlags, and all 15 files landed in the shared home, none in the shadow.
  • The same held for a shared home named t3 sqlite's shared. codex.exe received -c "sqlite_home=\"C:\\…\\t3 sqlite's shared\"" and decoded the exact path.

config.toml guard, on real files. A root sqlite_home is refused in the shadow's or the shared config with LF, CRLF, BOM+CRLF and a quoted key. The MCP environment key, a commented-out key and the key inside """ are allowed. Refusals left both trees unchanged.

Override guard, on cfd6b1bdfb.

  • It refuses a symlink, a junction (mklink /J) and a directory symlink to a shadow holding a symlinked goals_1.sqlite.
  • It allows the shared home lowercased or with a trailing backslash, and a shadow with only its own databases.
  • Refusals leave the files byte-identical.

Terminal merge. The client beats the host whichever spelling each uses, and the instance beats both. T3CODE_CODEX_SQLITE_HOME, which carries the pin, never reaches the terminal.

Real Codex app-server. With CODEX_SQLITE_HOME set to the shared home, it created every database and its sidecars only in the shared folder.

Linux (Raspberry Pi), Codex CLI 0.170.0

  • The same ten suites: 464 passed.
  • Server typecheck: 0 errors.
  • Formatter and built-in lint on the touched files: clean. The repo's lint plugin and knip ran out of memory there.
  • We ran 27 configs and launch args through the real codex app-server and compared where it opened its databases with T3's verdict. 25 agree, and 2 are refused on purpose (below). In none did Codex use sqlite_home while T3 allowed the launch.
  • Each follow-up commit's regression tests fail against the code before it.

Limitations

  • Stricter than Codex on purpose, in two cases:
    • A config.toml with an unclosed string or array is refused if any line could set the key. Codex drops such a file.
    • -c '"sqlite_home"=…' is refused. Codex 0.170.0 ignores it.
  • Hard links aren't detected. Only symlinked databases are. A hard-linked database also splits on Windows, which we reproduced natively, but T3 never creates hard links, so that is left for a separate change.
  • The pin covers only commands T3 builds. Codex typed by hand into a provider terminal still follows a sqlite_home from a layer T3 can't read, such as a project's .codex/config.toml.
  • The spawn test supplied its own cwd. CodexAdapterV2.open passes no cwd, so the test passed the project itself. config/read for that project still resolved to the pin.
  • Not tested: a sqlite_home from Codex's managed requirements, and real managed ChatGPT OAuth.
  • Out of scope: no automatic recovery of an already split database, and no cross-process SQLite locking. Stray lock files such as auth.json.lock disabling a provider are fix(server): new Codex home entries no longer disable shadow-home providers #32058's problem, not this one.

🤖 Generated with Claude Code

nkcbuilds and others added 27 commits November 3, 2026 02:18
The dirty-legacy SQLite regression test was reading process.env inside
Effect.gen, which effect-tsgo rejects (TS377077). Match the other
managed-runtime tests and take PATH/SystemRoot from HostProcess.Environment.
Route provider-bound terminals through canonical SQLite storage, reject mixed or overlapping homes, and preserve actionable recovery guidance. Serialize concurrent shadow setup and cover Windows path aliases, overrides, and legacy state with focused regressions.

Prepared with GPT-6 through ChatGPT using the T3/MoonDesk harness.
Only run SQLite override and shadow-terminal materialization cases on hosts with symlink support. Keep safe preflight and non-overlay tests active on all platforms.

Prepared with GPT-6 through ChatGPT using the T3/MoonDesk harness.
Legacy SQLite links in a shadow home are inert once Codex opens databases
from CODEX_SQLITE_HOME, and on macOS and Linux SQLite already kept their
sidecars beside the shared file. Refusing them would stop every existing
shadow-home instance after upgrade. Refuse only a shadow WAL or journal that
still holds data for a database it does not own.

Drop the realpath alias check, in-process materialization lock and terminal
shadow materialization, which the SQLite fix does not need, and cut tests to
the routing cases.

AI-assisted; verified with the tests and tools listed above.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
AI-assisted; verified with the tests and tools listed above.
…hers

AI-assisted; verified with the tests and tools listed above.
…nking

When a shadow home keeps its own SQLite databases and the shared home has
others, the shadow was routed to its own SQLite home with the shared
databases symlinked in. POSIX SQLite opens a symlinked database by its
target's path, so its WAL stays beside the shared file. Windows SQLite
derives the WAL from the link's path, so a shadow writer and a shared-home
writer would each keep their own WAL for one database, which is the split
reported in pingdotgg#24486.

On Windows, refuse that layout before touching either home, including a
shadow whose legacy link points at a shared database that is gone, and
never link a database during setup even if the shared home gained one
after the check. Name both homes and both databases in the recovery
message. macOS and Linux keep the current routing.

Tests cover the reviewer's case, the dangling legacy link and the
check-to-setup race on Windows, keep the existing mixed-home test on
POSIX, and use node:sqlite to show a POSIX shadow and the shared home
share one WAL through the link.
Since pingdotgg#35912 a Codex shadow home shares the instance environment's
CODEX_HOME when no home path is set. Provider terminals still resolved the
shared home from settings alone, so they routed CODEX_SQLITE_HOME to
~/.codex while the provider used the environment's home. Resolve it the
way the provider does.
…se through a link

An explicit CODEX_SQLITE_HOME skipped every SQLite check. On Windows, naming
a directory that holds a linked database, such as a shadow home with legacy
links or the goals_1.sqlite link the old mixed-home fallback made, launched
Codex against the link. Windows SQLite names the WAL after the path it is
given, so that Codex kept its own WAL apart from the shared home's writers.
This is the split reported in pingdotgg#24486, reached through the override instead of
the fallback.

On Windows, refuse an explicit SQLite home that holds a symlinked database,
unless it is the shared home. The directory itself is read, so an alias of
the shadow is caught too. The provider, the orchestration adapter, the managed
runtime and provider terminals all resolve their environment here. macOS and
Linux are unchanged, since SQLite follows the link itself there. A relative
home is not inspected, because Codex resolves it per session.

The Windows regression test now covers the review's fixture end to end: a
real shadow state_5.sqlite beside a shared goals_1.sqlite that a shared-home
writer holds open. It checks the provider and terminal paths, with and
without the override, and that two writers routed to the shared home share
one WAL. The legacy-link test uses the reported layout of four linked
databases with index and empty WAL sidecars.

Written by Claude Fable 5.2 in the T3 Code Claude Code harness.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…T3 checked

Review of cc72e96d54 found three ways a Windows shadow home could still open
a shared database through a link, so that it kept a WAL apart from the shared
home's writers (pingdotgg#24486).

Codex ranks `sqlite_home`, from any config.toml or a `-c` launch arg, above
CODEX_SQLITE_HOME. A shadow whose config named itself opened its legacy links
while T3 reported the shared home. T3 now passes the directory it checked as
a final `-c sqlite_home=...` on every Codex command it builds: app-server
sessions and probes, exec and mcp list. Codex 0.170.0 takes the last `-c`
over every config layer, including a trusted project's .codex/config.toml,
which T3 cannot read before launch. A `sqlite_home` in the launch args or in
the shadow's or shared home's config.toml is refused with guidance, because
Codex typed into a provider terminal gets no argument from T3, and silently
outranking the setting would move a user's history. The config file is
matched by line rather than parsed, and a line that only looks like the key
is refused too.

A relative CODEX_SQLITE_HOME skipped the check, and probes, sessions and text
generation each resolve it against a different directory. It is refused.

The shared home was recognised by its exact spelling only. An alias of it was
inspected for links, and another letter case or a trailing separator reached
Codex as written. The override is now compared by real path, ignoring case,
and Codex gets the shared home's own spelling or the directory's real path.

macOS and Linux are unchanged: SQLite opens a link by its target there.

Written by Claude Fable 5.2 in the T3 Code Claude Code harness.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…tever its casing

Windows reads environment names without regard to case, but a provider
terminal merged the host, client and instance environments as plain objects.
A client's `codex_sqlite_home` sat beside the host's CODEX_SQLITE_HOME, the
host's spelling was the one checked, and its value was written back over the
client's choice (pingdotgg#24486 review).

The terminal now merges the three by name ignoring case on Windows, keeping
instance over client over host. The SQLite check reads the variable the same
way and returns a single CODEX_SQLITE_HOME, so the provider and the terminal
agree on which value Codex gets.

Written by Claude Fable 5.2 in the T3 Code Claude Code harness.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…Codex's own setting

The Windows check for a `sqlite_home` that would outrank CODEX_SQLITE_HOME
matched the text `sqlite_home =` anywhere. It refused a shadow whose MCP
server had an environment variable of that name, whose instructions
mentioned the setting inside a multi-line string, or whose launch args set
`mcp_servers.<name>.env.sqlite_home`. Codex 0.170.0 accepts all three and
keeps its databases where CODEX_SQLITE_HOME points (pingdotgg#24486 review).

Codex reads `sqlite_home` from the root table only. config.toml is now
scanned for that key alone: before the first table header, outside strings,
comments, arrays and inline tables, bare or quoted, with a quoted key's
escapes decoded. A launch arg counts only when the key before its first `=`
is `sqlite_home`. `sqlite_home.x` and `[sqlite_home]` are tables, which
Codex does not read as a path, so they are not refused.

The scanner is 85 lines and apps/server has no TOML parser to reuse. Where
a string or array is never closed, the old line match still decides, so a
file Codex cannot parse is refused if any line could assign the key. A
quoted key in a launch arg is refused too, although Codex 0.170.0 ignores
it.

The old match also missed a root key written with an escape, such as
"sqlite_home", or behind a byte order mark. Codex honours both, and
both are now refused.

Written by Claude Fable 5.2 in the T3 Code Claude Code harness.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Three CodexHomeLayout tests failed on native Windows. CI runs on Linux and
macOS only, so it did not see them.

Two expected only CODEX_SQLITE_HOME. On Windows T3 also passes the checked
directory as T3CODE_CODEX_SQLITE_HOME, for the final -c sqlite_home pin, and
resolves a rooted path such as /elsewhere onto the working directory's
drive. The expectations now say so.

The relative CODEX_SQLITE_HOME test took path.relative from the working
directory to a temp shadow. When the temp directory is on another drive
there is no relative path, so the test handed the guard an absolute one.
It now writes the relative path out.

Written by Claude Opus 5.5 in the T3 Code Claude Code harness.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 18, 2026

Copy link
Copy Markdown
Contributor

Caution

Review failed

An error occurred during the review process. Please try again later.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

@github-actions github-actions Bot added the vouch:unvouched Author is not yet trusted in the VOUCHED list. label Nov 18, 2026
@GibSmith

Copy link
Copy Markdown
Author

Closing in favour of continuing #29574, as CONTRIBUTING asks. The branch and the Windows evidence are posted there.

@GibSmith GibSmith closed this Nov 18, 2026
@juliusmarminge juliusmarminge added the takeover:in-progress A takeover lead is working on this PR label Nov 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

takeover:in-progress A takeover lead is working on this PR vouch:unvouched Author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Codex shadow home splits each SQLite database from its WAL, corrupting shared state and disabling the default Codex instance

3 participants