Skip to content

Keep watch alive after rebuild errors - #8641

Merged
fhammerschmidt merged 11 commits into
masterfrom
fix/watch-rebuild-recovery
Oct 20, 2026
Merged

fhammerschmidt merged 11 commits into
masterfrom
fix/watch-rebuild-recovery

Conversation

@fhammerschmidt

@fhammerschmidt fhammerschmidt commented Sep 10, 2026 •

Copy link
Copy Markdown
Member

Watch mode could panic or exit when a configuration or source-layout change made full rebuild initialization fail. Initialization errors now propagate through the build lock, are reported, and keep the watcher alive for recovery.

After a failed initialization, source edits retry full initialization until the current configuration can be loaded, preventing compilation with stale suffixes, dependencies, or options. Malformed dependency configurations retain their resolved directory so fixing a newly added workspace dependency can trigger recovery. Unsupported JSX versions return configuration errors, and a Drop guard releases the build lock on errors and unwinding.

Dependency failures preserve exit code 2 and package/root-path diagnostic context. The package-tree progress line remains available; the detailed error is reported once.

Both recovery tests use temporary projects. The invalid-config test covers invalid JSON and unsupported JSX, records emitted files across recovery, and uses the same assertions for Rust and OCaml. Disabling Rust's source-retry fix makes the test fail with stale output, without relying on debug-log wording.

Recovery limit: installing a missing dependency or replacing a non-local dependency directory does not itself trigger a retry in Rust. Edit a watched source file or touch rescript.json after installation or replacement. Automatic detection remains a follow-up.

Validation:

  • Full CI passes on Linux x64 (OCaml 5.0 and 5.5), Linux ARM64, macOS ARM64, and Windows x64, including native watcher unit, parity, and focused integration tests.
  • All 57 shared canonical fixtures pass with the OCaml CLI on every platform and with the Rust CLI on Linux ARM64, including both recovery scenarios.
  • PR package publication, all eight npm/pnpm installation tests, coverage, changelog validation, and DCO pass.
  • Locally, all 150 Rust unit tests, Clippy with warnings denied, formatting, shell syntax, and whitespace checks pass. The stale-output negative control and 28 dependency-error checks verify the recovery and diagnostic assertions.

@fhammerschmidt
fhammerschmidt marked this pull request as ready for review September 10, 2026 09:25
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 10, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-09T14:22:30.923223Z f83cd4f Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@fhammerschmidt
fhammerschmidt force-pushed the fix/watch-rebuild-recovery branch from 26da9d5 to 38467bf Compare September 10, 2026 10:58
@fhammerschmidt
fhammerschmidt changed the base branch from master to fix/clean-invalid-projects September 10, 2026 10:59
@fhammerschmidt
fhammerschmidt added this pull request to stack #8642 September 10, 2026 10:59
@fhammerschmidt
fhammerschmidt force-pushed the fix/clean-invalid-projects branch from 96c35a5 to 9760751 Compare September 10, 2026 11:16
@fhammerschmidt
fhammerschmidt force-pushed the fix/watch-rebuild-recovery branch from 38467bf to 527e72a Compare September 10, 2026 11:16
@fhammerschmidt
fhammerschmidt force-pushed the fix/clean-invalid-projects branch from 9760751 to 9f7e428 Compare September 10, 2026 11:20
@fhammerschmidt
fhammerschmidt force-pushed the fix/watch-rebuild-recovery branch from 527e72a to 95cbeca Compare September 10, 2026 11:20
@fhammerschmidt
fhammerschmidt force-pushed the fix/clean-invalid-projects branch from 9f7e428 to a90a656 Compare September 10, 2026 11:38
@fhammerschmidt
fhammerschmidt force-pushed the fix/watch-rebuild-recovery branch from 95cbeca to 08c262a Compare September 10, 2026 11:40
@fhammerschmidt
fhammerschmidt force-pushed the fix/clean-invalid-projects branch from a90a656 to 766e69a Compare September 10, 2026 11:44
@fhammerschmidt
fhammerschmidt force-pushed the fix/watch-rebuild-recovery branch from 08c262a to 23913a4 Compare September 10, 2026 11:44
@fhammerschmidt
fhammerschmidt force-pushed the fix/clean-invalid-projects branch from 766e69a to 35b5524 Compare September 11, 2026 13:46
@fhammerschmidt
fhammerschmidt force-pushed the fix/watch-rebuild-recovery branch from 23913a4 to 4cc230a Compare September 11, 2026 13:46
@fhammerschmidt
fhammerschmidt force-pushed the fix/clean-invalid-projects branch from 35b5524 to 60b4bb6 Compare September 11, 2026 13:49
@fhammerschmidt
fhammerschmidt force-pushed the fix/watch-rebuild-recovery branch from 4cc230a to 784f83f Compare September 11, 2026 13:49
@fhammerschmidt
fhammerschmidt force-pushed the fix/clean-invalid-projects branch from 60b4bb6 to 5cea2cf Compare September 11, 2026 14:03
@fhammerschmidt
fhammerschmidt force-pushed the fix/watch-rebuild-recovery branch from 784f83f to bb678d0 Compare September 11, 2026 14:03
@fhammerschmidt
fhammerschmidt force-pushed the fix/clean-invalid-projects branch from 5cea2cf to 7132187 Compare September 11, 2026 15:38
@fhammerschmidt
fhammerschmidt force-pushed the fix/watch-rebuild-recovery branch from bb678d0 to 52e838b Compare September 11, 2026 15:38
@fhammerschmidt
fhammerschmidt force-pushed the fix/clean-invalid-projects branch from 7132187 to a32a29e Compare September 11, 2026 16:41
@fhammerschmidt
fhammerschmidt force-pushed the fix/watch-rebuild-recovery branch 2 times, most recently from a4a65f3 to 3e45f81 Compare September 11, 2026 20:24
@fhammerschmidt
fhammerschmidt removed this pull request from stack #8642 September 11, 2026 20:25
@fhammerschmidt
fhammerschmidt changed the base branch from fix/clean-invalid-projects to fix/watch-config-test-timeout September 11, 2026 20:25
@fhammerschmidt
fhammerschmidt added this pull request to stack #8650 September 11, 2026 20:25
@pkg-pr-new

pkg-pr-new Bot commented Sep 11, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

rescript

npm i https://pkg.pr.new/rescript@8641

@rescript/belt

npm i https://pkg.pr.new/@rescript/belt@8641

@rescript/darwin-arm64

npm i https://pkg.pr.new/@rescript/darwin-arm64@8641

@rescript/linux-arm64

npm i https://pkg.pr.new/@rescript/linux-arm64@8641

@rescript/linux-x64

npm i https://pkg.pr.new/@rescript/linux-x64@8641

@rescript/runtime

npm i https://pkg.pr.new/@rescript/runtime@8641

@rescript/win32-x64

npm i https://pkg.pr.new/@rescript/win32-x64@8641

commit: 0e3814b

Signed-off-by: Florian Hammerschmidt <florianh89@gmail.com>
@fhammerschmidt
fhammerschmidt removed this pull request from stack #8650 October 9, 2026 13:07
@fhammerschmidt
fhammerschmidt force-pushed the fix/watch-rebuild-recovery branch from 5e7c5a1 to 3377441 Compare October 9, 2026 13:09
@fhammerschmidt
fhammerschmidt changed the base branch from fix/watch-config-test-timeout to master October 9, 2026 13:10
@fhammerschmidt

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3377441a3d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread rewatch/src/watcher.rs
Comment thread rewatch/src/watcher.rs
Comment thread rewatch/tests/watch/07-watch-recovers-from-invalid-config.sh Outdated
Comment thread rewatch/tests/watch/07-watch-recovers-from-invalid-config.sh Outdated
Signed-off-by: Florian Hammerschmidt <florianh89@gmail.com>
@fhammerschmidt

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep them coming!

Reviewed commit: f83cd4ffd8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Signed-off-by: Florian Hammerschmidt <florianh89@gmail.com>
Signed-off-by: Florian Hammerschmidt <florianh89@gmail.com>
@github-actions

Copy link
Copy Markdown

@cknitt

cknitt commented Oct 13, 2026

Copy link
Copy Markdown
Member

🤖 This review was generated by Claude (Claude Code) on behalf of @cknitt.

Verdict: approve with nits. No correctness bugs found in the watch loop. Init errors propagate instead of panicking. initialization_failed forces a full re-init rather than an incremental build on stale state. Every fallible step in initialize_build runs before cleanup_previous_build, so a failed init leaves nothing half-cleaned. Both new tests fail without the fix.

  • Exit code change: the two process::exit(2) calls in packages.rs are gone and main.rs's exit_code returns 1. rescript build/clean/format now exit 1 instead of 2 when a dependency is missing or has a malformed config. weird_deps/weird_devdeps only assert != 0, so CI doesn't catch it. Please acknowledge it in the PR/changelog, or keep 2.
  • Missing dependency dir gets no recovery watch: only DependencyConfigError adds one. Example: a dependency added to rescript.json before npm install. Finishing the install produces no event, so watch stays failed until a .res file or rescript.json is touched. The same happens when a non-local dependency with a bad config is reinstalled, because the directory is replaced. Still far better than exiting; could be documented, or the root node_modules watched non-recursively while init is failing.
  • Duplicate test number: 07-watch-recovers-from-invalid-config.sh sits next to the existing 07-changed-interface-after-failed-implementation.sh. Renumber the new ones to 08/09.
  • Test 07 can pass without proving anything: if the mv of the invalid config spans two debounce windows, a second init failure can bump initial_errors before the Test.res edit is processed, and the "no marker in Test.mjs" check then passes trivially. This can't cause a flaky failure, only a false pass. Waiting for a marker tied to the edit would tighten it.
  • Nits:
    • With show_progress, dependency errors are printed twice (inspect_err in packages.rs, then eprintln!("{error:#}")).
    • The rustfmt re-indent in watcher.rs (~509-540) and packages.rs makes the diff noisier.
    • Out of scope, but other panics can still kill watch mode after a config change, e.g. "JSX version {v} is unsupported" (config.rs:834) and .expect("Could not canonicalize") (packages.rs:562). Possible follow-up.

Signed-off-by: Florian Hammerschmidt <florianh89@gmail.com>
@codecov

codecov Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.13%. Comparing base (6a3eed1) to head (0e3814b).

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #8641   +/-   ##
=======================================
  Coverage   81.13%   81.13%           
=======================================
  Files         456      456           
  Lines       63750    63750           
=======================================
  Hits        51724    51724           
  Misses      12026    12026           
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Log the source event path and its requested compile type once, then wait for the edited source to request full initialization and for an error after that event. Delayed config failures can no longer satisfy the recovery assertion before the edit is processed.

Signed-off-by: Florian Hammerschmidt <florianh89@gmail.com>

Copy link
Copy Markdown
Member Author

The follow-ups document the exit-code change and dependency-install recovery limit, renumber the tests, and remove duplicate diagnostics. The recovery regression now waits for the source event and a subsequent initialization failure; disabling recovery makes it fail and reproduces stale output. Automatic detection of newly installed or replaced dependencies remains a follow-up; editing a watched source or touching rescript.json triggers recovery. CI, including the package-installation checks, coverage, and changelog validation now all pass.

Retain the overlapping-source regression, renumber the recovery scripts, and update parity checks for Rust recovery and dependency error exits. Preserve and document the existing OCaml differences.

Signed-off-by: Florian Hammerschmidt <florianh89@gmail.com>

@cknitt cknitt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked against current master (applies cleanly, bugs still reproduce: invalid root config panics at watcher.rs:619, missing/malformed dependency exits 2, and lib/build.lock + lib/watch.lock are left behind). The OCaml implementation is not affected. The core watcher fix works for me: no stale output while the config is broken, and a correct rebuild once it's fixed.

A few things before merging:

  1. Same class of panic remains. Changing the config to "jsx": {"version": 3} during watch still panics in Config::get_jsx_args (panic!("JSX version {version} is unsupported")) and again leaves both lock files behind. Turn that into a config error. A Drop guard in with_build_lock that releases the lock would also make this robust.
  2. Exit code 2 → 1 breaks parity. Rust and OCaml currently both exit 2 for dependency errors (OCaml deliberately, rescript_ocaml.ml maps Package_error to 2). I'd keep 2 in Rust (e.g. downcast the dependency error in exit_code) instead of documenting a new intentional difference. If we do want 1, change both implementations together, and it's a user-visible change rather than a .fix.
  3. Error messages lost context. The malformed-dependency error no longer names the package (Could not build package tree for 'bad' at path … → Could not initialize build: Could not read dependency config at …), and the missing-dependency error loses at path '<root>'. Restoring those would also avoid loosening require_both_errors_contain to per-implementation fragments.
  4. Tests. Test 09 skips the source-retry check for OCaml, so its "no stale output" assertion passes trivially there. The Rust branch depends on the wording of a debug log line. Test 10 edits the shared testrepo/packages/standalone config, so a temporary fixture like test 07 would be safer.
  5. Minor: dropping show_progress from packages::make also removes the [1/2] … Error building package tree stdout line, which is outside this PR's scope.

Optional follow-up: OCaml recovers automatically when a missing dependency gets installed (it watches resolution candidates); Rust still needs a manual touch, as the description notes.

Reject unsupported JSX versions while reading configuration, and release the
build lock through a Drop guard on every return or unwind. Retain dependency
failures as typed error context so the CLI preserves exit code 2, package and
root-path diagnostics, and the package-tree progress message.

Review: Both recovery tests use temporary projects. The invalid-config test
records every emitted output across invalid JSON and JSX configurations and
recovery, with the same assertions for Rust and OCaml. Disabling Rust's source
retry makes it detect stale output without depending on debug-log wording.

Validation: Native compiler/CLI and Rust builds, 150 Rust unit tests, all 42
build fixtures, 297 configuration parity cases, 28 focused dependency-error
checks, Clippy with warnings denied, formatting, changelog, and shell syntax
checks pass. Both recovery scripts pass for both CLIs using isolated polling
builds; native filesystem events are unavailable locally and remain for CI.

Area: rewatch tests docs
Output: unchanged; no generated JavaScript changes
Tests: ~rewatch/src/build.rs ~rewatch/src/config.rs ~rewatch/tests/watch/09-watch-recovers-from-invalid-config.sh ~rewatch/tests/watch/10-watch-recovers-new-dependency-config.sh ~rewatch-ocaml/tests/check_command_validation.sh ~rewatch-ocaml/tests/config_acceptance_cases.tsv ~tests/build_tests/weird_deps/input.js ~tests/build_tests/weird_devdeps/input.js
PR: #8641
Signed-off-by: Florian Hammerschmidt <florianh89@gmail.com>
Bring in #8902, which waits for duplicate change events to settle before
asserting that an unrelated edit retries a failed parse. Keep the reviewed
watcher recovery changes and their regressions intact, alongside the merged
cleanup and playground API compatibility changes.

Validation: Native compiler/CLI and Rust builds, 150 Rust unit tests,
Clippy with warnings denied, Rust formatting,
canonical suite coverage, changelog, shell syntax, and whitespace checks pass.
47 of 48 OCaml build-system unit cases pass; the native watcher case hits the
local filesystem-event EMFILE limit and remains for CI. The retained-parse
test matches master exactly.

Signed-off-by: Florian Hammerschmidt <florianh89@gmail.com>
The Windows recovery assertions pass, but cleanup can fail when the background
launcher exits between the process check and kill. Match the existing watcher
test's best-effort launcher cleanup in both new recovery scripts. Shutdown
errors from exit_watcher still fail before cleanup removes the fixture.

Validation: The native toolchain builds. A focused probe reproduces the old
failure in both scripts and verifies the fix for an already-exited launcher,
a running launcher, and a failed watcher shutdown. Shell syntax and whitespace
checks pass.

Signed-off-by: Florian Hammerschmidt <florianh89@gmail.com>
Git Bash's background-process ID is not the native Windows process ID
expected by process_is_running. Read the native PID from watch.lock for the
recovery test's liveness assertion. Let Bash kill and reap its own launcher
without passing that launcher ID through the native-process check.

Validation: The native toolchain builds. A focused probe reproduces the
false failure with different Bash and native IDs, accepts the live native
watcher, and rejects an exited watcher. Both cleanup routines pass the
already-exited and running-launcher probes and preserve shutdown failures.
Shell syntax and whitespace checks pass.

Signed-off-by: Florian Hammerschmidt <florianh89@gmail.com>
@github-actions

Copy link
Copy Markdown

@cknitt cknitt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rechecked at 0e3814b. All points from my previous comment are addressed:

  1. "jsx": {"version": 3} is now a config error instead of a panic, and the build lock is released by a Drop guard.
  2. Dependency errors exit 2 again, matching OCaml.
  3. Errors name the package and root path again; the parity check uses one shared fragment.
  4. Recovery tests use temporary projects, detect stale output via js-post-build instead of log wording, and assert the same for Rust and OCaml.
  5. The [1/2] Error building package tree. line is back.

Verified locally with a build of this head: cargo test passes. In watch mode, invalid root JSON, an unsupported JSX version, a missing dependency and a malformed dependency all keep the watcher alive; a source edit while the config is broken emits nothing, and fixing the config rebuilds with the new suffix. build/clean/format exit 2 for dependency errors and 1 for root config errors and leave no locks behind.

Not blocking: automatic recovery on dependency installation remains a follow-up, as described. Separately, stopping watch with SIGTERM leaves lib/watch.lock behind, but master does the same, so that's not from this PR.

LGTM.

@fhammerschmidt
fhammerschmidt merged commit 8556f0a into master Oct 20, 2026
28 checks passed
@fhammerschmidt
fhammerschmidt deleted the fix/watch-rebuild-recovery branch October 20, 2026 15:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants