Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m
`docs/RELEASE-PROCESS.md` §8.

## Unreleased
- fix(build-tree): **`bin/build-tree.py` excludes `node_modules` at any depth, not only at the project root (#764)**: `skip()` ran only for top-level names, so a custom widget's nested `widgets/<W>/node_modules` (1.2 GB in the field) was copied into `.mxtk/build/model/` on every full build, and one exec stalled for 13+ minutes. A nested `node_modules` already in an existing copy is removed on the next sync; the other excluded names (`deployment`, `docs`, `.mxcli`, `.mxtk`, ...) stay top-level only, since a widget may hold its own. The one exception is the top-level `javascriptsource/` tree, whose `node_modules` hold JavaScript actions' npm dependencies that mxbuild bundles: those are still copied. `test-build-tree.sh` B4c covers both. — a Greenfield field run (2026-11-13)

## v2026.11.16
- fix(gitignore): **`bin/doctor.sh` appends its machine-local entries (`.doctor-receipt`, `toolkit.env`, `progress-report-sessions.txt`, `.property-sets-receipt`) through `mxtk_gitignore_ensure`**: a CRLF `.gitignore` got LF lines (mixed endings) and could get an entry it already had added again; LF files get the same bytes as before, and the "(added … to the project's .gitignore)" note now prints only when a line was actually written — follow-up to the .gitignore line-ending fixes, 2026-11
Expand Down
19 changes: 14 additions & 5 deletions project-bin/build-tree.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,14 @@
Excluded top-level names: version control, the build output, caches and the toolkit's own state, and
the project-output folders the toolkit keeps beside a single-tree model (analysis/, sources/,
knowledge-base/, docs/); MXTK_BUILD_TREE_EXCLUDE adds space-separated names. Studio Pro's
*.mpr.lock is never copied (it belongs to the real model, #577).
*.mpr.lock is never copied (it belongs to the real model, #577). Those names are excluded at the top
only: below it they can be legitimate content (a widget's own docs/ or deployment/ folder).
node_modules is excluded at ANY depth (#764): a custom widget's source folder held a 1.2 GB
node_modules that was copied on every exec (one exec stalled 13+ minutes); the build reads the
widget's built .mpk in widgets/, never its sources' dependencies. A nested node_modules already in
an older copy is removed on the next sync. The ONE exception is the top-level javascriptsource/ tree:
a JavaScript action's npm dependencies live in javascriptsource/<module>/**/node_modules and mxbuild
bundles them, so they are copied like any other source (review of #769).

Exit 0 = synced (stdout: "synced N file(s), removed M", the model directory copied, then the copy's
.mpr path); 1 = it could not
Expand All @@ -33,25 +40,27 @@

EXCLUDE = {".git", ".svn", ".hg", "deployment", "node_modules", ".mpr-snapshots", ".mxtk", ".mxcli",
".claude", "analysis", "sources", "knowledge-base", "docs"}
EXCLUDE_ANY_DEPTH = {"node_modules"} # dependency folders: never a model input, wherever they sit (#764)


def skip(name):
return name in EXCLUDE or name.endswith(".mpr.lock")


def sync_dir(src, dst, top, stats):
def sync_dir(src, dst, top, stats, keep_nm=False):
os.makedirs(dst, exist_ok=True)
want = set()
with os.scandir(src) as it:
for e in it:
if top and skip(e.name):
continue
if (top and skip(e.name)) or (e.name in EXCLUDE_ANY_DEPTH and not keep_nm):
continue # below the top it is not in `want`, so the pass below removes an old copy of it
want.add(e.name)
s, d = e.path, os.path.join(dst, e.name)
if e.is_dir(): # follows a symlinked folder: the build reads its content
if os.path.lexists(d) and not os.path.isdir(d):
os.remove(d)
sync_dir(s, d, False, stats)
# javascriptsource/ keeps its node_modules: JavaScript actions' npm dependencies, bundled by mxbuild
sync_dir(s, d, False, stats, keep_nm or (top and e.name == "javascriptsource"))
continue
st = e.stat()
try:
Expand Down
28 changes: 28 additions & 0 deletions tests/wave2/test-build-tree.sh
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@
# B3 two-tree (app/X.mpr): app/deployment/ unchanged, the copy is the app folder
# B4 the sync: a file deleted from the model leaves the copy; the copy's deployment/ survives a
# sync; .git, .mxcli, sources/, *.mpr.lock are not copied; .mxtk/ ignores itself
# B4c a nested node_modules (widgets/W/node_modules) is never copied, and a stale one is removed (#764);
# javascriptsource/**/node_modules (JS action deps, bundled by mxbuild) is kept
# B5 a compile error printed from the copy names the REAL file, not .mxtk/build/
# B6 MXTK_GATE_IN_PLACE=1 builds in place, as before, and says so
# B7 the pre-push hook prints the restart line only after an in-place build
Expand Down Expand Up @@ -127,6 +129,32 @@ python3 -I "$BT" "$P/Shop.mpr" "$P/.mxtk/build/model" >/dev/null 2>&1 # portabi
rm -f "$RO"
[ -z "$(cd "$P" && git status --porcelain -- .mxtk)" ] && ok "B4 git status shows nothing under .mxtk/" || bad "B4 .mxtk shows in git status"

echo "-- B4c a nested node_modules (a custom widget's sources) is never copied (#764)"
W="$WORK/widget"; mkdir -p "$W/widgets/W/node_modules/dep" "$W/widgets/W/src" "$W/widgets/W/docs"
printf 'not a real model\n' > "$W/Shop.mpr"; printf 'module.exports = 1;\n' > "$W/widgets/W/node_modules/big.js"
printf 'x\n' > "$W/widgets/W/node_modules/dep/index.js"; printf 'export const W = 1;\n' > "$W/widgets/W/src/W.tsx"
printf 'widget docs\n' > "$W/widgets/W/docs/README.md"
OUT="$(python3 -I "$BT" "$W/Shop.mpr" "$W/.mxtk/build/model" 2>&1)"; RC=$? # portability-ok: fixture runs the script directly
[ "$RC" -eq 0 ] && ok "B4c exit 0" || bad "B4c exit $RC: $OUT"
[ -f "$W/.mxtk/build/model/widgets/W/src/W.tsx" ] && ok "B4c the widget's src/W.tsx is copied" || bad "B4c src/W.tsx missing from the copy"
[ ! -e "$W/.mxtk/build/model/widgets/W/node_modules" ] && ok "B4c widgets/W/node_modules is not copied" || bad "B4c widgets/W/node_modules was copied"
[ -f "$W/.mxtk/build/model/widgets/W/docs/README.md" ] && ok "B4c a nested docs/ (top-level-only name) is still copied" || bad "B4c a nested docs/ was dropped"
# An older copy (synced before #764) already holds the nested node_modules: the next sync removes it.
mkdir -p "$W/.mxtk/build/model/widgets/W/node_modules/dep"; printf 'stale\n' > "$W/.mxtk/build/model/widgets/W/node_modules/big.js"
python3 -I "$BT" "$W/Shop.mpr" "$W/.mxtk/build/model" >/dev/null 2>&1 # portability-ok: fixture runs the script directly
[ ! -e "$W/.mxtk/build/model/widgets/W/node_modules" ] && ok "B4c a stale nested node_modules in the copy is removed" || bad "B4c a stale nested node_modules lingers in the copy"
[ -f "$W/.mxtk/build/model/widgets/W/src/W.tsx" ] && ok "B4c and the widget's sources stay" || bad "B4c the resync dropped src/W.tsx"
# A JavaScript action's npm dependencies live in javascriptsource/<m>/**/node_modules and mxbuild bundles
# them (review of #769): that one tree keeps its node_modules, on the first sync and on a resync.
mkdir -p "$W/javascriptsource/mymod/actions/node_modules/big-js"
printf 'module.exports = 2;\n' > "$W/javascriptsource/mymod/actions/node_modules/big-js/index.js"
python3 -I "$BT" "$W/Shop.mpr" "$W/.mxtk/build/model" >/dev/null 2>&1 # portability-ok: fixture runs the script directly
[ -f "$W/.mxtk/build/model/javascriptsource/mymod/actions/node_modules/big-js/index.js" ] \
&& ok "B4c javascriptsource/**/node_modules (JS action deps) is copied" || bad "B4c javascriptsource node_modules was dropped"
python3 -I "$BT" "$W/Shop.mpr" "$W/.mxtk/build/model" >/dev/null 2>&1 # portability-ok: fixture runs the script directly
[ -f "$W/.mxtk/build/model/javascriptsource/mymod/actions/node_modules/big-js/index.js" ] \
&& ok "B4c and survives a resync" || bad "B4c a resync removed javascriptsource node_modules"

echo "-- B5 a compile error from the copy names the real file"
OUT="$(vm "$P" FAKE_JAVA_ERR=1)"; RC=$?
[ "$RC" -eq 1 ] && ok "B5 exit 1 (errors)" || bad "B5 exit $RC"
Expand Down
Loading