Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m
`docs/RELEASE-PROCESS.md` §8.

## Unreleased
- fix(contract-check): **`check` and `--impact` now report inline `call rest service` activities per app instead of letting 0/0 read as "no consumers" (#674)**: a separate mxcli call (`refresh catalog full` + `catalog.activities` where `ActionType = 'RestCallAction'`; the 13-query call and every committed baseline are unchanged) prints `inline: N inline REST call(s) in <app> not checked (<microflows>)`, `inline: 0 inline REST calls in <app>`, or `not measured` when the call fails or the activities table is empty — never 0 by absence. Informational only: the `denominator:` line, verdicts and exit codes do not change; matching the call URL to a publisher is not done ("WHAT IT CANNOT SEE" says so). Captured with mxcli v0.28.0 from toolkit-built demo apps — field run, 2026-11
- fix(doctor,test-scratch): **a cached Mendix runtime that is incomplete is now named, with what it lacks and the repair**: mxcli accepts `~/.mxcli/runtime/<v>/runtime` on `launcher/runtimelauncher.jar` alone, so a cache without `bundles/org.osgi*.jar` killed `test --local` at startup (`NoClassDefFoundError org/osgi/...`) with no test verdict and a green mxbuild check above it; `bin/doctor.sh` and `test-scratch.sh` (before the build) now warn, also for the `mxbuild/<v>/runtime` link, point at a complete matching Studio Pro `Contents/runtime` on macOS, else name the download (`mxcli setup mxruntime -p <app>.mpr` for `~/.mxcli/runtime/<v>/`, `mxcli setup mxbuild -p <app>.mpr` for `~/.mxcli/mxbuild/<v>/`, per `setup --help`), and carry on (never blocks, exit codes unchanged, silent when complete; #692) — field run, 2026-11

## v2026.11.15
- fix(gitignore): **`bin/init-project.sh`, `bin/sync-project.sh` and `project-bin/model-stamp.sh` no longer add an entry a CRLF `.gitignore` already has, and append in the file's own line ending**: `grep -qE '^/?x/?$'` missed `x\r` (init-project re-appended `sources/`, `.mxtk/`, `.claude/settings.local.json` and the progress-report cache on every run), and sync-project/model-stamp wrote LF lines into a CRLF file (mixed endings). One helper, `mxtk_gitignore_ensure` (`bin/lib/gitignore-ensure.sh`, copy in `project-bin/_common.sh` for installed scripts); LF files get the same bytes as before; new `tests/wave2/test-gitignore-crlf.sh` — follow-up to the snapshot-mpr fix, 2026-11
Expand Down
11 changes: 8 additions & 3 deletions bin/contract-check.sh
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,10 @@
# mandatory chain step that writes it; with no registry this exits 2 and says so). The snapshots
# are produced by `snapshot` below and must be committed — `check` reads the committed copy.
#
# WHAT IT READS. Per app, two read-only mxcli calls (no builds; mxcli refreshes its own catalog
# cache .mxcli/ beside the .mpr): catalog tables for published OData and REST services and
# WHAT IT READS. Per app, three mxcli calls that never write the model (no builds; mxcli's catalog
# cache .mxcli/catalog.db beside the .mpr is built in fast mode by a plain SELECT, and the third call,
# the inline REST count, refreshes it to full mode). It takes NO lock: do not run it while an exec is
# writing the model. The first two calls read: catalog tables for published OData and REST services and
# operations, business event services and messages, consumed REST services and operations,
# consumed OData services and external entities, attributes, associations, generalizations and
# constants (a base URL held in a constant resolves to its default value); plus `describe odata
Expand Down Expand Up @@ -64,7 +66,10 @@
# (no registry, malformed row, no .mpr, no mxcli, mxcli failed, an app has no baseline).
# Exit (--impact): 0 answered (consumers or none) · 2 cannot run.
#
# WHAT IT CANNOT SEE: business event subscribers are snapshotted and diffed but not joined across
# WHAT IT CANNOT SEE: inline `call rest service` activities in a microflow are not a consumed REST
# service document, so they are never matched against a publisher and are not in the denominator;
# an `inline:` line per app counts them (microflow names) or says "not measured" (never 0) when
# the catalog's activities table cannot be read; business event subscribers are snapshotted and diffed but not joined across
# apps (no subscriber fixture existed); published OData microflow actions and other publish
# kinds are printed as UNPARSED, never silently dropped; external-entity attributes are matched
# by name (a remote name that differs from the local one reads as "not exposed"); REST request
Expand Down
11 changes: 11 additions & 0 deletions bin/doctor.sh
Original file line number Diff line number Diff line change
Expand Up @@ -1641,6 +1641,17 @@ else
note "like the container build. MXBUILD_PATH=<path to mxbuild> also works."
fi

# Cached runtime complete? (#692) mxcli takes a runtime on its launcher jar alone; an incomplete one
# kills `test --local` at startup with no verdict, and a green mxbuild above says nothing about it.
# Silent when every cached runtime is complete (or none is cached). Never changes GATE_OK.
if command -v mxtk_runtime_report >/dev/null 2>&1; then
_RV="$(mxtk_model_version 2>/dev/null || true)"
if ! _RR="$(mxtk_runtime_report "$_RV" "")"; then
warn "a cached Mendix runtime is incomplete — local tests can die at startup (#692)"
printf '%s\n' "$_RR" | while IFS= read -r l; do note "$l"; done
fi
fi

# Java: mxbuild is invoked with an explicit --java-exe-path; if none resolves, the gate skips.
if [ -n "$JAVA_EXE" ] && [ -x "$JAVA_EXE" ]; then
# -version writes to stderr, and wrappers can prepend noise (JAVA_TOOL_OPTIONS pickup
Expand Down
54 changes: 52 additions & 2 deletions bin/lib/contract-check.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,11 @@
"""contract-check.py — the engine behind bin/contract-check.sh. Read that file's header first;
this one documents only the internals.

Two mxcli calls per app, both read-only on the model (mxcli refreshes its own catalog cache,
.mxcli/catalog.db beside the .mpr — nothing else is written):
Three mxcli calls per app (the first two as listed here; the third is the inline-REST count in
INLINE_Q, run by inline_rest()), none writing the model. A plain SELECT builds mxcli's catalog cache
.mxcli/catalog.db beside the .mpr in fast mode; the third call's `refresh catalog full` rewrites it
in full mode. Nothing else is written. NO LOCK is taken: do not run this while an exec is writing
the model:

1. one `-c` string of SELECTs over the catalog tables listed in QUERIES, with --json. mxcli
prints one JSON array per statement on stdout (counts and warnings go to stderr), so the
Expand Down Expand Up @@ -142,6 +145,50 @@ def run_mxcli(mx, mpr, stmts):
return r.stdout, r.stderr


# Inline `call rest service` activities are not a consumed REST service document, so the 13 QUERIES
# above never see them. They are counted by a SEPARATE mxcli call (the 13-result-set check stays
# valid) and only REPORTED, never joined, never part of the denominator or an exit code. The
# catalog's activities table is empty until `refresh catalog full`, so the call refreshes first.
INLINE_Q = ("refresh catalog full; select count(*) as n from catalog.activities; "
"select MicroflowQualifiedName from catalog.activities where ActionType = 'RestCallAction' "
"order by MicroflowQualifiedName;")


def inline_rest(app):
"""[microflow, ...] (one entry per inline REST call), or None = not measured (no mxcli, call
failed, unexpected shape, or an empty activities table: an empty table is not a clean app)."""
try:
mpr = resolve_mpr(app["path"])
mx = find_mxcli(mpr) if mpr else None
if not mx:
return None
argv = (["bash", mx] if mx.endswith(".sh") else [mx])
r = subprocess.run(argv + ["-p", mpr, "--json", "-c", INLINE_Q], capture_output=True, text=True,
encoding="utf-8", errors="replace", timeout=600)
if r.returncode != 0:
return None
vals = json_values(r.stdout)
if len(vals) != 2 or int(vals[0][0]["n"]) == 0:
return None
return [row["MicroflowQualifiedName"] for row in vals[1]]
except (OSError, subprocess.TimeoutExpired, ValueError, KeyError, IndexError, TypeError):
return None


def print_inline(apps, skip=None):
for app in apps:
if app["name"] == skip:
continue
calls = inline_rest(app)
if calls is None:
print("inline: inline REST calls in %s: not measured (catalog.activities unavailable or empty)" % app["name"])
elif not calls:
print("inline: 0 inline REST calls in %s" % app["name"])
else:
print("inline: %d inline REST call(s) in %s not checked (%s)" % (
len(calls), app["name"], ", ".join(sorted(set(calls)))))


def norm_path(*parts):
segs = []
for p in parts:
Expand Down Expand Up @@ -624,6 +671,7 @@ def main():
if not tps:
print("NOT-EXPOSED %s:%s is in no published OData, REST or business event contract of %s" % (an, elem, an))
print("denominator: 0 contract members; 0 consumer apps")
print_inline(apps, an)
return 0
consumers = set()
for kind, svc, member, sub in tps:
Expand All @@ -645,6 +693,7 @@ def main():
print("NOTE business event subscribers are not joined across apps (not implemented: no subscriber fixture)")
print("denominator: %d contract member(s); %d consumer app(s) in a registry of %d: %s" % (
len(tps), len(consumers), len(apps), ", ".join(sorted(consumers)) or "none"))
print_inline(apps, an)
print("note: `mxcli impact` / `refs` do not report OData exposure (measured on v0.28.0); this list is the cross-app answer")
return 0

Expand Down Expand Up @@ -682,6 +731,7 @@ def main():
print("UNPARSED %s:%s" % (app["name"], u))
rows = join(apps, snaps)
print_join(rows)
print_inline(apps)
broken = len([r for r in rows if r[0] == "BROKEN"])
members = sum(len(e["members"]) for s in snaps.values() for o in s["published"]["odata"] for e in o["entity_sets"])
brk = len([o for o in out if o[0] == "BREAKING"])
Expand Down
88 changes: 88 additions & 0 deletions project-bin/_common.sh
Original file line number Diff line number Diff line change
Expand Up @@ -401,6 +401,94 @@ find_sp_app() {
fi
}

# ---------------------------------------------------------------------------
# mxtk_runtime_gaps <runtime-dir> — what a cached Mendix runtime lacks. Prints one missing item
# per line and returns 1 when anything is missing; prints nothing and returns 0 when complete.
#
# WHY (#692, field run 2026-11). `mxcli test --local` died with NoClassDefFoundError
# org/osgi/service/event/EventHandler: the cache under ~/.mxcli/runtime/<v>/runtime had
# launcher/runtimelauncher.jar (the only file mxcli checks) but no bundles/org.osgi*.jar.
# "Complete" here = the launcher jar, a bundles/ directory, and an org.osgi*.jar in it: the file
# the field run found missing plus what the launcher is started from. Golden layout:
# tests/wave2/fixtures/runtime-layout-11.13.0.txt (a real cached runtime). Stat calls only, no
# forks. This is NOT mxcli's validator, only the check the toolkit can make from outside.
# ---------------------------------------------------------------------------
mxtk_runtime_gaps() {
local d="$1" f found=0 rc=0
[ -f "$d/launcher/runtimelauncher.jar" ] || { echo "launcher/runtimelauncher.jar"; rc=1; }
if [ -d "$d/bundles" ]; then
for f in "$d"/bundles/org.osgi*.jar; do [ -e "$f" ] && { found=1; break; }; done
[ "$found" = 1 ] || { echo "bundles/org.osgi*.jar"; rc=1; }
else
echo "bundles/"; rc=1
fi
return $rc
}

# mxtk_runtime_problems [<mendix-version>] — which cached runtimes are incomplete. Looks at
# ~/.mxcli/runtime/<v>/runtime and ~/.mxcli/mxbuild/<v>/runtime (the link mxcli also resolves; it
# can point at the same deficient runtime). With a version only that one is checked, without
# every cached one. Prints "<dir>|<missing, comma separated>"; returns 1 if any. Complete or
# absent caches: no output, returns 0. Never blocks anything: callers warn.
mxtk_runtime_problems() {
local ver="${1:-}" root="${MXTK_MXCLI_HOME:-$HOME/.mxcli}" d gaps rc=0 kind
for kind in runtime mxbuild; do
for d in "$root/$kind"/${ver:-*}/runtime; do
[ -d "$d" ] || continue
gaps="$(mxtk_runtime_gaps "$d")" && continue
printf '%s|%s\n' "$d" "$(printf '%s' "$gaps" | tr '\n' ',' | sed 's/,$//; s/,/, /g')"
rc=1
done
done
return $rc
}

# mxtk_sp_runtime [<mendix-version>] — a COMPLETE Studio Pro runtime, if one is installed. macOS
# only: the issue names <Studio Pro app>/Contents/runtime; no Windows or Linux location is named
# anywhere, so those print nothing rather than a guessed path. With a version, only an app whose
# name carries it.
mxtk_sp_runtime() {
local ver="${1:-}" app
[ "${MXTK_PLATFORM:-$(mxtk_platform)}" = macos ] || return 1
while IFS= read -r app; do
[ -n "$app" ] || continue
case "$app" in *"$ver"*) ;; *) continue ;; esac
if [ -d "$app/Contents/runtime" ] && mxtk_runtime_gaps "$app/Contents/runtime" >/dev/null; then
printf '%s\n' "$app/Contents/runtime"; return 0
fi
done <<SPLIST
$(mxtk_sp_list 2>/dev/null)
SPLIST
return 1
}

# mxtk_runtime_report <mendix-version> <prefix> — print the message for mxtk_runtime_problems, each
# line starting with <prefix>; returns 1 when it printed, 0 (silent) when nothing is wrong. Names
# every incomplete runtime and what it lacks, the matching Studio Pro runtime when there is one,
# and the repair. It repairs nothing itself and the caller carries on.
mxtk_runtime_report() {
local ver="${1:-}" pre="$2" probs sp d g
probs="$(mxtk_runtime_problems "$ver")" && return 0
sp="$(mxtk_sp_runtime "$ver" || true)"
printf '%s\n' "$probs" | while IFS='|' read -r d g; do
printf '%s incomplete Mendix runtime: %s\n%s missing: %s\n' "$pre" "$d" "$pre" "$g"
done
printf '%s mxcli accepts a runtime on launcher/runtimelauncher.jar alone, so a local test run can die at\n' "$pre"
printf '%s startup (NoClassDefFoundError org/osgi/...) with no test verdict (#692). Repair, not done for you:\n' "$pre"
if [ -n "$sp" ]; then
printf '%s matching complete Studio Pro runtime: %s\n' "$pre" "$sp"
printf '%s mv <dir above> <dir above>.incomplete && ln -s "%s" <dir above>\n' "$pre" "$sp"
else
printf '%s\n' "$probs" | while IFS='|' read -r d g; do
case "$d" in
*/mxbuild/*) printf '%s rm -rf "%s", then: mxcli setup mxbuild -p <app>.mpr (MxBuild download; caches under ~/.mxcli/mxbuild/<v>/)\n' "$pre" "${d%/runtime}" ;;
*) printf '%s rm -rf "%s", then: mxcli setup mxruntime -p <app>.mpr (runtime download; caches under ~/.mxcli/runtime/<v>/)\n' "$pre" "${d%/runtime}" ;;
esac
done
fi
return 1
}

# mxtk_sp_list — every installed Studio Pro root, one per line, unsorted (find_sp_app
# picks the newest; find_mxbuild first looks for the one matching the model's version).
mxtk_sp_list() {
Expand Down
3 changes: 3 additions & 0 deletions project-bin/test-scratch.sh
Original file line number Diff line number Diff line change
Expand Up @@ -272,6 +272,9 @@ WARM=0
MX_VER=""
if [ -n "${MXTK_MODEL_VERSION:-}" ]; then MX_VER="$MXTK_MODEL_VERSION"
elif command -v mxtk_model_version >/dev/null 2>&1; then MX_VER="$(mxtk_model_version "$MPR" 2>/dev/null || true)"; fi
# An incomplete cached runtime (#692): warn now, before the expensive build, and carry on. mxcli accepts a
# runtime on its launcher jar alone; the run then dies at startup with no verdict. Output only when wrong.
if command -v mxtk_runtime_report >/dev/null 2>&1; then mxtk_runtime_report "$MX_VER" "!" >&2 || true; fi
# One stat call for all files: a fork per file (wc -c) cost 25 to 30 s on a 600-file javasource on macOS.
_libs_list() {
local _ds=() _d
Expand Down
14 changes: 14 additions & 0 deletions tests/wave2/fixtures/contract-check/CAPTURE.txt
Original file line number Diff line number Diff line change
Expand Up @@ -42,3 +42,17 @@ Field-run numbers on the untrimmed captures (same as the fixture asserts):
bump same BREAKING, VERSION 1.0.0 -> 2.0.0, 0 VIOLATION, still BROKEN shop GetCars; exit 1
--impact cars:MyFirstModule.VCar.Brand -> CONSUMER shop:MyFirstModule.ECar (mxcli impact: none)
Two mxcli calls per app; snapshot of both apps 0.9 s wall.

captures/inline/<state>/<app>.inline.json — the SEPARATE inline-REST call (issue #674), recorded
2026-11-15 with mxcli v0.28.0 from two toolkit-built demo apps of one family (each Feedback module has one inline
`call rest service post` microflow, FeedbackModule.SUB_Feedback_PostToAppInsights; describe shows
`$Response = call rest service post '{1}' with ({1} = $ServerLocation) (...)`). Stdout only, untrimmed
(a copy of the app's .mpr + mprcontents; the originals were not touched). The call is
refresh catalog full; select count(*) as n from catalog.activities;
select MicroflowQualifiedName from catalog.activities where ActionType = 'RestCallAction' order by MicroflowQualifiedName;
pos one app, unmodified: n=423, one RestCallAction row
neg the other app after `drop microflow FeedbackModule.SUB_Feedback_PostToAppInsights`
(mxcli exec: succeeded): n=514, rows []
empty the two SELECTs WITHOUT the refresh on a fresh copy (activities table not built): n=0, rows []
Which app a file is used for is the test's choice ($CC_INLINE_<app>=pos|neg|empty); an app with no
file reads "not measured". The 13 base/rename/drop/bump captures above are unchanged.
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
[
{
"n": "0"
}
]
[]
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
[
{
"n": "514"
}
]
[]
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
[
{
"n": "514"
}
]
[]
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
[
{
"n": "423"
}
]
[
{
"MicroflowQualifiedName": "FeedbackModule.SUB_Feedback_PostToAppInsights"
}
]
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
[
{
"n": "423"
}
]
[
{
"MicroflowQualifiedName": "FeedbackModule.SUB_Feedback_PostToAppInsights"
}
]
10 changes: 9 additions & 1 deletion tests/wave2/fixtures/contract-check/mxcli-replay.sh
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,21 @@ while [ $# -gt 0 ]; do
done
app="$(basename "$(dirname "$mpr")")"
[ "$app" = "app" ] && app="$(basename "$(dirname "$(dirname "$mpr")")")" # two-tree layout
case "$stmts" in describe*) kind=describe ;; *) kind=select ;; esac
case "$stmts" in describe*) kind=describe ;; refresh*) kind=inline ;; *) kind=select ;; esac
eval "state=\${CC_STATE_$app:-base}"
f="${CC_CAPTURES:?CC_CAPTURES not set}/$state/$app.$kind.json"
# the inline-REST call is a separate call with its own captures: captures/inline/<state>/<app>.inline.json,
# <state> from $CC_INLINE_<app>; unset = no capture = "not measured"
if [ "$kind" = inline ]; then
eval "ist=\${CC_INLINE_$app:-absent}"
f="$CC_CAPTURES/inline/$ist/$app.inline.json"
fi
if [ -n "${CC_RECORD:-}" ]; then
mkdir -p "$(dirname "$f")"
"$CC_RECORD" -p "$mpr" --json -c "$stmts" > "$f"; rc=$?
cat "$f"; exit $rc
fi
[ -f "$f" ] || { echo "mxcli-replay: no capture $f" >&2; exit 3; }
cat "$f"
# $CC_INLINE_FAIL_<app>=1: the inline call prints its capture, then exits non-zero (a failed mxcli)
if [ "$kind" = inline ]; then eval "fl=\${CC_INLINE_FAIL_$app:-}"; [ -z "$fl" ] || exit 1; fi
Loading
Loading