Repository navigation

JVM: keep NaN, inf and -inf in to_json_list and in nested values of the columnar readers #210

Workflow file for this run

.github/workflows/test-jvm.yml at f5a10f3

	# The JVM build: the `arcadedb-jvm` distribution. release-jvm.yml calls this workflow on a version tag and publishes the wheels it builds.
	# One wheel per platform, py3-none-<platform tag> (setup.py): the `build` job builds each once, and the `test` job installs that one file
	# into every Python (3.11 to 3.14) and runs the suite. The jars and the bundled JRE do not depend on the Python, so nothing is built per Python.
	# The native build, `arcadedb`, is test-native.yml / release-native.yml.
	name: Test JVM (arcadedb-jvm)

	on:
	push:
	branches: [main]
	paths-ignore:
	# Changes that cannot alter what these jobs build or run. README.md and
	# tests/README.md are deliberately NOT here: the wheel build copies README.md
	# (scripts/Dockerfile.build, pyproject `readme`) and test_no_monorepo_wording.py
	# reads both. docs/** is safe to skip while tests/conftest.py collect-ignores
	# test_docs_examples.py (it needs docs/index.md); if that Markdown site comes
	# back, take docs/** out of this list. Schedule, dispatch and workflow_call
	# ignore these filters and always run everything.
	- "docs/**"
	- "*/.adoc"
	- "CLAUDE.md"
	- "examples/README.md"
	- "LICENSE*"
	# the native build has its own workflows; nothing under native/ is in the JVM wheel
	- "native/**"
	- ".github/workflows/test-native.yml"
	- ".github/workflows/release-native.yml"
	pull_request:
	# ready_for_review: draft PRs skip the platform matrices (see the job conditions),
	# so marking one ready is what starts them.
	types: [opened, synchronize, reopened, ready_for_review]
	paths-ignore:
	# Same list as under push.
	- "docs/**"
	- "*/.adoc"
	- "CLAUDE.md"
	- "examples/README.md"
	- "LICENSE*"
	# the native build has its own workflows; nothing under native/ is in the JVM wheel
	- "native/**"
	- ".github/workflows/test-native.yml"
	- ".github/workflows/release-native.yml"

	# A new upstream snapshot is published as arcadedata/arcadedb:latest without
	# any change here, so build and test against it every night as well. The tag
	# moves (ARCADEDB_IMAGE_TAG defaults to latest), so a night with no commit
	# here still tests a new engine; that is why there is no skip-if-main-unchanged
	# guard. One nightly per workflow, staggered an hour apart.
	schedule:
	- cron: "0 3 * * *"

	# Allow being called by other workflows (release-jvm.yml)
	workflow_call:
	inputs:
	arcadedb-tag:
	description: "arcadedata/arcadedb image tag to take the jars from (e.g. 26.10.2); default latest"
	required: false
	type: string
	default: "latest"
	build-version:
	description: "Override package version (PEP 440) for build.sh"
	required: false
	type: string

	# Allow manual trigger
	workflow_dispatch:
	inputs:
	arcadedb-tag:
	description: "arcadedata/arcadedb image tag to take the jars from (e.g. 26.10.2); default latest"
	required: false
	default: "latest"

	# SUPERSEDE, do not race (same rule in test-python-examples.yml).
	#
	# A force-push to a PR branch left the OLD run going alongside the new one: the
	# stale run reports a failure that the newer commit already fixed, and it holds
	# macOS/Windows runners the organisation shares with ArcadeData/arcadedb.
	#
	# cancel-in-progress is scoped to pull_request ONLY. Every other event gets a
	# group of its own (run_id), so a push to main or a nightly run is never
	# cancelled and never replaced while queued behind another run. The group names
	# the workflow literally: github.workflow is the CALLER's name in a
	# workflow_call, which would put this workflow and its sibling in one group.
	concurrency:
	group: test-jvm-${{ github.event.pull_request.number \|\| github.run_id }}
	cancel-in-progress: ${{ github.event_name == 'pull_request' }}

	env:
	# The image download-jars takes the jars from. Unset (push, pull_request, schedule)
	# means latest; a release build passes its tag.
	ARCADEDB_IMAGE_TAG: ${{ inputs.arcadedb-tag \|\| 'latest' }}

	permissions:
	contents: read

	jobs:
	bandit:
	name: Bandit security scan
	runs-on: ubuntu-latest
	steps:
	- name: Checkout code
	uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

	- name: Set up Python
	uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0
	with:
	python-version: "3.12"

	# Bandit and pip-audit run through uvx (a throwaway environment each), so the jobs install nothing into the runner's Python.
	- name: Setup UV package manager
	shell: bash
	run: \|
	# Retry transient network failures (a connection reset failed a whole examples job on #8897); the
	# script is downloaded first so a retried transfer is never piped half-written into sh.
	installer="$(mktemp)"
	curl -LsSf --retry 5 --retry-all-errors --retry-delay 5 -o "$installer" https://astral.sh/uv/0.12.0/install.sh
	sh "$installer"
	echo "$HOME/.local/bin" >> "$GITHUB_PATH"
	export PATH="$HOME/.local/bin:$PATH"
	uv --version

	- name: Run Bandit on src and tests (must be clean)
	run: uvx bandit==1.9.4 -c pyproject.toml -r src tests --severity-level low --confidence-level low

	- name: Run Bandit on examples (must be clean at medium+/high-confidence)
	run: uvx bandit==1.9.4 -c pyproject.toml -r examples --severity-level medium --confidence-level high

	# Meterian cannot scan this folder: `uv lock` refuses on a project carrying
	# `[tool.uv] managed = false`, so it produces no lockfile to read (see #5616).
	# This audits the dependency floors declared in pyproject.toml instead.
	dependency-floors:
	name: Dependency floor audit
	runs-on: ubuntu-latest
	steps:
	- name: Checkout code
	uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

	- name: Set up Python
	uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0
	with:
	python-version: "3.12"

	# The uv version is pinned in every install URL of this repo. It matters most here:
	# this job's own output depends on how uv resolves, so a silent resolver change
	# would silently change what gets audited.
	- name: Setup UV package manager
	shell: bash
	run: \|
	# Retry transient network failures (a connection reset failed a whole examples job on #8897); the
	# script is downloaded first so a retried transfer is never piped half-written into sh.
	installer="$(mktemp)"
	curl -LsSf --retry 5 --retry-all-errors --retry-delay 5 -o "$installer" https://astral.sh/uv/0.12.0/install.sh
	sh "$installer"
	echo "$HOME/.local/bin" >> "$GITHUB_PATH"
	export PATH="$HOME/.local/bin:$PATH"
	uv --version

	# --resolution lowest-direct pins every declared dependency to the oldest
	# release its own constraint admits, which is the version this manifest
	# actually promises to work with. The default resolution would pick the
	# newest instead, and a floor left open to a vulnerable release would
	# never show up.
	#
	# The dev extra is excluded on purpose: black, isort, mypy and pytest-cov
	# declare no lower bound, so lowest-direct drops them onto their earliest
	# PyPI releases (mypy 0.1, which pulls Python 2-era sdists that fail to
	# build). It is contributor tooling and is never installed from the wheel.
	#
	# Every interpreter the package claims to support is resolved, because the
	# closure is interpreter-dependent even though the declared floors are not:
	# an older interpreter pulls backports (typing-extensions, for one) that a
	# newer one does not. --python-version selects the resolution target
	# without installing anything, so this stays one job rather than a matrix.
	# The list is read from the classifiers rather than repeated here, so adding
	# an interpreter to the package cannot leave it unaudited.
	#
	# `uv pip compile` is not used: the floors are resolved with `uv lock` on a throwaway project, one per
	# interpreter, that carries the package's dependencies and the audited extras and nothing else (pyproject.toml
	# itself stays a non-uv project, and `uv lock` would refuse it with `managed = false` and would also resolve
	# the dev extra). `requires-python = "==3.X.*"` makes the lock resolve for that interpreter only, which is what
	# `--python-version` did. `uv export` writes the pinned file; --disable-pip makes pip-audit read it directly
	# rather than asking pip to resolve it. Without it, a floor whose version has no wheel for the running
	# interpreter fails as an sdist build error instead of reporting the advisory that caused the regression.
	- name: Audit declared dependency floors on every supported interpreter
	run: \|
	# The trailing dot in the prefix is load-bearing: the classifiers also
	# carry a bare "Programming Language :: Python :: 3", which without it
	# yields version 3 and fails.
	versions=$(uv run --no-project python -c 'import tomllib;p="Programming Language :: Python :: 3.";print(" ".join(c.rsplit(" ",1)[1] for c in tomllib.load(open("pyproject.toml","rb"))["project"]["classifiers"] if c.startswith(p)))')
	# No classifiers would audit nothing and still pass, which is the kind
	# of silent gap this job exists to close.
	[ -n "$versions" ] \|\| { echo "no Python version classifiers found in pyproject.toml"; exit 1; }
	echo "auditing declared floors for: $versions"
	for v in $versions; do
	echo "::group::Resolve floors for Python $v"
	proj="$RUNNER_TEMP/floors-$v"
	mkdir -p "$proj"
	uv run --no-project python - "$v" "$proj/pyproject.toml" << 'PY'
	import sys
	import tomllib

	version, out = sys.argv[1], sys.argv[2]
	extras = ["test", "vector", "examples", "arrow", "pandas"]
	with open("pyproject.toml", "rb") as f:
	project = tomllib.load(f)["project"]
	deps = list(project["dependencies"])
	for extra in extras:
	deps += project["optional-dependencies"][extra]
	with open(out, "w") as f:
	f.write('[project]\nname = "floors"\nversion = "0"\n')
	f.write('requires-python = "==%s.*"\n' % version)
	f.write("dependencies = [\n")
	for d in deps:
	f.write(" %r,\n" % d)
	f.write("]\n")
	PY
	(cd "$proj" && uv lock --resolution lowest-direct && uv export --frozen --no-hashes --no-annotate --no-emit-project -o "$RUNNER_TEMP/floors-$v.txt")
	cat "$RUNNER_TEMP/floors-$v.txt"
	echo "::endgroup::"
	uvx pip-audit==2.10.1 --no-deps --disable-pip -r "$RUNNER_TEMP/floors-$v.txt"
	done

	# First job: Download ArcadeDB JARs (platform-agnostic)
	download-jars:
	name: Download ArcadeDB JARs
	runs-on: ubuntu-latest
	# Draft PRs skip this and the platform matrix; bandit and the dependency audit
	# above are cheap Linux jobs and keep running.
	if: github.event_name != 'pull_request' \|\| github.event.pull_request.draft == false
	outputs:
	arcadedb-tag: ${{ steps.jars.outputs.arcadedb-tag }}
	matrix: ${{ steps.matrix.outputs.matrix }}
	wheel-matrix: ${{ steps.matrix.outputs.wheel-matrix }}
	steps:
	- name: Checkout code
	uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

	# The platform x Python legs of this run (scripts/select_matrix.py): a small matrix on
	# pull requests, the full one on every other event. The organisation shares one runner
	# cap with ArcadeData/arcadedb, so a pull request does not run every combination.
	# The wheel matrix is the platforms of those legs, one build each: the wheel is
	# py3-none-<platform>, the same file for every Python.
	# Done first, so a failure here costs no Docker pull.
	- name: Select the test matrix
	id: matrix
	shell: bash
	env:
	EVENT_NAME: ${{ github.event_name }}
	run: \|
	matrix="$(python3 scripts/select_matrix.py jvm --event "$EVENT_NAME")"
	wheel_matrix="$(python3 scripts/select_matrix.py jvm --event "$EVENT_NAME" --wheels)"
	echo "$matrix"
	echo "$wheel_matrix"
	echo "matrix=$matrix" >> "$GITHUB_OUTPUT"
	echo "wheel-matrix=$wheel_matrix" >> "$GITHUB_OUTPUT"

	- name: Set up Docker Buildx
	uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1

	- name: Download JARs from ArcadeDB Docker image
	id: jars
	shell: bash
	run: \|
	# Download JARs from the official Docker image: ARCADEDB_IMAGE_TAG,
	# `latest` (upstream's current snapshot) unless the caller pinned a tag. Every build job, Linux included, builds
	# from these jars (the artifact below), not from its own pull: upstream
	# re-pushes the snapshot tag on every deploy, so a pull per job could put
	# two different engine builds under one version in a single run.
	echo "📦 Downloading JARs from arcadedata/arcadedb:${ARCADEDB_IMAGE_TAG}..."

	# Create output directory
	mkdir -p src/arcadedb/jars

	# Create a temporary container and copy JARs from it
	CONTAINER_ID=$(docker create "arcadedata/arcadedb:${ARCADEDB_IMAGE_TAG}")
	docker cp $CONTAINER_ID:/home/arcadedb/lib/. src/arcadedb/jars/
	docker rm $CONTAINER_ID

	# Verify JARs were downloaded
	ls -lh src/arcadedb/jars/
	JAR_COUNT=$(ls -1 src/arcadedb/jars/*.jar 2>/dev/null \| wc -l)
	echo "✅ Downloaded $JAR_COUNT JAR files"

	ARCADEDB_TAG=$(python3 scripts/extract_version.py src/arcadedb/jars --format=docker)
	echo "📌 ArcadeDB version: $ARCADEDB_TAG"
	# The build jobs pull arcadedata/arcadedb:<jar version> as their base image,
	# so a pinned tag must name that version exactly (26.10.2, not an alias).
	if [ "$ARCADEDB_IMAGE_TAG" != "latest" ] && [ "$ARCADEDB_IMAGE_TAG" != "$ARCADEDB_TAG" ]; then
	echo "❌ arcadedb-tag '$ARCADEDB_IMAGE_TAG' holds engine $ARCADEDB_TAG; pass the exact version tag"
	exit 1
	fi
	echo "arcadedb-tag=$ARCADEDB_TAG" >> "$GITHUB_OUTPUT"

	# Unfiltered on purpose: build-native.sh applies scripts/jar_exclusions.txt
	# to whatever lands in the jars directory, so the list lives in one place.
	- name: Upload JARs as artifact
	uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
	with:
	name: arcadedb-jars
	path: src/arcadedb/jars/*.jar
	retention-days: 1

	# Second job: build the wheel of each platform, once. The wheel is py3-none-<platform tag> (setup.py), so the file built here is
	# the one every Python of the test job below installs: nothing is built per Python. build.sh takes a Python only for the Linux
	# container that builds and smoke-tests the wheel; any of them gives the same file.
	build:
	name: Build arcadedb-jvm wheel (${{ matrix.platform }})
	runs-on: ${{ matrix.runs-on }}
	needs: download-jars
	if: github.event_name != 'pull_request' \|\| github.event.pull_request.draft == false
	env:
	BUILD_VERSION: ${{ inputs.build-version }}
	strategy:
	# Don't cancel other matrix jobs if one fails
	fail-fast: false
	# One leg per platform of the test matrix, from the "Select the test matrix" step of download-jars
	# (scripts/select_matrix.py --wheels): {"include": [{"platform": ..., "runs-on": ...}, ...]}.
	matrix: ${{ fromJSON(needs.download-jars.outputs.wheel-matrix) }}

	steps:
	- name: Checkout code
	uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

	- name: Set up Python
	uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0
	with:
	python-version: "3.12"

	- name: Setup UV package manager
	shell: bash
	run: \|
	# Retry transient network failures (a connection reset failed a whole examples job on #8897); the
	# script is downloaded first so a retried transfer is never piped half-written into sh.
	installer="$(mktemp)"
	curl -LsSf --retry 5 --retry-all-errors --retry-delay 5 -o "$installer" https://astral.sh/uv/0.12.0/install.sh
	sh "$installer"
	echo "$HOME/.local/bin" >> "$GITHUB_PATH"
	export PATH="$HOME/.local/bin:$PATH"
	uv --version

	- name: Download ArcadeDB JARs artifact
	# Every platform builds from this artifact, see download-jars.
	uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
	with:
	name: arcadedb-jars
	path: src/arcadedb/jars

	- name: Set up Java (for native builds on macOS/Windows)
	if: matrix.platform != 'linux/amd64' && matrix.platform != 'linux/arm64'
	uses: actions/setup-java@b843ebf9b913146b38da2eaa7aeddf76074ae1ad # v6.1.0
	with:
	distribution: 'corretto'
	java-version: '25'

	- name: Set up Docker Buildx (Linux only)
	if: matrix.platform == 'linux/amd64' \|\| matrix.platform == 'linux/arm64'
	uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1

	- name: Set UTF-8 encoding (Windows only)
	if: matrix.platform == 'windows/amd64' \|\| matrix.platform == 'windows/arm64'
	shell: bash
	run: \|
	echo "PYTHONIOENCODING=utf-8" >> $GITHUB_ENV
	echo "PYTHONUTF8=1" >> $GITHUB_ENV

	# Windows currently enabled, no symlink needed
	# - name: Create python3 symlink (Windows only)
	# if: matrix.platform == 'windows/amd64' \|\| matrix.platform == 'windows/arm64'
	# shell: bash
	# run: \|
	# PYTHON_DIR=$(dirname "$(which python)")
	# ln -s "$PYTHON_DIR/python.exe" "$PYTHON_DIR/python3.exe" \|\| true
	# python3 --version

	- name: Build arcadedb (${{ matrix.platform }})
	shell: bash
	env:
	# The engine version download-jars read from the ARCADEDB_IMAGE_TAG jars;
	# the Docker build takes its base image by this tag, the jars from the artifact
	ARCADEDB_TAG: ${{ needs.download-jars.outputs.arcadedb-tag }}
	run: \|
	echo "🔨 Building arcadedb-jvm for ${{ matrix.platform }}..."
	# The jars download-jars fetched, on every platform, so all wheels of this
	# run carry the same engine build (and build.sh checks the integration jar)
	./scripts/build.sh ${{ matrix.platform }} 3.12 src/arcadedb/jars

	# build.sh already ran this; it is repeated here so the tag shows in this job's log and summary, and so a change to build.sh
	# cannot drop it unnoticed. The wheel must be py3-none-<platform>, the file the test job installs on every Python.
	- name: Check the wheel tag
	shell: bash
	run: \|
	python scripts/verify_wheel_tag.py dist/*.whl
	{
	echo "## 📦 Wheel: arcadedb-jvm (${{ matrix.platform }})"
	echo
	for w in dist/*.whl; do
	echo "- \`$(basename "$w")\`, $(du -m "$w" \| cut -f1) MB"
	done
	} >> "$GITHUB_STEP_SUMMARY"

	# One wheel per platform: the test job of the platform installs this file on every Python, and the release workflow
	# publishes it (verify-wheels downloads wheel-<platform>).
	- name: Upload wheel artifact
	uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
	with:
	name: wheel-${{ matrix.platform == 'linux/amd64' && 'linux-amd64' \|\| matrix.platform == 'linux/arm64' && 'linux-arm64' \|\| matrix.platform == 'darwin/amd64' && 'darwin-amd64' \|\| matrix.platform == 'darwin/arm64' && 'darwin-arm64' \|\| matrix.platform == 'windows/amd64' && 'windows-amd64' \|\| 'windows-arm64' }}
	path: dist/*.whl
	if-no-files-found: error
	retention-days: 7

	# Third job: install the wheel of the platform into each Python and run the suite. Nothing is built here: the wheel is the file the
	# build job made (py3-none-<platform tag>), so each Python is tested against the same file that is published.
	test:
	name: Test arcadedb (${{ matrix.platform }}, Python ${{ matrix.python-version }})
	runs-on: ${{ matrix.runs-on }}
	needs: [download-jars, build]
	if: github.event_name != 'pull_request' \|\| github.event.pull_request.draft == false
	strategy:
	# Don't cancel other matrix jobs if one fails
	fail-fast: false
	# Chosen by the "Select the test matrix" step of download-jars from the event that
	# started the run (scripts/select_matrix.py holds the platform and Python lists):
	# small on pull requests, the full 4 platforms x 4 Pythons on every other event.
	# Every leg carries platform, runs-on and python-version, so the job names stay
	# "Test arcadedb (<platform>, Python <version>)".
	matrix: ${{ fromJSON(needs.download-jars.outputs.matrix) }}

	steps:
	- name: Checkout code
	uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

	- name: Set up Python
	uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0
	with:
	python-version: ${{ matrix.python-version }}

	- name: Setup UV package manager
	shell: bash
	run: \|
	# Retry transient network failures (a connection reset failed a whole examples job on #8897); the
	# script is downloaded first so a retried transfer is never piped half-written into sh.
	installer="$(mktemp)"
	curl -LsSf --retry 5 --retry-all-errors --retry-delay 5 -o "$installer" https://astral.sh/uv/0.12.0/install.sh
	sh "$installer"
	echo "$HOME/.local/bin" >> "$GITHUB_PATH"
	export PATH="$HOME/.local/bin:$PATH"
	uv --version

	# The wheel of this platform, built once by the build job and the same file for every Python.
	- name: Download the wheel of ${{ matrix.platform }}
	uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
	with:
	name: wheel-${{ matrix.platform == 'linux/amd64' && 'linux-amd64' \|\| matrix.platform == 'linux/arm64' && 'linux-arm64' \|\| matrix.platform == 'darwin/amd64' && 'darwin-amd64' \|\| matrix.platform == 'darwin/arm64' && 'darwin-arm64' \|\| matrix.platform == 'windows/amd64' && 'windows-amd64' \|\| 'windows-arm64' }}
	path: test-install

	- name: Set UTF-8 encoding (Windows only)
	if: matrix.platform == 'windows/amd64' \|\| matrix.platform == 'windows/arm64'
	shell: bash
	run: \|
	echo "PYTHONIOENCODING=utf-8" >> $GITHUB_ENV
	echo "PYTHONUTF8=1" >> $GITHUB_ENV

	# The tests below run on the runner (glibc 2.39). This step runs the wheel where its tag says it must
	# work: a manylinux_2_28 container (glibc 2.28, AlmaLinux 8, the oldest system the Linux wheels are tagged for).
	# uv (like pip) refuses a wheel tagged above 2.28 there, so a JRE that raises the floor again fails here as well as in
	# scripts/verify_wheel_platform_tag.py. The wheel's own JRE does the work: no Java in the container.
	- name: Install and run the wheel in manylinux_2_28 (Linux only)
	if: startsWith(matrix.platform, 'linux/')
	shell: bash
	env:
	PLATFORM: ${{ matrix.platform }}
	PYVER: ${{ matrix.python-version }}
	run: \|
	case "$PLATFORM" in
	linux/amd64) ARCH=x86_64 ;;
	linux/arm64) ARCH=aarch64 ;;
	*) echo "unexpected platform $PLATFORM" >&2; exit 1 ;;
	esac
	PYTAG="cp${PYVER//./}-cp${PYVER//./}"
	ls -l test-install/*.whl
	# The runner's uv is mounted into the container (a glibc 2.17 build); the image's own Python is used, never downloaded.
	docker run --rm -i -e PYTAG="$PYTAG" -e UV_PYTHON_DOWNLOADS=never -v "$PWD/test-install:/wheels:ro" \
	-v "$(command -v uv):/usr/local/bin/uv:ro" \
	"quay.io/pypa/manylinux_2_28_${ARCH}:latest" bash -es <<'CONTAINER'
	ldd --version \| head -n 1
	test -x "/opt/python/$PYTAG/bin/python" \|\| { echo "no $PYTAG in this image" >&2; ls /opt/python >&2; exit 1; }
	with=()
	for w in /wheels/*.whl; do with+=(--with "$w"); done
	cd /tmp
	# --no-build: a source build of JPype must not hide a missing wheel
	uv run --no-project --no-build --quiet --python "/opt/python/$PYTAG/bin/python" "${with[@]}" python - <<'PYSMOKE'
	import tempfile
	import arcadedb
	with tempfile.TemporaryDirectory() as tmp:
	with arcadedb.create_database(tmp + "/smoke") as db:
	rows = db.query("sql", "SELECT 1 AS one").to_list()
	assert rows and rows[0]["one"] == 1, rows
	print("manylinux_2_28 smoke OK:", arcadedb.__version__, rows)
	PYSMOKE
	CONTAINER

	# Note: Java is NOT required for arcadedb (JRE is bundled)
	# This package works without any external Java installation!

	- name: Run pytest on host
	id: pytest
	# A HUNG TEST ENDS THE STEP, NOT THE DAY. On 2026-09-25 the windows/3.13
	# job sat in test_lsm_vector_quantization_int8_comprehensive for 86 minutes
	# (re-run: passed in 7), and with no limit it would have held the run for
	# GitHub's 6-hour default. The step takes 108-319 s on every platform.
	timeout-minutes: 30
	shell: bash
	env:
	PYVER: ${{ matrix.python-version }}
	run: \|
	echo "🧪 Testing arcadedb (with bundled JRE)..."

	# The wheel and its `test` extra go into a throwaway `uv run` environment, so nothing is installed into the
	# runner's Python. The extra mirrors what this job once listed by hand (it had drifted: pyarrow and pandas were
	# missing, so to_arrow() and to_dataframe() tests skipped themselves while the job reported green; setuptools is
	# in it for tests/test_wheel_packages.py and tests/test_wheel_tags.py, which run setup.py on a copy of the
	# sources). The guard below fails the build if any test skips for a missing import, so a drift is loud.
	wheels=(test-install/*.whl)
	[ "${#wheels[@]}" -eq 1 ] \|\| { echo "expected one wheel in test-install, found ${#wheels[@]}" >&2; exit 1; }
	UVRUN=(uv run --no-project --python "$PYVER" --with "${wheels[0]}[test]")

	# Run pytest with JUnit XML output for structured results
	# pytest runs with its default fd capture on every platform. Windows used to run with
	# --capture=sys (humemai/arcadedb-embedded-python#10): the default capture swaps fds 1
	# and 2 with dup2 around every test, which on Windows closed the OS handle the JVM had
	# cached for System.out/err, and the next Java log line blocked forever in WriteFile
	# (caught 2026-09-29, LSMVectorIndex.build's progress log). The package no longer logs
	# to the console on Windows by default (ARCADEDB_LOG_CONSOLE, src/arcadedb/_logdir.py),
	# which removes the cause; tests/test_log_redirect.py redirects fd 1 and 2 on purpose.
	set +e # Don't exit on test failures
	"${UVRUN[@]}" python -m pytest tests/ -v --tb=short --color=yes --junitxml=test-results.xml
	PYTEST_EXIT_CODE=$?
	set -e

	# Parse JUnit XML for test counts (cross-platform XML parsing)
	if [ -f "test-results.xml" ]; then
	# Extract counts from testsuite element
	TOTAL=$(grep -oE 'tests="[0-9]+"' test-results.xml \| grep -oE '[0-9]+' \| head -1)
	FAILED=$(grep -oE 'failures="[0-9]+"' test-results.xml \| grep -oE '[0-9]+' \| head -1)
	SKIPPED=$(grep -oE 'skipped="[0-9]+"' test-results.xml \| grep -oE '[0-9]+' \| head -1)
	ERRORS=$(grep -oE 'errors="[0-9]+"' test-results.xml \| grep -oE '[0-9]+' \| head -1)
	# JUnit counts an expected failure (a strict xfail: an open upstream bug the suite tracks)
	# as skipped, so a run with 14 of them read "14 skipped". Count them apart from real skips.
	# grep exits 1 when nothing matches, and this shell runs with pipefail and -e: a suite with no expected failures
	# (every upstream bug fixed, no strict xfail left) used to abort the step here, silently, after all tests passed.
	XFAILED=$({ grep -oE '<skipped[^>]*type="pytest.xfail"' test-results.xml \|\| true; } \| wc -l \| tr -d ' ')

	# Default to 0 if not found
	TOTAL=${TOTAL:-0}
	FAILED=${FAILED:-0}
	SKIPPED=${SKIPPED:-0}
	ERRORS=${ERRORS:-0}
	XFAILED=${XFAILED:-0}

	# Calculate passed tests (JUnit's skipped total includes the expected failures)
	PASSED=$((TOTAL - FAILED - SKIPPED - ERRORS))
	SKIPPED=$((SKIPPED - XFAILED))

	echo "passed=$PASSED" >> $GITHUB_OUTPUT
	echo "skipped=$SKIPPED" >> $GITHUB_OUTPUT
	echo "xfailed=$XFAILED" >> $GITHUB_OUTPUT
	echo "failed=$FAILED" >> $GITHUB_OUTPUT

	echo "✅ Test results: $PASSED passed, $XFAILED expected failures (xfail), $SKIPPED skipped, $FAILED failed"

	# A test that skips because a dependency is missing is not a
	# passing test, and it does not look like a failure either: the
	# suite goes green having exercised nothing.
	#
	# This has now happened three times. pyarrow was dropped by an
	# install command and the to_arrow() tests skipped; pandas was
	# never in the `test` extra at all, so to_dataframe() had no
	# executed coverage in any CI run this package has ever had; and
	# the hand-written dependency list in this very job was missing
	# both. Each time the signal was a green build.
	#
	# So make it loud. importorskip failures name the module, and a
	# missing dependency is a CI bug rather than an expected condition.
	IMPORT_SKIPS=$(grep -oE 'could not import .[a-zA-Z0-9_.]+.' test-results.xml \| sort -u \|\| true)
	if [ -n "$IMPORT_SKIPS" ]; then
	echo "❌ Tests skipped because dependencies are missing:"
	echo "$IMPORT_SKIPS" \| sed 's/^/ /'
	echo ""
	echo " These tests did not run. Add the module to the 'test' extra in"
	echo " pyproject.toml AND to the install step above,"
	echo " or delete the test if the integration is no longer supported."
	exit 1
	fi

	# Any other skip must be one somebody accepted (scripts/check_test_skips.py
	# names each, with the platform it applies to and why). A skip because the
	# runtime "lacks" a feature the wheel always ships is how a wrong empty answer
	# once passed as a skip; the import grep above is the special case.
	python scripts/check_test_skips.py test-results.xml --platform "${{ matrix.platform }}" \|\| exit 1

	# Force failure if there are failures or errors
	if [ "$FAILED" -gt 0 ] \|\| [ "$ERRORS" -gt 0 ]; then
	echo "❌ Tests failed (failures=$FAILED, errors=$ERRORS)"
	exit 1
	fi
	else
	echo "⚠️ test-results.xml not found"
	exit 1
	fi

	# Exit with pytest's exit code
	exit $PYTEST_EXIT_CODE

	- name: Generate test summary
	if: always()
	shell: python
	run: \|
	import os
	import zipfile
	import glob
	from pathlib import Path

	summary_file = os.environ.get('GITHUB_STEP_SUMMARY', '/dev/null')

	with open(summary_file, 'a', encoding='utf-8') as f:
	f.write("## 🧪 Test Results: arcadedb (${{ matrix.platform }})\n")
	f.write("\n")

	# Check test status
	if "${{ steps.pytest.outcome }}" == "success":
	f.write("✅ Status: PASSED\n")
	else:
	f.write("❌ Status: FAILED\n")

	f.write("\n")
	f.write("\| Metric \| Count \|\n")
	f.write("\|--------\|-------\|\n")

	# Show test results
	f.write("\| ✅ Passed \| ${{ steps.pytest.outputs.passed \|\| 'N/A' }} \|\n")
	f.write("\| ⏭️ Skipped \| ${{ steps.pytest.outputs.skipped \|\| 'N/A' }} \|\n")
	f.write("\| 🔖 Expected failures (xfail) \| ${{ steps.pytest.outputs.xfailed \|\| 'N/A' }} \|\n")
	f.write("\| ❌ Failed \| ${{ steps.pytest.outputs.failed \|\| 'N/A' }} \|\n")
	f.write("\n")

	f.write("### 📦 Package Info:\n")
	f.write("\n")

	# Get wheel size and analyze contents
	wheel_files = glob.glob("test-install/*.whl")
	if wheel_files:
	wheel_file = wheel_files[0]

	# Get wheel size
	wheel_size_bytes = os.path.getsize(wheel_file)
	wheel_size_mb = wheel_size_bytes / 1024 / 1024

	# Analyze wheel contents
	jre_size_mb = "N/A"
	jar_size_mb = "N/A"
	installed_size_mb = "N/A"

	try:
	# Extract and analyze (cross-platform)
	import tempfile
	import shutil

	with tempfile.TemporaryDirectory() as temp_dir:
	with zipfile.ZipFile(wheel_file, 'r') as whl:
	whl.extractall(temp_dir)

	# Calculate JRE size
	jre_dir = None
	for root, dirs, files in os.walk(temp_dir):
	if 'jre' in dirs:
	jre_dir = os.path.join(root, 'jre')
	break

	if jre_dir and os.path.isdir(jre_dir):
	jre_size_kb = 0
	for root, dirs, files in os.walk(jre_dir):
	for file in files:
	jre_size_kb += os.path.getsize(os.path.join(root, file))
	jre_size_mb = jre_size_kb / 1024 / 1024

	# Calculate JAR size
	jar_files = []
	for root, dirs, files in os.walk(temp_dir):
	for file in files:
	if file.endswith('.jar'):
	jar_files.append(os.path.join(root, file))

	if jar_files:
	jar_size_kb = sum(os.path.getsize(f) for f in jar_files) / 1024
	jar_size_mb = jar_size_kb / 1024

	# Calculate total size
	total_size_kb = 0
	for root, dirs, files in os.walk(temp_dir):
	for file in files:
	total_size_kb += os.path.getsize(os.path.join(root, file))
	installed_size_mb = total_size_kb / 1024 / 1024
	except Exception as e:
	print(f"Warning: Could not analyze wheel contents: {e}")

	f.write(f"- Wheel Size: {wheel_size_mb:.1f}M (compressed)\n")
	if isinstance(jre_size_mb, str):
	f.write(f"- JRE Size: {jre_size_mb} (uncompressed)\n")
	else:
	f.write(f"- JRE Size: {jre_size_mb:.1f}M (uncompressed)\n")
	if isinstance(jar_size_mb, str):
	f.write(f"- JARs Size: {jar_size_mb} (uncompressed)\n")
	else:
	f.write(f"- JARs Size: {jar_size_mb:.1f}M (uncompressed)\n")
	if isinstance(installed_size_mb, str):
	f.write(f"- Installed Size: {installed_size_mb} (total uncompressed)\n")
	else:
	f.write(f"- Installed Size: ~{installed_size_mb:.0f}M (total uncompressed)\n")

	f.write("- Platform: ${{ matrix.platform }}\n")
	f.write("- Features: All ArcadeDB features (some JARs excluded, see jar_exclusions.txt)\n")

	- name: Upload test results
	if: always()
	uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
	with:
	name: test-results-${{ matrix.platform == 'linux/amd64' && 'linux-amd64' \|\| matrix.platform == 'linux/arm64' && 'linux-arm64' \|\| matrix.platform == 'darwin/amd64' && 'darwin-amd64' \|\| matrix.platform == 'darwin/arm64' && 'darwin-arm64' \|\| matrix.platform == 'windows/amd64' && 'windows-amd64' \|\| 'windows-arm64' }}-py${{ matrix.python-version }}
	path: \|
	pytest-output.txt
	.coverage
	retention-days: 7

	# Summary job that checks all platforms
	test-summary:
	name: Test Summary
	needs: [build, test]
	runs-on: ubuntu-24.04
	# Skipped (not red) when the matrix was skipped for a draft PR.
	if: always() && (github.event_name != 'pull_request' \|\| github.event.pull_request.draft == false)
	steps:
	- name: Check test results
	run: \|
	echo "## 🎯 Overall Test Summary" >> $GITHUB_STEP_SUMMARY
	echo "" >> $GITHUB_STEP_SUMMARY

	if [ "${{ needs.build.result }}" = "success" ] && [ "${{ needs.test.result }}" = "success" ]; then
	echo "✅ All platforms passed testing!" >> $GITHUB_STEP_SUMMARY
	echo "" >> $GITHUB_STEP_SUMMARY
	echo "The arcadedb package has been successfully built (one py3-none wheel for each platform) and tested." >> $GITHUB_STEP_SUMMARY
	echo "" >> $GITHUB_STEP_SUMMARY
	echo "Package: arcadedb" >> $GITHUB_STEP_SUMMARY
	echo "" >> $GITHUB_STEP_SUMMARY
	echo "ℹ️ Note: Some platform/Python combinations are excluded from testing:" >> $GITHUB_STEP_SUMMARY
	echo "- Windows ARM64 (GitHub-hosted windows-11-arm runners exist, but jpype1 1.7.1 ships no win_arm64 wheel and GraalVM CE 25.4.4.1.1 has no Windows ARM64 build, so neither package can be built there)" >> $GITHUB_STEP_SUMMARY
	echo "- macOS x86_64 (temporarily disabled)" >> $GITHUB_STEP_SUMMARY
	else
	echo "❌ Some platforms failed testing" >> $GITHUB_STEP_SUMMARY
	echo "" >> $GITHUB_STEP_SUMMARY
	echo "Please check the individual test jobs for details." >> $GITHUB_STEP_SUMMARY
	exit 1
	fi

Navigation Menu

Repository navigation

JVM: keep NaN, inf and -inf in to_json_list and in nested values of the columnar readers #210

Workflow file

JVM: keep NaN, inf and -inf in to_json_list and in nested values of the columnar readers #210

Uh oh!

Workflow file for this run