Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/test-python-bindings.yml
Original file line number Diff line number Diff line change
Expand Up @@ -148,8 +148,8 @@ jobs:
#
# Every interpreter the package claims to support is resolved, because the
# closure is interpreter-dependent even though the declared floors are not:
# 3.10 pulls exceptiongroup, tomli and typing-extensions that 3.12 does not,
# and 3.14 drops one more. --python-version selects the resolution target
# an older interpreter pulls backports (typing-extensions, for one) that a
# newer one does not. --python-version selects the resolution target
# without installing anything, so this stays one job rather than a matrix.
# The list is read from the classifiers rather than repeated here, so adding
# an interpreter to the package cannot leave it unaudited.
Expand Down Expand Up @@ -267,7 +267,7 @@ jobs:
fail-fast: false
# Chosen by the "Select the test matrix" step of download-jars from the event that
# started the run (scripts/select_matrix.py holds the platform and Python lists):
# small on pull requests, the full 4 platforms x 5 Pythons on every other event.
# small on pull requests, the full 4 platforms x 4 Pythons on every other event.
# Every leg carries platform, runs-on and python-version, so the job names stay
# "Test arcadedb (<platform>, Python <version>)".
matrix: ${{ fromJSON(needs.download-jars.outputs.matrix) }}
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/test-python-examples.yml
Original file line number Diff line number Diff line change
Expand Up @@ -178,15 +178,15 @@ jobs:
BUILD_VERSION: ${{ inputs.build-version }}
# Pin the Hugging Face cache inside the workspace so it is cacheable with
# one path on every platform (the default differs per OS). The examples
# embed with sentence-transformers, and the full matrix is 20 jobs, so
# uncached that is 20 independent model downloads per release and 20
# embed with sentence-transformers, and the full matrix is 16 jobs, so
# uncached that is 16 independent model downloads per release and 16
# chances for a transient hub outage to fail a release. It has happened.
HF_HOME: ${{ github.workspace }}/.hf-cache
strategy:
fail-fast: false
# Chosen by the "Select the test matrix" step of download-jars from the event that
# started the run (scripts/select_matrix.py holds the platform and Python lists):
# small on pull requests, the full 4 platforms x 5 Pythons on every other event.
# small on pull requests, the full 4 platforms x 4 Pythons on every other event.
# Every leg carries platform, runs-on and python-version, so the job names stay
# "Test Python Examples (<platform>, Python <version>)".
matrix: ${{ fromJSON(needs.download-jars.outputs.matrix) }}
Expand Down
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,6 @@ python -m bandit -c pyproject.toml -r examples --severity-level medium --confide

## CI

`.github/workflows/test-python-bindings.yml` builds wheels for linux/amd64, linux/arm64, darwin/arm64, windows/amd64 and tests Python 3.10–3.14; also runs nightly against `arcadedata/arcadedb:latest`. The org shares a runner cap with upstream, so a pull request runs a small matrix (bindings: linux/amd64 on all five Pythons, the other three platforms on 3.10 and 3.14 = 11 legs; examples: all four platforms on 3.14 = 4 legs, 21 checks in all), while push to main, the nightly run, dispatch and the release run all 20 legs of each workflow (46 checks). `scripts/select_matrix.py` holds both lists (`PULL_REQUEST_LEGS`), the first job of each workflow runs it, and `tests/test_select_matrix.py` pins the full matrix; docs page `development/ci.adoc`. Keep the job names `Test arcadedb (<platform>, Python <version>)` unchanged. Both workflows accept an `arcadedb-tag` input to pin the image. `test-python-examples.yml` runs `examples/`. A dependency-floor audit resolves extras with `--resolution lowest-direct` for each Python classifier and runs pip-audit, so raise lower bounds when a floor has an advisory.
`.github/workflows/test-python-bindings.yml` builds wheels for linux/amd64, linux/arm64, darwin/arm64, windows/amd64 and tests Python 3.11–3.14; also runs nightly against `arcadedata/arcadedb:latest`. The org shares a runner cap with upstream, so a pull request runs a small matrix (bindings: linux/amd64 on all four Pythons, the other three platforms on 3.11 and 3.14 = 10 legs; examples: all four platforms on 3.14 = 4 legs, 20 checks in all), while push to main, the nightly run, dispatch and the release run all 16 legs of each workflow (38 checks). `scripts/select_matrix.py` holds both lists (`PULL_REQUEST_LEGS`), the first job of each workflow runs it, and `tests/test_select_matrix.py` pins the full matrix; docs page `development/ci.adoc`. Keep the job names `Test arcadedb (<platform>, Python <version>)` unchanged. Both workflows accept an `arcadedb-tag` input to pin the image. `test-python-examples.yml` runs `examples/`. A dependency-floor audit resolves extras with `--resolution lowest-direct` for each Python classifier and runs pip-audit, so raise lower bounds when a floor has an advisory.

`.github/workflows/release.yml` publishes on a bare-version tag (`26.11.1.dev0`, `26.11.1`, `26.11.1.post1`; no `v`, never `aN`/`bN`/`rcN`). Its first job runs `scripts/check_version.py` (the devN/postN counting rule, read-only, fails closed); `scripts/after_tag_verify.sh` checks the finished release. The rule is stated once, on the docs page `development/release.adoc`.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ pip install arcadedb # or: uv add arcadedb

This package was published as arcadedb-embedded until 26.10.x; install arcadedb now.

- Python 3.10 to 3.14. CI builds and tests all five versions on every platform below for `main`, every night and each release. A pull request runs a smaller selection.
- Python 3.11 to 3.14. CI builds and tests all four versions on every platform below for `main`, every night and each release. A pull request runs a smaller selection.
- Linux x86_64 and ARM64, macOS on Apple Silicon, Windows x86_64. One wheel per platform and Python version.
- Import name: `arcadedb`. Optional extras: `arrow`, `pandas`, `vector`.

Expand Down
12 changes: 6 additions & 6 deletions docs/modules/ROOT/pages/development/ci.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

Two workflows test the package: `test-python-bindings.yml` builds a wheel and runs the unit tests, and `test-python-examples.yml` runs the scripts in `examples/`. Both build the wheels on a native runner of each platform.

The package supports four platforms and five Python versions (3.10 to 3.14). That is 20 combinations for each workflow. A pull request does not run all of them, because the organisation shares one runner limit with ArcadeData/arcadedb. Each macOS or Windows job on a pull request delays the upstream jobs too.
The package supports four platforms and four Python versions (3.11 to 3.14). That is 16 combinations for each workflow. A pull request does not run all of them, because the organisation shares one runner limit with ArcadeData/arcadedb. Each macOS or Windows job on a pull request delays the upstream jobs too.

== What runs when

Expand All @@ -14,15 +14,15 @@ The script `scripts/select_matrix.py` chooses the combinations. The first job of
| Event | Unit tests (`Test arcadedb`) | Examples (`Test Python Examples`)

| Pull request
| `linux/amd64` on all five Python versions. `linux/arm64`, `darwin/arm64` and `windows/amd64` on Python 3.10 and 3.14.
| `linux/amd64` on all four Python versions. `linux/arm64`, `darwin/arm64` and `windows/amd64` on Python 3.11 and 3.14.
| All four platforms on Python 3.14.

| Push to `main`, the nightly run, a manual run, a release
| All 20 combinations.
| All 20 combinations.
| All 16 combinations.
| All 16 combinations.
|===

A pull request runs 21 checks: 15 test jobs (11 and 4), the two `Download ArcadeDB JARs` jobs, the two summary jobs, `Bandit security scan` and `Dependency floor audit`. The full matrix runs 46.
A pull request runs 20 checks: 14 test jobs (10 and 4), the two `Download ArcadeDB JARs` jobs, the two summary jobs, `Bandit security scan` and `Dependency floor audit`. The full matrix runs 38.

The names of the test jobs do not change: `Test arcadedb (<platform>, Python <version>)`. A check keeps its history when a pull request runs fewer of them.

Expand All @@ -36,4 +36,4 @@ A release always runs the full matrix, because `release.yml` expects one wheel f

== Change the selection

Edit `PULL_REQUEST_LEGS` in `scripts/select_matrix.py`. The test `tests/test_select_matrix.py` fixes the full matrix at 20 combinations, so a change that shrinks it fails. When you add a Python version, add it to `PYTHONS` in the script and to the classifiers in `pyproject.toml`. A test checks that both lists are the same.
Edit `PULL_REQUEST_LEGS` in `scripts/select_matrix.py`. The test `tests/test_select_matrix.py` fixes the full matrix at 16 combinations, so a change that shrinks it fails. When you add a Python version, add it to `PYTHONS` in the script and to the classifiers in `pyproject.toml`. A test checks that both lists are the same, and that `requires-python` is the oldest of them. When you drop one, remove it from the same places and update the counts in the test.
2 changes: 1 addition & 1 deletion docs/modules/ROOT/pages/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ Both packages import as `{import-name}`, so an environment holds one of them. Th

== Requirements

* Python 3.10 to 3.14. The wheels' CI runs all five versions on every supported platform for `main`, the nightly run and each release. A pull request runs a smaller selection (xref:development/ci.adoc[Continuous integration]).
* Python 3.11 to 3.14. The wheels' CI runs all four versions on every supported platform for `main`, the nightly run and each release. A pull request runs a smaller selection (xref:development/ci.adoc[Continuous integration]).
* Linux (x86_64 and ARM64), macOS on Apple Silicon, or Windows (x86_64).
* No Java installation.

Expand Down
2 changes: 1 addition & 1 deletion docs/modules/ROOT/pages/installation.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ The package is self-contained. It carries a Java 25 runtime for its platform, so

== Requirements

* Python 3.10 to 3.14. The wheels' CI runs all five versions on every supported platform for `main`, the nightly run and each release. A pull request runs a smaller selection (xref:development/ci.adoc[Continuous integration]).
* Python 3.11 to 3.14. The wheels' CI runs all four versions on every supported platform for `main`, the nightly run and each release. A pull request runs a smaller selection (xref:development/ci.adoc[Continuous integration]).
* One of the four platforms below.

Every wheel is platform-specific, built on a native runner of its platform, and carries a JRE for that platform, so pip needs no hint to pick the right one.
Expand Down
3 changes: 1 addition & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ name = "arcadedb"
version = "0.0.0"
description = "ArcadeDB embedded multi-model database with bundled JRE - no Java installation required"
readme = "README.md"
requires-python = ">=3.10"
requires-python = ">=3.11"
license = { text = "Apache-2.0" }
authors = [
{ name = "Taewoon Kim", email = "taewoon@humem.ai" }
Expand All @@ -25,7 +25,6 @@ classifiers = [
"Intended Audience :: Developers",
"License :: OSI Approved :: Apache Software License",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
Expand Down
2 changes: 1 addition & 1 deletion scripts/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ print_usage() {
echo ""
echo "PYTHON_VERSION:"
echo " Python version for wheel (default: 3.12); used by Linux (Docker) builds only"
echo " Examples: 3.10, 3.11, 3.12, 3.13, 3.14"
echo " Examples: 3.11, 3.12, 3.13, 3.14"
echo ""
echo "JAR_LIB_DIR (optional):"
echo " Directory containing ArcadeDB JARs to embed"
Expand Down
4 changes: 2 additions & 2 deletions scripts/select_matrix.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/usr/bin/env python3
"""Pick the platform x Python legs a CI workflow runs for the event that started it.

The wheels support four platforms and five Python versions, which is 20 legs per workflow. A
The wheels support four platforms and four Python versions, which is 16 legs per workflow. A
pull request does not need all of them: the organisation shares one runner cap with
ArcadeData/arcadedb, so every extra macOS or Windows leg on a pull request delays upstream too.

Expand Down Expand Up @@ -42,7 +42,7 @@

# Every Python the package supports. tests/test_select_matrix.py compares this with the
# classifiers of pyproject.toml, so adding a version there without adding it here fails.
PYTHONS = ("3.10", "3.11", "3.12", "3.13", "3.14")
PYTHONS = ("3.11", "3.12", "3.13", "3.14")
OLDEST = PYTHONS[0]
NEWEST = PYTHONS[-1]

Expand Down
29 changes: 18 additions & 11 deletions tests/test_select_matrix.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@
import importlib.util
import itertools
import json
import re
import subprocess # nosec B404 - test-controlled child processes, no untrusted input
import sys
import tomllib
from pathlib import Path

import pytest
Expand All @@ -19,15 +19,15 @@
"examples": ROOT / ".github" / "workflows" / "test-python-examples.yml",
}

# The matrix both workflows had before the selection existed: 4 platforms x 5 Pythons. Written
# out here on purpose, not imported from the script, so the full matrix cannot shrink unnoticed.
# The full matrix of both workflows: 4 platforms x 4 Pythons. Written out here on purpose, not
# imported from the script, so the full matrix cannot shrink unnoticed.
PLATFORMS = {
"linux/amd64": "ubuntu-24.04",
"linux/arm64": "ubuntu-24.04-arm",
"darwin/arm64": "macos-15",
"windows/amd64": "windows-2025",
}
PYTHONS = ["3.10", "3.11", "3.12", "3.13", "3.14"]
PYTHONS = ["3.11", "3.12", "3.13", "3.14"]
FULL = {(p, v) for p, v in itertools.product(PLATFORMS, PYTHONS)}
NON_PULL_REQUEST_EVENTS = [
"push",
Expand Down Expand Up @@ -65,16 +65,16 @@ def _pairs(matrix):
@pytest.mark.parametrize("event", NON_PULL_REQUEST_EVENTS)
def test_every_event_but_pull_request_runs_the_full_matrix(workflow, event):
matrix = sm.select(workflow, event)
assert len(matrix["include"]) == 20
assert len(matrix["include"]) == 16
assert _pairs(matrix) == FULL


def test_pull_request_bindings_matrix():
pairs = _pairs(sm.select("bindings", "pull_request"))
assert {v for p, v in pairs if p == "linux/amd64"} == set(PYTHONS)
for platform in ("linux/arm64", "darwin/arm64", "windows/amd64"):
assert {v for p, v in pairs if p == platform} == {"3.10", "3.14"}, platform
assert len(pairs) == 11
assert {v for p, v in pairs if p == platform} == {"3.11", "3.14"}, platform
assert len(pairs) == 10


def test_pull_request_examples_matrix():
Expand All @@ -95,15 +95,22 @@ def test_unknown_workflow_is_refused():
def test_pythons_match_the_classifiers_of_pyproject():
# release.yml expects one wheel per classifier Python and platform from the full matrix; a
# Python added to one place and not the other would fail the release, not a pull request.
# A regex, not tomllib: this test also runs on Python 3.10, which has no tomllib.
text = (ROOT / "pyproject.toml").read_text(encoding="utf-8")
classifiers = re.findall(r'"Programming Language :: Python :: (3\.\d+)"', text)
project = tomllib.loads((ROOT / "pyproject.toml").read_text(encoding="utf-8"))[
"project"
]
prefix = "Programming Language :: Python :: 3."
classifiers = [
c.rsplit(" ", 1)[1] for c in project["classifiers"] if c.startswith(prefix)
]
assert classifiers, "no Python version classifiers found in pyproject.toml"
assert list(sm.PYTHONS) == classifiers
# The oldest Python of the matrix is the floor of requires-python, so dropping or adding the
# oldest one cannot leave the install metadata saying something else.
assert project["requires-python"] == f">={sm.PYTHONS[0]}"


def test_command_line_prints_one_line_of_matrix_json():
for event, count in (("pull_request", 11), ("push", 20)):
for event, count in (("pull_request", 10), ("push", 16)):
out = subprocess.run( # nosec B603 - fixed argument list, no untrusted input
[sys.executable, "-I", str(SCRIPT_PATH), "bindings", "--event", event],
capture_output=True,
Expand Down
Loading