[mrrobot47]

Builds, smoke-tests and publishes every image in the build matrix for linux/amd64 and linux/arm64, so EasyEngine can run on arm64 Linux hosts and Apple Silicon. This PR edits .github/workflows/test_and_publish.yml (rewritten) and adds .github/scripts/smoke-test.sh. It also adds a separate one-time workflow for arm64 variants of the images outside the matrix (.github/workflows/legacy-arm64.yml, .github/legacy-arm64/, .github/scripts/legacy-arm64.sh; see the section below).

What changes

Workflow (test_and_publish.yml):

• One build job, matrix image (12) × arch (amd64, arm64) = 24 jobs, fail-fast: false. amd64 runs on ubuntu-24.04, arm64 on the native ubuntu-24.04-arm runner (free for public repos). No QEMU: emulated PHP builds are about 17× slower, and a wrong-arch binary still "works" under binfmt.
• Each job builds the image for its arch with load: true, then runs .github/scripts/smoke-test.sh against it.
• On publish runs it builds again on the same builder (reusing the tested layers) and pushes by digest; the digest goes into a digests-<img>-<arch> artifact (retention 1 day).
• A merge job per image (needs: build, if: !cancelled(), so one failed image doesn't block the others) downloads both digests, refuses to publish unless both arches are present, creates easyengine/<img>:<tag> with docker buildx imagetools create, and checks that the index lists exactly linux/amd64,linux/arm64. An image that fails on either arch is not published at all, never half-published.
• Modes:
  • v* tag pushed to EasyEngine/dockerfiles: Docker Hub login (same DOCKERHUB_USERNAME/DOCKERHUB_TOKEN secrets) before the first build, push by digest, multi-arch tag. Publishing still happens only on v* tags, as before. No latest tag, no build on develop pushes, every matrix image pushed on each tag, as before.
  • pull_request: build and smoke-test both arches; no push, no merge job.
  • Anything else (workflow_dispatch, which is new, or tags on forks): a secret-free dry run. Each arch's OCI layout goes through artifacts into a throwaway registry:3 on the runner, and the same manifest + verify steps run against localhost:5000. After merging, gh workflow run test_and_publish.yml --ref develop exercises the whole publish path before tagging.
• Every build gets GITHUB_TOKEN as the optional BuildKit secret github_token. Secrets don't land in layers or in the cache key; builds that don't mount it ignore it.
• No layer cache, on purpose: the Dockerfiles fetch the latest New Relic, wp-cli and composer and some run apt-get upgrade, so a cache would ship stale layers on tags.

Smoke test (.github/scripts/smoke-test.sh <image> <ref> <arch>, also usable locally): checks the image platform, that each image's key binaries are ELF files for the target arch (e_machine x86-64 / AArch64), then per image: php modules (incl. imagick, redis, memcached, imap, gmagick and New Relic where built), gd formats, composer, wp-cli, php-fpm -t; nginx -t; docker-gen, forego and the proxy template; redis PING with EE's config; mariadb SELECT VERSION(); postfix start + status; ofelia validate; a MailHog SMTP round trip with EE's -invite-jim=false; the New Relic daemon staying up.

Per-image fixes:

• newrelic-daemon: copied the hardcoded newrelic-daemon.x64, so an arm64 build would ship an amd64 binary. It now picks x64/aarch64 from apk --print-arch and fails the build on any other arch.
• mailhog: mailhog/mailhog:v1.0.1 is amd64-only (a single 2020 manifest) and upstream is unmaintained. The image now builds the same v1.0.1 tag from its vendored sources (golang:1.27.3-alpine3.24, GOPATH mode, CGO_ENABLED=0) and ships it on alpine:3.24.3 with the same user (mailhog, uid 1000), workdir, entrypoint, ports and flags. Behaviour is unchanged (same UI and API; EE's -invite-jim=false still applies). The Go toolchain is no longer in the image, which shrinks it from 572 MB to about 29 MB. Dependabot's existing /mailhog entry now tracks the golang and alpine bases.
• Everything else needed no change: every other base image already publishes arm64, PECL extensions compile per arch, and the New Relic PHP installer picks agent/aarch64 itself.

Which images get arm64: all 12 in the matrix (php8.1–8.5, nginx, nginx-proxy, postfix, redis, cron, mailhog, newrelic-daemon). mariadb leaves the tag-triggered matrix: core stays on MariaDB 10.11, so the tag run no longer builds mariadb/Dockerfile (12.3); the multi-arch 10.11.14 image comes from the one-time workflow below. php5.6–8.0 also stay out of the matrix under the EOL-base policy and get arm64 variants once through the same workflow. easyengine/php (stable) stays amd64-only. Existing tags are never re-pushed, so only the next dockerfiles tag is multi-arch.

Verification

Dry-run publish on a fork branch (the trigger widened to that branch is the only workflow difference from this PR):

• 39996556827 (current head, mariadb out of the matrix): all 36 jobs green (24 build + smoke, 12 merges), every merged manifest linux/amd64,linux/arm64. The runs below still had mariadb (12.3) in the matrix.

• 39961111371: all 39 jobs green (26 build + smoke, 13 merges). 88 ELF checks passed; every merged manifest verified linux/amd64,linux/arm64.

• amd64 equivalence with the published v4.13.3 images (develop's tree equals the v4.13.3 tag), comparing image config (User, Entrypoint, Cmd, WorkingDir, Env, ExposedPorts, Volumes, Labels, StopSignal), the full package list (dpkg-query/apk info) and, for php, php -m and the PHP/composer/wp-cli/New Relic versions:

  Image	Config	Packages	Notes
  php8.1–8.5	identical	identical, except tzdata 2026c → 2026e on 8.1 (its apt-get upgrade)	php -m identical; PHP 8.1.34/8.2.35/8.3.36/8.4.27/8.5.12 (same builds), Composer 2.10.3, WP-CLI 2.12.0, New Relic 12.12.0.41 (not 8.5)
  nginx-proxy, redis, mariadb (12.3.3), cron	identical	identical
  postfix	identical	tzdata 2026c → 2026e only
  newrelic-daemon	identical	brotli-libs 1.2.0 → 1.2.1 only
  nginx	base labels only (openresty re-pushed its image on 2026-10-22)	ca-certificates, perl-base Debian point updates only
  mailhog	intended: the Go PATH/GOPATH/GOLANG_VERSION env is gone; user, entrypoint, workdir, ports and labels identical	alpine 3.12 → 3.24.3 (15 → 16 packages)	built from source, see above

  The non-mailhog differences are upstream package updates published after the v4.13.3 tag run, not changes from this PR.

• 39961114837: the same branch merged with every open PR (build(php): install extensions with PIE and fail the build on missing modules #367–fix(php): drop deprecated session ID ini settings from 8.4 and 8.5 #372, build(php): install extensions with PIE and fail the build on missing modules #367 resolved as below): all 39 jobs green, 88 ELF checks, 13/13 manifests linux/amd64,linux/arm64, so the PIE builds, postfix 3.11.7 and the other refreshed images also build and pass on arm64.

CI time

PR runs go from 13 amd64 jobs to 24 jobs (12 per arch, in parallel on native runners). Tag runs go from 13 jobs to 36 (24 builds + 12 short merge jobs). Wall time: about 8 minutes (8 m 15 s for 39961111371; arm64 PHP builds 5–6.5 min, the same as amd64) for a full dry run, against about 6 minutes for the old amd64-only tag run; PHP builds take about the same on arm64 as on amd64. Runner minutes roughly double, which is free on public repos.

Conflicts with open PRs

• build(php): install extensions with PIE and fail the build on missing modules #367 (PIE) conflicts in test_and_publish.yml only: it adds secrets: github_token=${{ secrets.GITHUB_TOKEN }} to the two PHP build steps of the old workflow. This PR already passes that secret to both build steps of every image, so whichever merges second resolves the conflict by keeping this PR's version of test_and_publish.yml unchanged (git checkout --ours/--theirs to the multi-arch side, no hand-editing). The PIE Dockerfiles then build on both arches (verified in 39961114837).
• chore(deps): bump alpine from 3.24.2 to 3.24.3 in /newrelic-daemon #371 (newrelic-daemon alpine 3.24.2 → 3.24.3) merges cleanly: it changes only the FROM line, this PR only the RUN body.
• fix(postfix): update postfix to 3.11.7 and ship /etc/aliases #368, fix(nginx-proxy): apply pending Debian updates at build time #369, fix(cron): apply pending Alpine updates at build time #370 and fix(php): drop deprecated session ID ini settings from 8.4 and 8.5 #372 merge cleanly.

One-time arm64 builds of the legacy images

php5.6, php7.0, php7.2, php7.3, php7.4, php8.0 and mariadb (10.11, the version core pins) aren't in the build matrix, so the tag run doesn't make them multi-arch. A separate workflow, .github/workflows/legacy-arm64.yml (workflow_dispatch only), gives each of them an arm64 variant once. It never touches the amd64 images: it builds only arm64, then publishes a new tag (v4.14.0) whose index holds the published amd64 image by digest (unchanged) plus the new arm64 image. Existing tags aren't overwritten, and the legacy images don't join the regular matrix. easyengine/php (stable) isn't included: no site uses it. There's no php7.1 recipe either: it isn't supported by EasyEngine and has no Docker Hub repository.

Recipes (.github/legacy-arm64/recipes.json plus one <image>.patch each): each recipe names the published amd64 digest, the commit that matches that image, and a small patch.

• The published v4.7.4 php5.6–7.3 images weren't built from the v4.7.4 tree (they predate the tmux/usermod change, and 7.2/7.3 also lack WebP). Their layers and history match 681d01a (5.6), 7b74e92 (7.2) and 2807ccb (7.3). php7.0 matches a528d29 plus an apt-get upgrade -y and the openssl package that its history shows but no commit has. php7.4 and php8.0 use the v4.13.0 tree.
• Patches: the frozen Debian bases (stretch, buster, bullseye at the 2024 build date) install from snapshot.debian.org at the amd64 build date. For stretch, whose archive keys have expired, the hashes of the release files apt stores are pinned to the copies verified against those keys as of that date, and checked right after apt-get update, before anything is upgraded or installed; apt then checks every index and package against those files. PECL versions are pinned to the ones in the amd64 images, and php8.0's New Relic agent to 12.11.0.40.
• The amd64 images inherit some failed PECL steps (for example, redis needs PHP ≥ 7.4, and the docker-php-ext-enable line that would also enable imagick then fails), so several extensions are compiled but not enabled. The arm64 builds reproduce that exactly instead of diverging between arches.
• mariadb: easyengine/mariadb:v4.9.1 is MariaDB 10.11.14 on Ubuntu 22.04, although its tag's Dockerfile says FROM mariadb:11.5. Its eight base layers are the amd64 image of mariadb:10.11.14@sha256:e4ad7c5a… (the index of 2025-09-02 recorded in docker-library/repo-info), which is still pullable and has an arm64 image from the same build. The recipe builds FROM that index with the tag's ee.cnf and my.cnf, which are byte-identical to the published layers. No version change.

Checks before anything is pushed (arm64 build vs. the published amd64 image, collected on native runners): image config, PHP version, php -m, every extension version, every ini value, php-fpm -t, and the checksum of every file under /usr/local/etc must be identical. For MariaDB: SELECT VERSION(), SHOW GLOBAL VARIABLES (minus host-specific ones) and every file under /etc/mysql. php, php-fpm, every extension .so, and mariadbd/mariadb/gosu must be AArch64 ELF files. Package lists and Composer/WP-CLI versions are logged for information. Then the new tag is refused if it exists, the index is created, and the job checks that it lists exactly linux/amd64,linux/arm64 with the published amd64 digest and the tested arm64 digest. Finally it pulls the tag on the arm64 runner and repeats the checks against the pulled image.

Known, documented arm64 differences:

• New Relic isn't available for PHP 7.x on arm64. New Relic has never shipped an aarch64 agent for PHP 7.x, so the arm64 php7.0–7.4 images ship without it ("newrelic": "skip"), while their amd64 images keep it. The agent step is dropped, newrelic.ini is kept without its extension= line so the entrypoint's edit still works, and the check allows exactly the New Relic lines to differ. php5.6 never had New Relic; php8.0 has it on both arches (agent 12.11.0.40). A recipe set to "unavailable" stops with an explicit message instead.
• php8.0 JIT: PHP 8.0 has no JIT on AArch64, so the opcache.jit* settings don't exist there (JIT is off on amd64: opcache.jit_buffer_size=0).
• Packages: the only package differences are arch-specific packages (toolchain, libquadmath0, binNMU rebuilds).
• Tools: Composer and WP-CLI are downloaded as the latest versions, as the Dockerfiles do, so they are newer on arm64.

Verification (dry runs on a fork branch; the only temporary changes were a push trigger and default inputs):

39996556771 (current head, recipes exactly as committed, tag=v4.14.0): all 15 jobs green. All seven arm64 images match their published amd64 images on every check; the only allowed differences are New Relic on php7.0–7.4 and php8.0's absent opcache.jit*. ELF checks: 10–29 per PHP image and 4 for mariadb. v4.14.0 is free on Docker Hub for all seven. In each combined index the amd64 digest equals the published one: php5.6 0d02a5f8, php7.0 dc42a302, php7.2 bd388833, php7.3 e85e80a6, php7.4 84463e41, php8.0 b7da1852, mariadb 60774518. MariaDB returned 10.11.14-MariaDB-ubu2204 on both arches, with identical server variables (minus hostname-derived ones), /etc/mysql and package list.

• 39995725548: php5.6 and php7.0 after binding apt to the verified stretch release files: all three _Release files OK right after apt-get update, every check identical, amd64 digests unchanged.

Earlier runs:

• 39992484522: all 15 jobs green (7 reference + 7 arm64 builds + setup). Every arm64 image matched its published amd64 image on all checks (php8.0's absent opcache.jit* and php7.x's New Relic are the only allowed differences). ELF checks: 10–29 binaries per PHP image, 4 for mariadb. MariaDB returned 10.11.14-MariaDB-ubu2204 on both arches, with identical server variables (minus hostname-derived ones), /etc/mysql and package list. In each combined index the amd64 digest equals the published one: php5.6 0d02a5f8, php7.0 dc42a302, php7.2 bd388833, php7.3 e85e80a6, php7.4 84463e41, php8.0 b7da1852, mariadb 60774518.
• Package differences between arches are only arch-specific packages (binutils/gcc runtime, libquadmath0, binNMU +b1 rebuilds). Composer/WP-CLI differ only on the images built before 2026 (e.g. php5.6: Composer 2.2.21 → 2.2.30, WP-CLI 2.7.1 → 2.12.0).
• 39994288151: refusal test with images=mariadb, tag=v4.9.1: the build and checks passed, then "Refuse an existing tag" failed with easyengine/mariadb:v4.9.1 already exists before anything was pushed.

Runbook (once, after this PR is merged, from develop):

# 1. Dry run: everything except the Docker Hub push; publishes into a registry on the runner.
gh workflow run legacy-arm64.yml -R EasyEngine/dockerfiles --ref develop -f images='php5.6 php7.0 php7.2 php7.3 php7.4 php8.0 mariadb' -f tag=v4.14.0 -f publish=false
# 2. The real run (Docker Hub login with the existing DOCKERHUB_* secrets).
gh workflow run legacy-arm64.yml -R EasyEngine/dockerfiles --ref develop -f images='<same list>' -f tag=v4.14.0 -f publish=true
# 3. Check from outside.
docker buildx imagetools inspect --raw easyengine/<img>:v4.14.0 | jq -c '[.manifests[] | select(.platform.architecture != "unknown") | {a: .platform.architecture, d: .digest}]'

images defaults to all seven and tag to v4.14.0. A subset can be re-run (-f images=mariadb); an image whose tag already exists is refused. Build jobs for the same image and tag share a non-cancelling concurrency group, so a second run waits and then refuses the tag the first one created. The job summary lists the amd64 and arm64 digests per image. Core then moves the pins in img-versions.json to v4.14.0.

After merging

Tag as usual; the tag run publishes two-platform indexes. arm64 hosts then need core to pin the new tags in img-versions.json.

[Copilot]

🟢 Approval recommended

No unresolved findings remain, and the reported verification runs passed builds, smoke tests, and manifest checks for both architectures.

No open issues 🎉

What changed in this PR

Adds native linux/amd64 and linux/arm64 builds for all 13 images in the publishing matrix, enabling future releases to support arm64 hosts while preserving tag-only Docker Hub publishing.

Changes:

• Builds and smoke-tests both architectures, then combines their digests into multi-architecture tags; adds a local-registry dry-run mode.
• Adds platform, binary architecture, and per-image runtime smoke checks.
• Builds MailHog from source and selects the correct New Relic daemon binary for each architecture.

File	Description
newrelic-daemon/​Dockerfile	Selects the daemon binary for amd64 or arm64.
mailhog/​Dockerfile	Replaces the amd64-only base with a multi-stage source build.
.github/​workflows/​test_and_publish.yml	Adds dual-architecture builds, smoke tests, digest merging, and dry runs.
.github/​scripts/​smoke-test.sh	Checks image platforms, binary architectures, and service functionality.

_{Review effort: Balanced}

[Copilot]

🟡 Changes recommended

Legacy package metadata is not effectively authenticated, and concurrent legacy publish runs can overwrite the same tag.

2 open issues

• APT metadata and packages are not authenticated against pinned hashes · New
• Serialize legacy publishes to prevent concurrent tag overwrites · New

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

The release pipeline rewrite and reconstruction of legacy images warrant human sign-off before production publication, despite the reported successful dry runs.

No open issues 🎉

2 resolved since last review

• APT metadata and packages are not authenticated against pinned hashes
• Serialize legacy publishes to prevent concurrent tag overwrites

_{Review effort: Balanced}