Skip to content

Cross-compile workflow broken: dockcross images can no longer install packages #209

Open

Description

@pietrygamat

The "Cross-compile native libraries" workflow currently fails. The pinned dockcross images (20231208-4e4d3ad) are based on Debian bullseye, which has reached end of life, so apt-get install of Maven and the JDK inside them now fails.

We pinned those images on purpose, to keep the old glibc (#190), so simply moving to newer images isn't an option. More generally, I'd like the native build to meet two goals:

  1. Maximum backwards compatibility. No silent changes to glibc, toolchains or SDKs in the released natives.
  2. Stability over time. Docker image tags, package mirrors and GitHub runner images will keep disappearing, and the build shouldn't need fixing every time one does.

Quick fix: #210 points apt at archive.debian.org. It doesn't touch the compilers, so the natives are unaffected, and the workflow is green again on my fork.

Longer term: In a true 2026 spirit, I bought an AI sub and let it think for me. It wrote up the options and a proposal here: [github wiki link] . In short: build in our own pinned builder images, and fail a release when the natives would need a newer glibc or macOS than today's. Comments and other ideas are welcome.

EDIT: old link I posted originaly requires login, so replaced with github wiki link in my fork.

Activity

  1. hiddenalpha commented on Oct 17, 2026

    @hiddenalpha

    Cannot access
    https://claude.ai/code/artifact/16243e3b-be5e-4ee7-8aaf-bb6682b3ee26
    as it seems to force registration to do so:

    Image

    Therefore I'll put some of my thoughts here, independend of the AIs thoughts.

    OldOldStable

    For some other projects I usually choose the oldest available image for builds. So usually I use something like debian oldoldstable or oldstable. Then whenever they run EOL, I go forward to the one that is now the next oldoldstable.

    Possible downside: I don't know what the oldest available systems for github CICD are. Maybe github doesn't care to provdie such old images.

    Further, this still introduces those silent glibc updates. But usually not a problem for me, as it still uses a few years old glibc.

    Usually debian LTS and debian releases are good points to start decisions IMHO (thanks to the debian and related teams for all this work).

    FrankenDebian

    I think it depends on how far we can go with the glibc updates. If we really want to stay on an old glibc, then maybe (I never did that, don't know how complicated this is), we have to store the required deb packages (eg gcc-x86-64-linux-gnu and its dependencies) ourself. Eg downloading all the deb files and keep them somewhere ourself. Then for builds, install the toolchain from those deb files.

    I do not have the required experience about installing outdated deb packages on a more recent system. Sounds like this would end up in some kind of FrankenDebian. Some say it can break the system. But for a use-once-then-throw-away build machine I assume this to be less of a problem.

    Keep build dockerimages ourself

    Another option I think is to host the required dockerimages ourself (so store the build system as a whole) and use this one for builds. But I've no idea how to use custom docker images within github CICD. So maybe that's not as easy.

    HINT: Docker has an option to --export / --import whole machines into/from tar files, which are pretty handy to keep somewhere for later use.

    Qemu qcow2 base images (personal favourite)

    Yet another option (the one I choose for most of my other toy projects) is keeping myself qemu qcow2 images of my build machines. Recently I found that I can prime the apt cache while setting up such a VM (eg I pull all the apt packages I'll likely need into the VMs apt cache without installing them). Then I can just keep that qcow2 baseimage basically forever, and can setup a fresh CICD machine whenever I need one. Only downside here: If the system goes EOL, I can only use the packages I did prime the apt cache with. So whenever I need some new library, likely I cannot download it anymore. But I assume jssc not introducing native library dependencies, so that one shouldn't be a problem.

    Nevertheless, I'm not sure if such a setup is viable for a github hosted project. As the projects I'm talking about are all hosted on my own (offline operable) machines. Where I just keep a bunch of qcow2 base images organized in tarballs, alongside with notes and helper scripts.

    For this one, I once kept some notes here.

    Edit: Use archive.debian.org

    To me, using archive.debian.org looks like a valid way to solve our issue? What are the arguments we shouldn't use it?

  2. pietrygamat commented on Oct 17, 2026

    @pietrygamat
    CollaboratorAuthor

    @hiddenalpha I updated the link, so you can have more context. To me, using archive.debian.org is fine, but that leaves the risk of MacOs runners and/or dockcross base images going away, and I'd like to address that too. At this point, a reasonable and maintainable approach seems to be:

    1. Build the images ourselves, base on whatever works today, pull the dependencies once and never do docker pull/apt update ever again when compiling our work.
    2. Publish them in a ghcr.io in a repo under java-native org, but not this one.
    3. Decide about MacOS - is osxcross a viable path forward? Is freezing SDK version even reasonable, given the platform restrictions on notarization @tresf hit a while back?
  3. tresf commented on Oct 17, 2026

    @tresf

    I had a similar issue with another FOSS project and a member of the sharun team ended up helping us out. We haven't adopted it yet, but they have some tools to help escape Linux DLL hell. https://github.com/VHSgunzo/sharun

    I can ping the person that helped me if needed. I'm not entirely sure how they're doing it all. Here's the downstream fork that they're helping us with.

  4. tresf commented on Oct 17, 2026

    @tresf

    If that's not viable, using a debian archive is perfectly fine in my book but I'm also always fine dropping old Linuxes, people always have the option to use an older build.

  5. pietrygamat commented on Oct 17, 2026

    @pietrygamat
    CollaboratorAuthor

    This sharun project looks like something useful to the users of jssc, rather than us building it, though... I will point the build towards debian.archive, that'll unblock the pipeline for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions