Skip to content

Release contract: archives, image, SIF, entrypoint, release checks (A7) - #1

Merged
jcschaff merged 12 commits into
mainfrom
release/solver-contract
Sep 30, 2026
Merged

jcschaff merged 12 commits into
mainfrom
release/solver-contract

Conversation

@jcschaff

@jcschaff jcschaff commented Sep 30, 2026 •

Copy link
Copy Markdown
Member

This PR implements VCell's solver release contract (docs/plan-solver-repos.md §1, row A7) for VCellChombo2D_x64 and VCellChombo3D_x64. The details are in the new SOLVER-RELEASE.md.

What it adds

  • .github/workflows/release.yml. It runs on PRs and manual runs, which build and check everything and publish nothing, and on v* tags, which also publish.
    • Release assets: linux64.tgz (x86_64), linux64arm.tgz (aarch64), mac64.tgz (universal arm64+x86_64, ad-hoc signed) and SHA256SUMS.
    • Archive layout: flat. It holds the two executables under VCell's names, the bundled GCC runtime, LICENSE, THIRD-PARTY-LICENSES/ (Chombo, the submodules and every Conan package) and VERSION. There are no test binaries and no static libraries.
    • Linux: built on manylinux_2_28 with gcc-toolset-13. The whole Conan tree is compiled from source, because Conan Center's binaries need a newer glibc (their m4 doesn't even run there). packaging/bundle-linux.sh bundles libgfortran, libquadmath and libz with RUNPATH=$ORIGIN, and fails the build on any GLIBC_ symbol version newer than 2.28.
    • macOS: built with Homebrew GCC 13 on arm64 and x86_64. packaging/bundle_macos.py bundles the GCC runtime dylibs with @rpath install names and @loader_path as the only rpath, so nothing refers to /opt/homebrew. packaging/make-universal.sh then lipos the two builds together.
    • Windows: no win64.zip. Chombo's build needs GNU make, perl and a Unix shell, and the MinGW port on windows-ci doesn't build yet. This is documented.
  • Image ghcr.io/virtualcell/vcell-chombo:<X.Y.Z> and :latest, for amd64 and arm64. It is the release archive on debian:bookworm-slim, with no build stage, behind the standard /usr/local/bin/vcell-solver-entrypoint (ENTRYPOINT, CMD ["--help"]):
    • no arguments or --help prints the version and executables and exits 0;
    • a known executable name is execed with the arguments unchanged;
    • anything else prints usage and exits 2.
  • SIF ghcr.io/virtualcell/vcell-chombo_singularity:<X.Y.Z> and :latest (amd64, about 42 MB), pushed with ORAS.
  • Messaging is ON everywhere, so -tid works. libcurl is built without TLS, because vcell-messaging only speaks http:// to the broker's REST bridge. That also drops OpenSSL from the mac build.

Solver fixes found along the way

  1. Path buffers. BASE_FILE_NAME-derived paths went into 128-byte stack buffers, which overflowed on long absolute paths (see VCell's docs/chombo-solver-notes.md, defect 1). The buffers are now 4096 bytes. Two error paths that threw a pointer to a dead stack buffer now throw a std::string.
  2. Scratch file location. The scratch .sim.hdf5 for each timepoint was written into the current directory, so any run from a directory the user can't write failed with "cannot write zip archive". That covers a read-only container and an unprivileged user in a root-owned checkout, which is how the release check caught it. The file is now written next to the results, and it keeps its bare name in the .log and the zip, so what VCell reads is unchanged.

Checks: tests/release/check_release.py

The script runs the shipped artifact, with nothing from the build tree and only numpy and h5py on the host. Every run uses -tid 0 against a fake broker REST bridge, and must report starting, progress and completed (the last event) with no failure. It is run against:

  • each Linux archive, as an unprivileged user from a non-writable cwd;
  • the universal mac archive, on both an arm64 and an x86_64 runner;
  • both images, as the caller's uid;
  • the SIF, under apptainer run --containall --bind <work>:/simdata with a bare executable name, /simdata paths and -tid 0, exactly as SlurmProxy runs it.

The cases, each in 2D and in 3D, are usage, smoke (under a path of 287 characters or more), the regression baseline (rtol 1e-9), the analytic eigenmode, and new reference models.

The reference models reference{2,3}d.fvinput are the first inputs in this repo that move mass across the embedded boundary: U in a disc or sphere, V outside it, and a membrane flux k(U−V) between them. Results, identical on every target:

model Σ total t=0 → 0.2 max drift mass crossed (equilibrium share)
2D 32² 136.5660017 → 136.5660017 7.5e-10 64.2% (80.4%)
3D 16³ 41.50830499 → 41.50830495 1.5e-9 83.5% (93.5%)

The regression cases match the Linux/GCC 13 baselines on x86_64, aarch64 and macOS arm64, with a worst relative difference of 1.7e-15. The analytic ratio is 1.000.

Decisions

  • The Linux archives bundle libgfortran, libquadmath and libz. They treat glibc, libstdc++ and libgcc_s as system libraries: gcc-toolset needs only the GCC 8 ABI (GLIBCXX_3.4.22 max).
  • The effective minimum macOS is 15, because Homebrew's gcc@13 bottles on the macOS-15 runners carry minos 15.0.
  • Serial only. MPI Chombo (VCellChombo<N>D_x64parallel) is a follow-up.
  • The image and SIF are published on v* tags only (:<X.Y.Z> and :latest). Pushes to main publish nothing.
  • ci.yml is unchanged and stays the everyday build+ctest workflow.
  • vcell-messaging here is the 2.0 MessageEventManager rewrite. Its stopRequested flag is initialized, so the uninitialized-bStopRequested bug found in the older VCellMessaging copies does not apply. The check asserts that JOB_COMPLETED is the last event anyway.

For VCell

  • B2 (pom). Take linux64.tgz, mac64.tgz and SHA256SUMS and unpack them flat into localsolvers/{linux64,mac64}. There is no win64 asset. For notarization, the Mach-O files are the two executables plus libgfortran.5.dylib, libquadmath.0.dylib, libstdc++.6.dylib and libgcc_s.1.1.dylib.
  • C2 (submit.env). Set VCELL_HTC_VCELLCHOMBO_APPTAINER_IMAGE=oras://ghcr.io/virtualcell/vcell-chombo_singularity:<X.Y.Z> and VCELL_HTC_VCELLCHOMBO_SOLVER_LIST=Chombo.

🤖 Generated with Claude Code

jcschaff and others added 12 commits September 30, 2026 11:25
SimTool and ChomboScheduler sprintf'd BASE_FILE_NAME-derived paths into
128-byte stack buffers. VCell passes an absolute base name (the user's
simdata directory on the desktop, /simdata/<user>/ on the cluster), so a
long directory overran the stack and the next open failed on a truncated
name (noted in VCell's docs/chombo-solver-notes.md). They are 4096 bytes
now, and the error buffers leave room for a path plus the message.

Two error paths threw a local char array; main() caught the pointer after
the frame was gone. They throw a std::string copy instead, which main()
already catches.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
vcell-messaging only speaks plain http:// to the broker's REST bridge, so
TLS in libcurl is dead weight. Dropping it removes OpenSSL from the
dependency tree -- the longest compile on macOS, where Conan builds
everything from source -- and leaves the release binaries with no CA
bundle path to get wrong on someone else's machine.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
tests/release/check_release.py runs the packaged solvers -- an unpacked
archive, or a container command line with a bare executable name, the
way SlurmProxy runs the SIF -- against the smoke, regression, analytic
and new reference inputs, and checks the answers with only numpy and
h5py on the host. With --messaging it serves a fake broker REST bridge,
passes -tid 0, and requires starting/progress/completed events.

reference{2,3}d.fvinput are the first inputs that push mass through the
embedded boundary: U in a disc/sphere, V outside, a membrane flux
k(U - V) between them and zero flux on the box. The solver's own
variable statistics must show total(U)+total(V) conserved while mass
actually crosses.

The smoke case runs from a directory whose absolute path is longer than
256 characters, which the old 128-byte path buffers could not hold.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
release.yml builds linux64 (x86_64) and linux64arm (aarch64) on
manylinux_2_28 with gcc-toolset-13, and macOS arm64 and x86_64 with
Homebrew GCC 13 lipo'd into one universal mac64. Conan dependencies are
static; the GCC runtime is bundled next to the executables (RUNPATH
$ORIGIN on Linux, @rpath/@loader_path and ad-hoc signing on macOS), and
the Linux files are checked against the glibc 2.28 baseline.

The image is the Linux archive on debian:bookworm-slim behind the
standard vcell-solver-entrypoint; the SIF is built from it. Every
artifact is checked the way it will be run: the archives unpacked fresh
as an unprivileged user, the image as the caller's uid, and the SIF
under apptainer run --containall with /simdata bound, all with -tid 0
against a fake broker. On a v* tag the workflow publishes the release
assets with SHA256SUMS, ghcr.io/virtualcell/vcell-chombo:<X.Y.Z> and
:latest (amd64+arm64), and the SIF via ORAS.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
SOLVER-RELEASE.md records how this repository meets VCell's solver
release contract; README links it and CLAUDE.md notes the release traps.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
SimTool::writeData wrote each timepoint's .sim.hdf5 into the current
directory under its bare name, zipped it into <base>NN.hdf5.zip and
deleted it. An unprivileged run from a directory it cannot write -- the
release check, run as a separate user from the checkout, found it; a
read-only SIF or a desktop launched from anywhere would too -- failed
with 'cannot write zip archive'. The scratch file now lives in the
BASE_FILE_NAME directory, which must be writable anyway; the .log and the
zip entry keep the bare name, so what VCell reads is unchanged.

That leaves the image entry point nothing to work around: it just execs.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
FLUX <feature> in a jump condition is the flux into that feature, so the
exchange J = k(U - V) is -J on cyt and +J on ec. Written the other way
round the first run pushed mass uphill until V went to -8x the total --
while the total stayed conserved to 8e-9, which is why the check also
requires the crossed fraction to be positive and non-trivial.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@jcschaff
jcschaff marked this pull request as ready for review September 30, 2026 19:46
@jcschaff
jcschaff merged commit b07a3ac into main Sep 30, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant