CI installs a hand-mirrored dep list that omits the tests extra's jax/arviz/matplotlib — 69 tests skip

[wshlavacek]

Follow-up from #514 / #515. With bngsim now installed from PyPI, the bngsim-family tests run in CI and the residual skips are all one thing: optional extras that pyproject.toml declares as test dependencies but the CI composite action never installs.

The measurement

From the py3.12 leg of the first green run on #515 (identical locally): 3221 passed, 78 skipped. Broken down by skip reason:

missing	skipped	where
jax / blackjax	66	tests/test_hmc.py (52), tests/test_gradient_assembly.py (14)
arviz	2	tests/test_inference_data.py, tests/test_tutorial_am_diagnostics.py
matplotlib	1	tests/test_profile_likelihood.py
— (legitimately unconditional)	9	malformed-config fixtures, PYBNF_REGEN_GOLDEN gate, native-petab-only case

So 69 of the 78 remaining skips are this gap, and tests/test_hmc.py — the entire ADR-0059 HMC reference-sampler oracle — has never run on a hosted runner.

Why it happens

pyproject.toml's tests extra already declares every one of them, with comments explicitly stating the intent — e.g. on jax:

so the gradient path is exercised in CI rather than skipped

and on arviz:

so the bridge is exercised in CI rather than skipped

But .github/actions/setup-pybnf/action.yml does not install the extra. It hand-mirrors a subset (pytest, hypothesis, pytest-xdist, petab) and the comment tells you to keep it in sync with pyproject.toml by hand. That sync has drifted: jax, jaxlib, blackjax, arviz, h5netcdf, h5py and matplotlib are all in the extra and none reach the runner. The tests then importorskip and the job stays green while testing nothing.

This is the same failure shape as #514 — a suite that is written, marked, and switched off — just with a different cause.

Suggested fix

Add the missing pins to the composite action's install list, or install .[tests] and drop the hand-mirrored list entirely (the latter removes the drift class outright; it needs care with the petab-spec override, which must still win, and with --no-deps on the pybnf install).

Wheel availability is not a blocker on any leg: matplotlib, h5py and jaxlib all publish cp311–cp314 wheels (jaxlib 0.11.0 is cp312+, so py3.11 backtracks to an older release), and arviz/jax/blackjax are pure Python.

Worth deciding

jax + jaxlib are a large install relative to the current dep set. If that is unwelcome on all four matrix legs, one leg with the full tests extra would still take test_hmc.py from never-run to run — the same "second leg" shape #514 considered for bngsim.

[wshlavacek]

Fixed in e247c48.

The setup-pybnf composite action now installs the [tests] extra's optional deps that it previously hand-mirrored around: the light extras (arviz/h5netcdf/h5py/matplotlib) go on every matrix leg, and the heavy jax stack (jax/jaxlib/blackjax) is gated behind a new jax input and installed on a dedicated pytest (jax / HMC) job. The ADR-0059 HMC oracle (tests/test_hmc.py) and the jax gradient-assembly tests now run on a hosted runner for the first time instead of importorskip-skipping.

Running the HMC oracle on CI immediately surfaced a latent cross-platform fragility: test_hmc_recovers_banana_moments demanded exactly-zero NUTS divergences, which isn't portable across jaxlib/XLA builds (0 locally, 1 in ~10^4 transitions on the runner, moments still recovered). Bounded by a <0.5% divergence rate in bbe7f5f — exactly the kind of switched-off-suite finding this issue was about.

CI is green across all six jobs, including the py3.14 bngsim-less leg (the arviz/cp314 wheel concern didn't materialize).