ci: install bngsim from PyPI so the simulation suites run in CI (#514) - #515
Merged
Conversation
tests.yml ran the whole suite under PYBNF_NO_BNGSIM=1 on the premise that
bngsim was a private, macOS-only wheel. It is neither: bngsim 0.11.35 --
exactly the version pyproject.toml pins -- is on public PyPI with
manylinux_2_28_x86_64 wheels for cp310-cp313. The premise cost 323
bngsim/nfsim/rulemonkey/sbml-marked tests, including 14 dedicated
tests/test_bngsim_*.py files, none of which had ever run on a hosted runner;
their only gate was one maintainer's pre-push hook.
The published wheel is fully capable -- NFsim, RuleMonkey, libsbml, codegen,
forward and steady-state output sensitivities all compile in. Only
antimony_import needs a companion package (antimony, also a manylinux wheel),
so the composite action installs bngsim[antimony] and the bngsim_antimony
tests unskip too.
- setup-pybnf grows a `bngsim` input (default true). A leg that must stay
bngsim-less passes 'false' and keeps PYBNF_NO_BNGSIM=1: py3.14 (no cp314
wheel, and it now doubles as the guard on the bngsim-free code paths), docs
(autodoc never simulates), the native-BNGL petab leg, and integration.yml's
analytical slow tier.
- The pytest matrix moves to `include` so 3.14 can carry no-bngsim: '1'. On the
other legs PYBNF_NO_BNGSIM is empty, which _bngsim_caps reads as unset.
- The marker expression becomes `-m "not slow and not recovery"`. Naming -m
REPLACES the addopts default, so the old command was also selecting the
recovery tier -- free only because 129 of its 130 tests skip without bngsim.
With bngsim present they are real parameter-recovery fits and the job would
run over an hour.
- test_fd_acceptance_gate_estimated_scale is bngsim-marked and fits a
loguniform parameter, whose log-space chain rule autodiffs through jax --
an extra CI does not install. Its twelve siblings declare
pytest.importorskip('jax'); this one did not, so it failed rather than
skipped the moment bngsim arrived. Declared.
Verified by mirroring the CI install exactly (PyPI bngsim, not the dev wheel)
and running the job's command: 3221 passed, 78 skipped, 0 failed. No
bngsim-family test skips remain; the residual skips are the jax/arviz/
matplotlib extras CI has never installed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This was referenced Jul 24, 2026
wshlavacek
added a commit
that referenced
this pull request
Jul 24, 2026
…o per-push (#518) The `recovery` tier (real parameter-recovery fits through the bngsim backend, over an hour) ran nowhere automatically -- deselected from per-push CI (#515), absent from the pre-push hook, and not part of integration.yml's nightly slow job. Add a second job to integration.yml that installs bngsim and runs `-m recovery`. It is gated to workflow_dispatch only for now (not the nightly cron): the wall-clock is unmeasured on a 2-core runner and these are stochastic fits, so it needs one dispatch to size it and a flake policy settled before it goes on a schedule. Escalating is a one-liner (drop the `if:` for nightly, add `continue-on-error` for non-blocking), documented on the job. Also fix the small live regression #515 introduced: the Boehm `test_roadrunner_reproduces_published_data` is roadrunner-backed (bngsim-free, ~3s) but was recovery-marked at the class level, so it stopped running anywhere automatic. Move the class-level recovery marker down to just the bngsim cross-check; the roadrunner leg rejoins the default per-push tier, restoring the one end-to-end real-backend signal #515 dropped. This also aligns the code with the module docstring's stated rule (RoadRunner legs in the normal tier, the bngsim leg gated -m recovery). Closes #517.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #514.
tests.ymldisabled the entire bngsim surface on the premise that bngsim is a private, macOS-only wheel. It is neither —bngsim 0.11.35, exactly the versionpyproject.tomlpins, is on public PyPI withmanylinux_2_28_x86_64wheels for cp310–cp313.The published wheel is fully capable
Installed from PyPI into a clean venv, every capability PyBNF gates on comes back available:
nfsim,rulemonkeylibsbml,sbml_import,sbml_ssa,sbml_psacodegen,output_sensitivitiesSteadyStateResult.output_sensitivitiesantimony_importantimonyPyPI package (manylinux wheel exists)So the action installs
bngsim[antimony]— thebngsim_antimony-marked tests unskip along with everything else.Changes
setup-pybnfgrows abngsiminput (defaulttrue). A leg that must stay bngsim-less passes'false'and keepsPYBNF_NO_BNGSIM=1: py3.14 (no cp314 wheel — it would fall back to an sdist build), the docs job (autodoc never simulates), the native-BNGL petab leg, andintegration.yml's analytical slow tier.includeso 3.14 can carryno-bngsim: '1'. That leg is no longer an accident of packaging: it is now the deliberate guard that the bngsim-free code paths still work, which is worth a leg regardless. On the other legsPYBNF_NO_BNGSIMis empty, which_bngsim_caps.pyreads as unset.-m "not slow and not recovery". Naming-mon the command line replaces theaddoptsdefault, so the old command was also selecting therecoverytier — harmless only because 129 of its 130 tests skip without bngsim. With bngsim present they are real parameter-recovery fits and the job would run for over an hour. The cost of deselecting is the 130th (a roadrunner-backed Boehm oracle, bngsim-free); noted in the comment.test_fd_acceptance_gate_estimated_scaleis@pytest.mark.bngsimand fits aloguniform_varparameter, whose log-space chain rule autodiffs through jax — an extra CI does not install. Its twelve siblings in the file declarepytest.importorskip('jax'); this one did not, so it failed rather than skipped the moment bngsim arrived. Declared..pre-commit-config.yaml,integration.yml,docs.ymlandtests/README_integration.md. ADRs left alone as historical records.Verification
The CI install was mirrored exactly — bngsim from PyPI, not the local dev wheel — and the job's own command run against it:
No bngsim-family test skips remain. The 78 residual skips are all optional extras CI has never installed (jax/blackjax, arviz, matplotlib) —
test_hmc.pyalone loses ~40 tests to that, which is worth a separate issue.Landing as a hard gate rather than behind
continue-on-error: the local run is 0-failure against the published wheel, and a non-blocking gate is the failure mode #514 is about.Follow-ups (not in this PR)
testsextra's jax / arviz / matplotlib, so those oracles skip in CI.recoverytier still runs nowhere automatically.🤖 Generated with Claude Code