Skip to content

Pytest matrix runs a single interpreter; binding it exposes a macOS/3.11 hatch env failure #156

Description

@sophiagavrila

Summary

.github/workflows/pytest.yaml declares a 15-cell test matrix, but every cell runs the same interpreter, so the repo currently has no real Python version coverage. Fixing that reveals a second, separate problem: hatch env create dev fails on macos-latest + Python 3.11.

Raised by Copilot review on #155; removed from that PR to keep it scoped to dependency patching.

1. The matrix does not test what it claims

strategy:
  matrix:
    os: [ubuntu-latest, macos-latest, windows-latest]
    python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]
steps:
  - name: Set up Python
    uses: actions/setup-python@v7
    with:
      python-version: '3.x'      # <-- not bound to the matrix

python-version is hardcoded, so all 15 jobs install the newest available CPython. The job names (pytest (ubuntu-latest, 3.10)) claim coverage the run does not provide. requires-python = ">=3.10", so 3.10–3.13 are effectively untested.

The one-line fix is python-version: ${{ matrix.python-version }}.

2. That fix exposes a macOS + 3.11 environment failure

With the binding applied, pytest (macos-latest, 3.11) fails during Install dependencies:

Syncing environment plugin requirements
Environment `dev` has unknown type: pip-compile
##[error]Process completed with exit code 1.

Notes:

  • Reproducible across two independent runner allocations (initial run and gh run rerun --failed).
  • The plugin sync completes in ~240 ms and installs nothing before the failure.
  • pip-compile comes from hatch-pip-compile, declared in [tool.hatch.env] requires. The error means the plugin was not loaded, not that the dev dependencies are wrong.
  • Not reproducible locally. On macOS arm64 with CPython 3.11.14, against a clean main worktree, hatch env create dev exits 0 and hatch run dev:pytest gives 76 passed / 100% coverage. Same result on the chore: combine annotated-logger dependency updates #155 branch.
  • Other cells in the same run get past this step (ubuntu-latest, 3.10 ran the full suite green before fail-fast cancelled it).

So this looks like a hosted-runner-specific hatch plugin resolution problem, not a dependency or source defect.

3. Fail-fast hides the picture

strategy has no fail-fast: false, so the single macOS/3.11 failure cancelled the other 14 jobs mid-step. Adding fail-fast: false alongside the matrix fix would show the true per-version state in one run instead of one failure at a time.

Suggested order of work

  1. Add fail-fast: false.
  2. Bind python-version: ${{ matrix.python-version }}.
  3. Triage the resulting failures; investigate the macOS/3.11 hatch env create dev failure (likely pinning hatch/hatch-pip-compile, or installing the plugin explicitly before hatch env create).

Evidence

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions