Skip to content

Fix hosted PyPI pinning platform-only wheels (#701, #932) - #984

Open
Mikola Lysenko (mikolalysenko) wants to merge 6 commits into
mainfrom
agent/fix-hosted-pypi-platform-wheel
Open

Mikola Lysenko (mikolalysenko) wants to merge 6 commits into
mainfrom
agent/fix-hosted-pypi-platform-wheel

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Oct 7, 2026 •

Copy link
Copy Markdown
Collaborator

LLM Description written by Claude Code:claude-opus-5-5

Fixes #701
Fixes #932

Root cause

Every hosted PyPI lock writer is dispatched from rewrite_registry_redirect_withholding_vlt in crates/socket-patch-core/src/patch/redirect/mod.rs: requirements.txt, Hatch, uv.lock, PEP 723 script locks, pylock.toml, Poetry, PDM and Pipfile.lock. None of them looks at the granted wheel's tags. When the patch service grants a platform/ABI-tagged wheel (…-cp311-cp311-manylinux…whl), each writer narrows a cross-platform lock entry to that single wheel. The scan reports success, and installs on any other Python, OS or architecture then fail. Vendored mode already parses the tags (wheel_platform_from_filename → vendor_platform_locked), and hosted gem already fails closed on the equivalent case (redirect_gem_platform_unsupported).

Change

  • One gate at the shared boundary. pypi_platform_wheel_refusal refuses a pypi override whose artifact is a wheel whose tag triple isn't <py>-none-any. withhold_pypi_platform_wheels runs it once at the top of the dispatch, before pdm and Pipenv, so the patch is withheld from every PyPI rewriter and reported once as redirect_pypi_platform_wheel. Nothing is written or confirmed for that patch, so a same-run --vex doesn't attest it. Hosted refusals exit 0 with a warning, which is the existing contract precedent. Sibling patches in the same run are unaffected.
  • Vendored → hosted takeover. The takeover reverts a vendored PyPI purl before the rewriters run, so a platform grant would have stripped a live vendored patch and stranded the package. The takeover now asks the same refusal before it reverts anything, and the package stays vendored and patched, on the dry run too.
  • Shared rule. The tag classifier moved from vendor/pypi.rs to vendor/pypi_distribution.rs, so vendored and hosted mode agree on which wheels are portable. The group-equivalence oracle mirrors the new prefix step.
  • Docs. CLI_CONTRACT.md documents redirect_pypi_platform_wheel.
  • Inherited CI fix. main is red on test (macos/windows) and coverage because of utils::digest::tests::production_digests_go_through_the_helpers. I cherry-picked Route Gradle digests through utils::digest #878's fix (3298cee); it becomes a no-op once Route Gradle digests through utils::digest #878 merges.

I chose refuse over warn to match hosted gem, and because the warn-only alternative leaves a lock that breaks every other platform and that hosted rollback can't undo. Vendored mode keeps its existing warn-only vendor_platform_locked behavior.

Test evidence (red → green)

Every new test fails on main's logic (gate disabled) and passes with the fix:

Issue Lane Test
#701 uv.lock (LF + CRLF) platform_wheel_tests::uv_project_lock_refuses_a_platform_wheel
#701 PEP 723 script lock platform_wheel_tests::uv_script_lock_refuses_a_platform_wheel
#701 pylock.toml platform_wheel_tests::pylock_refuses_a_platform_wheel
#701 requirements.txt (LF, CRLF, hashed) platform_wheel_tests::requirements_refuses_a_platform_wheel
#932 Pipfile.lock (Pipenv 11 / 2026 / unknown) platform_wheel_tests::pipfile_lock_refuses_a_platform_wheel
#932 CLI end to end: Pipfile.lock + sibling requirements.txt untouched, exit 0, VEX doesn't attest in_process_redirect_pipenv::platform_wheel_is_not_pinned_into_the_lock
both poetry.lock, pdm.lock, Hatch pyproject platform_wheel_tests::{poetry,pdm,hatch}_*
both tag rule (cp311-none-any, sdist, abi3, win_amd64, query/fragment) platform_wheel_tests::only_platform_or_abi_tagged_wheels_are_withheld
both per-patch withholding, npm ignored platform_wheel_tests::a_platform_wheel_withholds_only_its_own_patch
both vendored → hosted takeover refused before revert (wet + dry) mode_migration_pypi::platform_wheel_takeover_is_refused_before_revert

Without the gate: 0 passed; 10 failed for platform_wheel_tests. The CLI test shows the cp311 wheel written into Pipfile.lock (the #932 symptom), and the takeover test reports redirect_takeover_reverted_vendored. With the fix all of them pass.

Local runs.

  • cargo clippy --workspace --all-features -- -D warnings is clean.
  • cargo test --workspace --all-features --no-fail-fast: 10,829 passed. The 12 failures are all permission-injection tests (chmod 0o555 / unremovable-file cases in covgap_commands_vendor, in_process_redirect, repair, and core copy_tree / vlt_heal / pypi_poetry / pypi_requirements). They fail only because this sandbox runs as root, which ignores file modes, and none touch this diff.
  • e2e_redirect_uv_build --include-ignored with uv 0.11.32: 18/22 pass. The 4 rollback lanes fail only because the sandbox's TLS proxy blocks the binary's pypi.org restore fetch (error sending request for url (https://pypi.org/pypi/six/1.16.0/json)); CI has direct access.
  • I didn't run a repo-wide cargo fmt --all because main has ~466 rustfmt diffs and CI has no fmt job. My hunks are rustfmt-clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Eh8HJTpZnGoKmxzQMniJHa


Note

Medium Risk
Changes hosted scan/redirect behavior for PyPI grants and vendored→hosted takeover paths; mistakes could block valid redirects or leave bad pins, but the change is narrowly scoped with broad test coverage.

Overview
Hosted PyPI redirect now refuses to pin patches when the granted artifact is a platform- or ABI-tagged wheel (anything other than a portable py-none-any wheel). A shared early gate (pypi_platform_wheel_refusal / withhold_pypi_platform_wheels) withholds those patches from all PyPI lock rewriters, emits redirect_pypi_platform_wheel once, leaves lockfiles unchanged, exits 0, and skips same-run VEX attestation—matching the hosted-gem fail-closed pattern.

Vendored → hosted takeover checks the same rule before reverting a live vendored PyPI patch, so a platform grant cannot strip vendoring and strand the package unpatched (wet and dry run).

Wheel tag classification is centralized in pypi_distribution so vendored vendor_platform_locked and hosted redirect share one rule. Several call sites now use utils::digest helpers for SHA1/SHA256 instead of inline hashing. CLI contract documents the new warning; tests cover every PyPI lane, takeover, and Pipenv CLI behavior.

Reviewed by Cursor Bugbot for commit 8725c55. Configure here.

Assisted-by: Claude Code:claude-opus-5-5
When the patch service granted a PyPI patch as a platform- or
ABI-tagged wheel (for example cp311 manylinux), hosted scan pinned
that one wheel into the project's cross-platform lock: uv.lock, PEP
723 script locks, pylock.toml, Pipfile.lock, poetry.lock, pdm.lock,
requirements.txt or Hatch's pyproject. It reported success, but
installs then failed on every other Python version, OS and
architecture, and hosted rollback refused to undo it.

Hosted mode now checks the granted wheel's tags once, where every
PyPI lock writer is dispatched. A platform-specific wheel is withheld
from all of them and reported with a redirect_pypi_platform_wheel
warning, the same way hosted gem refuses platform gems. Nothing is
written or attested for that patch; other patches in the run are
unaffected. The tag rule is the one vendored mode already uses for
vendor_platform_locked, now shared between both modes.

Fixes #701, #932.

Assisted-by: Claude Code:claude-opus-5-5
A vendored PyPI package that a hosted scan takes over is reverted to
its registry entry first, and only then pinned to the hosted wheel.
With platform-tagged hosted wheels now refused, that order would strip
the live vendored patch and leave the package unpatched.

The takeover now asks the same platform-wheel check before it reverts
anything, so the package stays vendored and patched, and both the wet
run and the dry run name redirect_pypi_platform_wheel as the reason.

Refs #701, #932.

Assisted-by: Claude Code:claude-opus-5-5
Assisted-by: Claude Code:claude-opus-5-5
main has failed socket-patch-core's lib tests since Gradle support
(#646) and the digest helpers (#865) both landed. The guard test
production_digests_go_through_the_helpers flags three files #646 added
that still hash inline: crawlers/gradle_cache.rs, patch/jvm_jar.rs and
patch/sidecars/maven.rs. That breaks test, test-release and coverage on
every open PR.

Each inline sha1/sha256 call now goes through sha1_hex_of or
sha256_hex_of, which compute the same lowercase hex. Behaviour is
unchanged.

Assisted-by: Claude Code:claude-opus-5-5
(cherry picked from commit 659ac2c)
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

[agent] test (macos-latest), test (windows-latest) and coverage failed on the empty start commit ff82eda. That failure isn't from this PR: the same three checks are red on main (9c43dfc, run 37420351192). The one failing test is utils::digest::tests::production_digests_go_through_the_helpers: the Gradle files from #646 hash inline (crawlers/gradle_cache.rs, patch/jvm_jar.rs, patch/sidecars/maven.rs), and #865's guard test rejects inline hashing. I ported #878's fix (659ac2c, cherry-picked as 3298cee) into this branch. It becomes a no-op once #878 lands on main.


Generated by Claude Code

The new Pipenv platform-wheel test asserted exit 0 on a run that also
asked for --vex. With nothing pinned, VEX correctly fails with
manifest_not_found, so the run exits 1 and the coverage job failed.

Assert the hosted refusal's exit 0 on a plain scan, then run --vex
separately and check only that it attests nothing.

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko
Mikola Lysenko (mikolalysenko) marked this pull request as ready for review October 7, 2026 05:34
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

BugBot review


Generated by Claude Code

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 8725c55. Configure here.

@mikolalysenko Mikola Lysenko (mikolalysenko) added the Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review label Oct 7, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

[agent] Ready for review at 8725c55.

  • CI: 547/547 check runs on 8725c55 are green or skipped.
  • Bugbot: reviewed 8725c55 and found no issues. There are no open review threads.
  • Look at: the refusal is deliberate (it matches hosted gem's redirect_gem_platform_unsupported). A platform/ABI-tagged wheel grant is now withheld from every hosted PyPI lock writer and reported as redirect_pypi_platform_wheel, instead of narrowing a cross-platform lock to one wheel. The PR carries Route Gradle digests through utils::digest #878's digest-helper fix, which becomes a no-op once Route Gradle digests through utils::digest #878 merges.

Generated by Claude Code

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review

Projects

None yet

2 participants