Skip to content

Commit e351525

Browse files
Fix Hatch environments being invisible to stale-install checks and VEX (#335) (#700)
* Find Hatch's out-of-tree project environments Hatch installs a project into envs under its data directory, never ./.venv, so stale-install checks, VEX and agent mode never looked at the environment hatch run actually uses. Model Hatch's placement rules (data dir, dirs.env.virtual, flat layouts, explicit env paths, the project id hash) and add those envs to local venv discovery. Hosted scans now warn about a stale Hatch env with the remedy that works (hatch env remove / prune), and vendored Hatch gets the same check as pypi_hatch_stale_install. A real-Hatch e2e covers both modes from an existing env through vex and the remedy. Fixes #335 Assisted-by: Claude Code:claude-opus-5-5 * Name Hatch's remedy in vendored vex and old layouts Hatch 1.0 to 1.2 keep envs at <name>-<id>/<env>, so discover that layout too. Vendored vex already warns when the installed tree is out of sync with the committed artifact; for a Hatch project the advice to re-run the install does nothing, so name hatch env remove instead. Refs #335 Assisted-by: Claude Code:claude-opus-5-5 * Document Hatch existing-env handling Explain where Hatch keeps environments, which warning each mode gives for a stale one and the remedy, and how to run the real-Hatch check. Refs #335 Assisted-by: Claude Code:claude-opus-5-5 * Match Hatch envs across symlinked path spellings On macOS /var is a symlink to /private/var, so an activated env and the discovered one can name the same directory differently, and the stale-install check then missed it. Compare resolved paths as a fallback, and leave dirs.env.virtual unresolved as Hatch does (only an env's explicit path is resolved). Refs #335 Assisted-by: Claude Code:claude-opus-5-5 * Keep Hatch envs visible under an activated venv An activated venv that is not one of Hatch's own is never used by hatch run, yet it returned from discovery before the Hatch envs were added, so a developer shell with any venv active hid the stale Hatch env again. Add Hatch's envs in that case too, and have the vendored probe judge Hatch's env prefixes directly. Refs #335 Assisted-by: Claude Code:claude-opus-5-5 * Pass the new Pipfile.lock argument in Hatch tests main added a pipenv_lock parameter to the hosted Python stale-install probe; the Hatch remedy test from this branch still called it with the old arity, so the CLI test build broke after the merge. Refs #335 Assisted-by: Claude Code:claude-opus-5-5 * Keep Hatch envs visible under a uv project env A Hatch project's pyproject alone reads as a uv project, so a set UV_PROJECT_ENVIRONMENT returned from discovery before Hatch's envs were added, hiding the env hatch run uses again. Add them on that path too. Refs #335 Assisted-by: Claude Code:claude-opus-5-5 * Add Hatch envs on every discovery path Instead of appending Hatch's envs at each early return, wrap the whole local discovery so a recorded PDM/uv env, an activated venv, Pipenv's or Poetry's resolution all keep the project's Hatch envs visible to stale-install checks, agent mode and VEX. Refs #335 Assisted-by: Claude Code:claude-opus-5-5 * Keep Hatch envs out of the Pipenv stale probe Hatch envs are now part of local discovery, so the vendored Pipenv probe judged them too and told users to fix a Hatch env with pipenv sync, which never clears it. Skip Hatch's envs there; the project's own venv still gets the Pipenv warning. Refs #335 Assisted-by: Claude Code:claude-opus-5-5 * Port #851: fix vex alias tests after #605 main is red: since the store-copy change (#605) the npm resolver already returns alias and nested-store copies, so two vex_consumed tests that assumed an alias-free set fail on main and on this branch. Same change as #851; it no-ops once main carries it. Refs #335 Assisted-by: Claude Code:claude-opus-5-5 * Judge the venvs Pipenv resolves in its stale probe Filtering every Hatch-claimed site out of the vendored Pipenv probe also dropped a venv the two share (a Hatch env with path = .venv), which pipenv sync does reinstall, so neither probe warned. Judge exactly the venvs local discovery resolves before Hatch's envs are added instead. Refs #335 Assisted-by: Claude Code:claude-opus-5-5 * Move hatch_env test helper above the tests module Fixes clippy::items_after_test_module under --all-targets. Co-Authored-By: Claude <noreply@anthropic.com> * Find Hatch matrix envs in ~/.virtualenvs When Hatch's env directory is the shared ~/.virtualenvs, only the env names the project configures were looked up there. Matrix variants such as test.py3.11 and the hatch-test.py3.X envs that `hatch test` creates were never found. A stale install in one of them therefore got no stale-install warning, and VEX could attest over it. The lookup now builds the matrix names the way Hatch does (Python variable first as py<version>, matrix-name-format, <env>. prefix except for default). It also takes hatch-test.* unless the project configures its own hatch-test env. Checked against Hatch 1.18.1's `hatch env show`. Assisted-by: Claude Code:claude-opus-5-5 * Route Gradle digests through utils::digest main has failed socket-patch-core's lib tests since Gradle support (#646) and the digest helpers (#865) both landed. The guard test production_digests_go_through_the_helpers flags three files #646 added that still hash inline: crawlers/gradle_cache.rs, patch/jvm_jar.rs and patch/sidecars/maven.rs. That breaks test, test-release and coverage on every open PR. Each inline sha1/sha256 call now goes through sha1_hex_of or sha256_hex_of, which compute the same lowercase hex. Behaviour is unchanged. Assisted-by: Claude Code:claude-opus-5-5 (cherry picked from commit 659ac2c) * Keep hatch-test.* envs when hatch-test is set A project's [tool.hatch.envs.hatch-test] table is layered over Hatch's built-in hatch-test config, so the default Python matrix still applies unless the project sets its own matrix. The ~/.virtualenvs lookup skipped hatch-test.* whenever the table existed at all, which hid those envs from the stale-install checks and VEX. It now skips them only when the project defines its own hatch-test matrix. Checked against Hatch 1.18.1's `hatch env show`. Assisted-by: Claude Code:claude-opus-5-5 * Format the Hatch discovery changes rustfmt the two files this branch touches so they match the repository's formatting; no behavior change. Assisted-by: Claude Code:claude-opus-5-5 * Resolve .. before the Hatch in-project check A relative [dirs.env] virtual such as ../envs still started with the project path, so discovery treated it as an in-project flat directory. It then claimed every venv in the parent directory and missed the nested <name>/<id>/<env> envs Hatch actually uses there. Hatch tests `root in data_directory.resolve().parents`, so the directory is now resolved like Python's Path.resolve() first, and the project root itself no longer counts as inside the project. Assisted-by: Claude Code:claude-opus-5-5 * Fold .. in Hatch env paths on Windows too canonicalize returns verbatim \\?\ paths on Windows. In those, `/` is not a separator and the OS does not fold `..`, so a virtual = "../envs" joined onto the project root stayed one opaque component. It still counted as inside the project, and reading the directory found nothing. Config paths are now joined component by component, and resolve() folds `.` and `..` before it canonicalizes the longest existing prefix. Assisted-by: Claude Code:claude-opus-5-5 --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 2519aa7 commit e351525

9 files changed

Lines changed: 1638 additions & 31 deletions

File tree

‎crates/socket-patch-cli/CLI_CONTRACT.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -191,7 +191,7 @@ The hidden alias `--no-apply` on `get --save-only` is **part of the contract**
191191

192192
`repair` keeps its `gc` visible alias.
193193

194-
**Python stale-install guard**: after a hosted redirect, `scan` / `get` use the Python crawler to inspect every matching installed package, including Poetry's out-of-tree virtualenvs and `--global-prefix`. A readable file that differs from the patch's `afterHash` emits `redirect_pypi_stale_install` in JSON `redirect.warnings[]` and human stderr. The probe changes no installed files, re-runs on idempotent scans, and falls back to persisted patch records when fresh record fetching fails. Missing/unreadable files alone do not prove staleness; lock-only checkouts stay quiet. Dry runs skip the probe. Same-run VEX excludes positively stale Python packages (qualifier-insensitive), even with `--vex-no-verify` or a healthy copy in another interpreter; if nothing remains to attest, the command exits 1 with `no_applicable_patches`. Reinstall from the rewritten lock in the affected interpreter and verify with `socket-patch vex`.
194+
**Python stale-install guard**: after a hosted redirect, `scan` / `get` use the Python crawler to inspect every matching installed package, including Poetry's out-of-tree virtualenvs, the project's Hatch environments (Hatch's data dir / `HATCH_DATA_DIR`, `[dirs.env] virtual`, explicit env `path`s) and `--global-prefix`. A readable file that differs from the patch's `afterHash` emits `redirect_pypi_stale_install` in JSON `redirect.warnings[]` and human stderr. The probe changes no installed files, re-runs on idempotent scans, and falls back to persisted patch records when fresh record fetching fails. Missing/unreadable files alone do not prove staleness; lock-only checkouts stay quiet. Dry runs skip the probe. Same-run VEX excludes positively stale Python packages (qualifier-insensitive), even with `--vex-no-verify` or a healthy copy in another interpreter; if nothing remains to attest, the command exits 1 with `no_applicable_patches`. Reinstall from the rewritten lock in the affected interpreter and verify with `socket-patch vex`. A stale Hatch environment instead names `hatch env remove <env>` / `hatch env prune`: Hatch's pip installer (and uv before Hatch 1.16) keeps a same-version release, so only a recreated env picks up the patch. The same Hatch envs are what agent mode patches and `vex` judges for a Hatch project.
195195

196196
### socket.yml patch policy (v5.0)
197197

@@ -329,7 +329,7 @@ Contract details:
329329
* **JSON success surface**: `apply` adds a top-level `vex` object to its envelope; `scan` adds a top-level `vex` key to its result. Both carry `{ path, statements, format: "openvex-0.2.0" }`.
330330
* `apply`'s no-manifest early exit (the `noManifest` success no-op; v5.0: its human line is `No patch manifest found; nothing to apply.` — it names the missing `.socket/manifest.json`, not the folder, since `.socket/` may legitimately hold vendored state) and `vendor`'s (`No manifest found, nothing to vendor.` — a project with hosted pins ejects instead, v5.0) still generate the document from the lockfiles and the vendor ledger (manifest-less VEX: hosted / vendored checkouts carry no manifest). Nothing referenced anywhere keeps the calm exit 0 (a stale document at the path is removed; `--json` carries any discovery diagnostics in `warnings[]`); any other VEX failure fails the command with exit 1 — including a run whose only candidates are omitted `record_unavailable` (an `--offline` run over a lockfile-wired checkout with no local records), so an ambient `SOCKET_VEX` there fails the install. `--dry-run` skips generation on both, and so does `apply --check` — it stays read-only and offline-safe, leaving the output path untouched. `scan` has no such early exit: with no manifest and nothing wired anywhere its `--vex` fails with `manifest_not_found`.
331331
* **Stale-doc removal (v3.5)**: a run that ends in a VEX error removes a recognizably-OpenVEX file (JSON whose `@context` names openvex.dev) already sitting at the output path — a pipeline reusing one path can never ship yesterday's attestation for a now-unpatched tree. Unrelated files at the path are never touched; a mid-write partial that no longer parses as JSON is left for downstream parsers to reject loudly.
332-
* **Additive warnings (v3.5)**: `product_not_iri` (the `--product`/`--vex-product` override is neither a `pkg:` purl nor an absolute IRI; honored verbatim, warned) and `vendored_tree_out_of_sync` (a healthy vendored attestation stands on the committed artifact + lock wiring while the PRESENT installed tree hash-mismatches the patched bytes — run the package manager's install; the attestation itself is unchanged). Both ride stderr in human mode and `warnings[]` in the standalone `vex --json` envelope. Same channel for `product_multiple_manifests` (auto-detect found several project manifests and names the one it used), `vex_stale_doc_removed` (the stale-doc removal above happened), the manifest-less plan's advisories — `vex_wiring_conflict` (the lockfiles wire a package to different patches: which files, which uuids, how to fix it), `vex_record_superseded` (a recorded patch replaced by the lockfile-wired one), `vex_claim_unwired` (a ledger claim whose patch the lockfiles still mention, but not as wiring), `vex_record_offline` / `vex_record_not_found` / `vex_record_fetch_failed` (why a lockfile-wired patch has no record — the detail behind a `record_unavailable` skip) and `api_auth_fallback` (the authenticated API refused the credentials and the public proxy served free patches only; `get` / `scan`'s warning text) — and, standalone only, `org_looks_like_path` (`-o`/`--org` given a file-shaped value — `-O` is `--output`). The standalone error envelope carries `warnings[]` too. An embedded `--vex` that fails also folds each omitted patch into the host command's `warnings[]` as `vex_omitted` (`<purl>: <why> (<errorCode>)` — standalone `vex` lists them as `skipped` events), and `--silent` lists them as `omitted: <purl> (<errorCode>)` lines under the error. A corrupt `.socket/vendor/state.json` is no longer degraded with a warning: every form of vex fails with `vendor_ledger_corrupt` (see the error-code table). A malformed pre-v5 `redirect-state.json` is, as of v5.0, only the `redirect_ledger_corrupt` warning (hosted mode keeps no ledger; the file is an optional migration record source).
332+
* **Additive warnings (v3.5)**: `product_not_iri` (the `--product`/`--vex-product` override is neither a `pkg:` purl nor an absolute IRI; honored verbatim, warned) and `vendored_tree_out_of_sync` (a healthy vendored attestation stands on the committed artifact + lock wiring while the PRESENT installed tree hash-mismatches the patched bytes — run the package manager's install (for a project with Hatch environments the detail also names `hatch env remove <env>`, since Hatch keeps an installed release); the attestation itself is unchanged). Both ride stderr in human mode and `warnings[]` in the standalone `vex --json` envelope. Same channel for `product_multiple_manifests` (auto-detect found several project manifests and names the one it used), `vex_stale_doc_removed` (the stale-doc removal above happened), the manifest-less plan's advisories — `vex_wiring_conflict` (the lockfiles wire a package to different patches: which files, which uuids, how to fix it), `vex_record_superseded` (a recorded patch replaced by the lockfile-wired one), `vex_claim_unwired` (a ledger claim whose patch the lockfiles still mention, but not as wiring), `vex_record_offline` / `vex_record_not_found` / `vex_record_fetch_failed` (why a lockfile-wired patch has no record — the detail behind a `record_unavailable` skip) and `api_auth_fallback` (the authenticated API refused the credentials and the public proxy served free patches only; `get` / `scan`'s warning text) — and, standalone only, `org_looks_like_path` (`-o`/`--org` given a file-shaped value — `-O` is `--output`). The standalone error envelope carries `warnings[]` too. An embedded `--vex` that fails also folds each omitted patch into the host command's `warnings[]` as `vex_omitted` (`<purl>: <why> (<errorCode>)` — standalone `vex` lists them as `skipped` events), and `--silent` lists them as `omitted: <purl> (<errorCode>)` lines under the error. A corrupt `.socket/vendor/state.json` is no longer degraded with a warning: every form of vex fails with `vendor_ledger_corrupt` (see the error-code table). A malformed pre-v5 `redirect-state.json` is, as of v5.0, only the `redirect_ledger_corrupt` warning (hosted mode keeps no ledger; the file is an optional migration record source).
333333

334334
### VEX provenance markers (contract)
335335

@@ -1319,6 +1319,7 @@ Every `--json` invocation emits a single JSON object that follows the **unified
13191319
| `pypi_poetry_symlink_unsupported` / `pypi_pipenv_symlink_unsupported` / `pypi_requirements_symlink_unsupported` | `failed` | vendor (pypi, v5.0): a target file (`pyproject.toml` / `poetry.lock`, `Pipfile` / `Pipfile.lock`, or any planned `requirements*.txt`) is a symlink — refused before any write on wire AND on revert (the revert keeps the artifact, `kept_artifact`); the twins of the existing pdm/uv symlink refusals. |
13201320
| `pypi_poetry_changed` / `pypi_pdm_changed` / `pypi_pipenv_changed` / `pypi_uv_changed` | `failed` | vendor (pypi, v5.0): the lock / project file changed between the read that planned the edit and the first write — refused before any write (worded like `pypi_lock_changed`: "<file> changed during vendoring; re-run"). |
13211321
| `pypi_pipenv_stale_install` | `skipped` (warning) | vendor (pipenv): the vendored twin of `redirect_pypi_stale_install` — the project's venv still holds the upstream release Pipenv will not reinstall over; the detail names the `pipenv run pip uninstall -y <pkg> && pipenv sync` remedy, with the same lock-category `sync` arguments as the hosted warning. |
1322+
| `pypi_hatch_stale_install` | `skipped` (warning) | vendor (hatch): an existing Hatch environment of the project (found under Hatch's data dir, `dirs.env.virtual` or an explicit env `path`) still holds the upstream release; Hatch keeps it on the next `hatch run`, so the detail names the env and the `hatch env remove <env>` / `hatch env prune` remedy. |
13221323
| `pypi_pipenv_installer_unknown` | `skipped` (warning) | vendor (pipenv): no `pipenv` answered on PATH; the vendored references assume Pipenv 2018 or later (7–11 cannot consume them — use hosted mode there); `SOCKET_PIPENV_MAJOR` pins the release. |
13231324
| `vendor_lock_entry_relocked` | revert `warnings[]` | vendor `--revert` / rollback (pipenv): a relock regenerated the wired entry to a registry reference, or removed it; the record is retired (artifact removed, ledger entry dropped) instead of drift-kept. |
13241325
| `pypi_{poetry,pdm,pipenv}_no_lockfile` | `failed` | vendor (pypi): a lock-less tool marker with no `requirements.txt` fallback — run `<tool> lock`. |

‎crates/socket-patch-cli/src/commands/scan/hosted/python.rs‎

Lines changed: 63 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,13 @@ pub(super) async fn stale_install_warnings(
4949
// fallback to the global interpreters would judge an unrelated Python's
5050
// copy of the release (a tool venv on PATH) and warn falsely.
5151
// --global / --global-prefix keep their meaning.
52+
// Hatch envs need their own remedy: a reinstall from the rewritten
53+
// pyproject does nothing there (#335).
54+
let hatch_envs = if common.is_global() {
55+
Vec::new()
56+
} else {
57+
socket_patch_core::crawlers::hatch_env::hatch_environments(&common.cwd).await
58+
};
5259
let paths = if common.is_global() {
5360
crawler
5461
.get_site_packages_paths(&common.crawler_options())
@@ -110,7 +117,11 @@ pub(super) async fn stale_install_warnings(
110117
// (`install`, `install --deploy`, `sync` all keep the installed
111118
// bytes on every major), and `pipenv uninstall` rewrites the
112119
// Pipfile and re-locks the patch away — name the verified remedy.
113-
let remedy = if pipenv_purls.contains(&purl) {
120+
let hatch_env =
121+
socket_patch_core::crawlers::hatch_env::environment_of(&hatch_envs, &site);
122+
let remedy = if let Some(env) = hatch_env {
123+
socket_patch_core::crawlers::hatch_env::stale_install_remedy(&env.name)
124+
} else if pipenv_purls.contains(&purl) {
114125
let name = strip_purl_qualifiers(&purl)
115126
.strip_prefix("pkg:pypi/")
116127
.and_then(|rest| rest.split('@').next())
@@ -221,6 +232,57 @@ mod tests {
221232
assert!(out.warnings.is_empty());
222233
}
223234

235+
/// #335: a Hatch env keeps the upstream release after the rewrite, and
236+
/// the probe must find it (Hatch keeps envs out of `./.venv`) and name
237+
/// Hatch's remedy, not "reinstall from the rewritten lock".
238+
#[tokio::test]
239+
async fn hatch_env_gets_the_stale_install_warning_with_hatch_remedy() {
240+
let tmp = tempfile::tempdir().unwrap();
241+
let project = tmp.path().join("app");
242+
std::fs::create_dir_all(&project).unwrap();
243+
std::fs::write(
244+
project.join("pyproject.toml"),
245+
"[project]\nname = \"app\"\nversion = \"0.1.0\"\ndependencies = [\"six==1.16.0\"]\n\n[tool.hatch.envs.default]\npath = \"../hatch-envs/app\"\n",
246+
)
247+
.unwrap();
248+
let env = tmp.path().join("hatch-envs").join("app");
249+
std::fs::create_dir_all(&env).unwrap();
250+
std::fs::write(env.join("pyvenv.cfg"), "home = /usr/bin\n").unwrap();
251+
let site = if cfg!(windows) {
252+
env.join("Lib").join("site-packages")
253+
} else {
254+
env.join("lib").join("python3.12").join("site-packages")
255+
};
256+
let dist = site.join("six-1.16.0.dist-info");
257+
std::fs::create_dir_all(&dist).unwrap();
258+
std::fs::write(dist.join("METADATA"), "Name: six\nVersion: 1.16.0\n").unwrap();
259+
std::fs::write(site.join("six.py"), b"upstream").unwrap();
260+
261+
let common = crate::args::GlobalArgs {
262+
cwd: project.clone(),
263+
..Default::default()
264+
};
265+
let purl = "pkg:pypi/six@1.16.0";
266+
let confirmed = vec![(purl.to_string(), "six-uuid".to_string())];
267+
let ledger = BTreeMap::from([("k".into(), record("six-uuid", "six.py", b"patched"))]);
268+
let out =
269+
stale_install_warnings(&common, &confirmed, &BTreeSet::new(), None, &ledger).await;
270+
assert_eq!(out.stale_purls, BTreeSet::from([purl.to_string()]));
271+
assert_eq!(out.warnings.len(), 1);
272+
let detail = out.warnings[0]["detail"].as_str().unwrap();
273+
assert!(detail.contains("hatch env remove default"), "{detail}");
274+
assert!(
275+
!detail.contains("Reinstall from the rewritten lock"),
276+
"{detail}"
277+
);
278+
279+
// Patched in the env: nothing to warn about.
280+
std::fs::write(site.join("six.py"), b"patched").unwrap();
281+
let out =
282+
stale_install_warnings(&common, &confirmed, &BTreeSet::new(), None, &ledger).await;
283+
assert!(out.warnings.is_empty());
284+
}
285+
224286
/// A legacy `.egg-info` install (pip < 23.1 building an sdist without
225287
/// `wheel`) is a real copy pip keeps on `install -r`, so the hosted
226288
/// stale-install guard must judge it like a `.dist-info` one (#447).

‎crates/socket-patch-cli/src/commands/vex.rs‎

Lines changed: 21 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -776,6 +776,26 @@ async fn generate_vex(
776776
// installed tree is present and running different bytes. Say so — a
777777
// build that bypasses the vendor wiring is unpatched until the next
778778
// package-manager install.
779+
// A Hatch env is never resynced by an install: Hatch keeps a present
780+
// release (#335), so name the remedy that recreates it.
781+
let hatch_note = if outcome.vendored_out_of_sync.is_empty() || common.is_global() {
782+
String::new()
783+
} else {
784+
match socket_patch_core::crawlers::hatch_env::hatch_environments(&common.cwd)
785+
.await
786+
.as_slice()
787+
{
788+
[] => String::new(),
789+
envs => format!(
790+
" A Hatch environment keeps an installed release on the next `hatch run`; \
791+
recreate it instead ({}).",
792+
envs.iter()
793+
.map(|env| format!("`hatch env remove {}`", env.name))
794+
.collect::<Vec<_>>()
795+
.join(", ")
796+
),
797+
}
798+
};
779799
for purl in &outcome.vendored_out_of_sync {
780800
note_warning(
781801
warnings,
@@ -785,7 +805,7 @@ async fn generate_vex(
785805
"{purl}: the installed tree does not match its vendored artifact; the \
786806
attestation is based on the committed .socket/vendor artifact (the lockfile \
787807
consumes it), but the live tree carries different bytes — re-run your \
788-
package manager's install to resync it."
808+
package manager's install to resync it.{hatch_note}"
789809
),
790810
);
791811
}

0 commit comments

Comments
 (0)