Fix pnpm lock/workspace readers missing a BOM (#903, #904, #905)

[Mikola Lysenko (mikolalysenko)]

LLM Description written by Claude Code:claude-opus-5-5

Fixes #903
Fixes #904
Refs #905 (this PR does steps 1 and 2 of its proposed change; step 3, re-pointing the ~50 inline strip_prefix('\u{feff}') sites, stays open as the issue's follow-up slice)

Summary

Windows editors that save "UTF-8 with signature" put a BOM at the start of pnpm-lock.yaml / pnpm-workspace.yaml. pnpm reads those files normally, but socket-patch's pnpm readers matched column-0 text and missed the first line:

• Hosted pnpm scan skips the trustLockfile: true auto-config when pnpm-lock.yaml starts with a UTF-8 BOM, so pnpm 11/12 frozen installs fail with ERR_PNPM_TARBALL_URL_MISMATCH after a successful scan #903: on a BOM lock, the hosted trust gate saw no lockfileVersion, so it skipped the trustLockfile: true auto-config and pnpm 11/12 frozen installs failed. rollback / remove / list then refused the lock the scan had just pinned ("not a pnpm lockfile"), and vendored mode refused it with a false "no lockfileVersion … re-lock" remedy.
• pnpm-workspace.yaml with a UTF-8 BOM: hosted and vendored miss the first top-level key and append a duplicate trustLockfile / overrides, so every pnpm install fails with "duplicate mapping key" after a successful scan #904: a BOM-prefixed first key in pnpm-workspace.yaml (trustLockfile: or overrides:) wasn't recognised. Hosted and vendored then appended a duplicate key, which pnpm refuses to parse, so every pnpm command broke. An explicit trustLockfile: false was also overridden.

Root cause

formats::pnpm had no BOM handling. head_lock_version, lock_versions (behind lock_version_major and may_need_store_flag), is_pnpm_lock_text, unsupported_early_shrinkwrap and workspace::top_level_key all match a column-0 literal. The crate has no shared BOM helper (#905): four named copies plus ~50 inline strips, and each new reader decides for itself.

Changes

• New formats::text::{split_bom, strip_bom}, which treats exactly one leading BOM as encoding. The four named copies (utils::serde::strip_bom, gradle::dsl::strip_bom, formats::yarn::strip_bom, cargo_manifest::split_bom) are deleted and their callers re-pointed.
• The pnpm lock sniffs read strip_bom(text). A new formats::pnpm::is_shrinkwrap_lock replaces the engine's inline shrinkwrapVersion: check.
• workspace::top_level_key skips a leading BOM, and so does block_insert_point on line 0. The splices keep the original line, so the BOM stays byte-exact on write and on revert.
• governing_root::workspace_lockfile_dir now reads through top_level_key, which deletes its private key grammar (pnpm lock and workspace readers don't skip a leading BOM, because BOM handling has no shared helper (4 named copies, ~50 inline strips) #905 step 2).
• The npm/PyPI/gem wrappers need no change; this is Rust-internal.
• CI port: cherry-picked Route Gradle digests through utils::digest #878's 659ac2c (7515fa7) so that coverage stops failing production_digests_go_through_the_helpers, which is red on main. It becomes a no-op once Route Gradle digests through utils::digest #878 merges.

Per-issue test mapping (red on main, green here)

Issue	Test	Without fix
#903	formats::pnpm::tests::bom_lock_reads_like_its_plain_twin (9.0, 6.0, 5.4, 5.2 and shrinkwrap twins: is_pnpm_lock, sniff_lock_grammar, lock_version_major, may_need_store_flag, check_v9_lock_version, entries)	FAILED
#903	vendor::pnpm_lock::tests::bom_lock_vendors_and_reverts_byte_exact	FAILED: Refused vendor_lockfile_version_unsupported: … has no lockfileVersion in its head
#903, #904	CLI in_process_redirect_pnpm::hosted_bom_lock_and_workspace_read_like_their_plain_twins (scan creates the trust scaffold, rollback restores the BOM lock byte-exact, a BOM trustLockfile: false/true is kept and not duplicated)	n/a (added after the fix; it covers the same readers)
#904	formats::pnpm::workspace::tests::top_level_key_skips_a_leading_bom	FAILED
#904	hosted::guidance::tests::workspace_trust_plan_reads_a_bom_first_key	FAILED
#904	vendor::pnpm_lock::tests::bom_workspace_override_inserted_beside_existing_not_duplicated	FAILED
#905	formats::text::tests::*, hosted::governing_root::tests::workspace_lockfile_dir_reads_the_top_level_key (extended)	refactor coverage

Verification

• CI on 7515fa7: all 547 check runs completed (541 success, 6 skipped, none failed or pending). Bugbot's review of 7515fa7 found no new issues, and there are no review threads.
• cargo clippy --workspace --all-features -- -D warnings: clean.
• cargo test -p socket-patch-core --all-features: the only local failures are 4 permission-denial tests that can't fail as uid 0. They fail identically on origin/main in this sandbox and pass in CI.
• cargo test -p socket-patch-cli --all-features --lib and the in_process_redirect_pnpm, in_process_rollback_hosted, hosted_memory_{engine,parity,rollout}, e2e_safety_pnpm, e2e_vendor_pnpm_build and e2e_npm suites: all pass locally.
• cargo fmt --all -- --check already fails on main with this toolchain, and CI doesn't gate on it. I formatted only the changed lines and left unrelated files alone.

Follow-ups

• pnpm lock and workspace readers don't skip a leading BOM, because BOM handling has no shared helper (4 named copies, ~50 inline strips) #905 step 3: re-point the ~50 inline BOM strips to formats::text.

🤖 Generated with Claude Code

https://claude.ai/code/session_0167b49WbzczWBJxsnUhCNEy

---

Note

Medium Risk
Changes lock/workspace detection and trust-config editing paths used by hosted scan and vendored revert; behavior is narrowly scoped to BOM-prefixed files but affects install-critical YAML edits.

Overview
Fixes BOM-blind pnpm lock/workspace parsing so Windows “UTF-8 with signature” files behave like pnpm does (#903, #904).

Adds shared formats::text::{split_bom, strip_bom} and routes existing BOM helpers (serde, Gradle DSL, Yarn, Cargo manifest) through it. pnpm lock sniffing (lockfileVersion, shrinkwrap detection, is_pnpm_lock_text) and workspace top-level key parsing now strip a leading BOM before matching column-0 text, while splices keep the BOM byte-exact on write/revert. workspace_lockfile_dir reads lockfileDir via the same top_level_key grammar.

Hosted/vendored behavior: BOM locks get trustLockfile: true when appropriate, rollback works, explicit BOM trustLockfile: false is not duplicated, and overrides merge beside existing sections instead of appending duplicate keys.

New unit, vendor, and CLI integration tests cover BOM lock/workspace scenarios.

^{Reviewed by Cursor Bugbot for commit 1607f5d. Configure here.}

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

BugBot review

---

Generated by Claude Code

[cursor]

Stale Bugbot comment from a previous run.

[Mikola Lysenko (mikolalysenko)]

[agent] coverage failed on 7235dc3 in utils::digest::tests::production_digests_go_through_the_helpers, an architecture guard that is red on main too: three Gradle files from #646 hash inline, but #865's guard requires them to use the utils::digest helpers. That isn't this PR's change. The fix is #878, so I cherry-picked its commit (659ac2c → 7515fa7) here. It becomes a no-op once #878 lands. The guard test passes locally after the port.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

BugBot review

---

Generated by Claude Code

[cursor]

Stale Bugbot comment from a previous run.

[Mikola Lysenko (mikolalysenko)]

[burn-down agent] Ready for review at head 7515fa7 (0 behind main).

• CI: 547/547 check runs completed, 0 failed/pending (541 success, 6 skipped).
• Bugbot: reviewed 7515fa7, no new issues; no unresolved review threads.
• Reviewer note: includes a cherry-pick of Route Gradle digests through utils::digest #878's Gradle digest fix (7515fa7) to keep coverage green; it no-ops once Route Gradle digests through utils::digest #878 merges.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

bugbot run

---

Generated by Claude Code

[cursor]

Stale Bugbot comment from a previous run.

[Mikola Lysenko (mikolalysenko)]

bugbot run

---

Generated by Claude Code

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 1607f5d. Configure here.}