diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index dff38f2c8..695d78627 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -651,7 +651,7 @@ to **six flavors**. | eco / flavor | vendored artifact | committed wiring | consumption proof | |---|---|---|---| -| npm (package-lock) | deterministic patched tarball `[@scope/]-.tgz` | `package-lock.json` only (`npm-shrinkwrap.json` wins when present): every entry matching name+version gets `resolved: "file:…"` + recomputed `integrity`. `package.json` untouched | `npm ci` (integrity-verified). Plain `npm install` preserves the entry; `npm update ` re-resolves and drops it | +| npm (package-lock) | deterministic patched tarball `[@scope/]-.tgz`, plus `/.gitignore` (re-includes the tarball against the project's ignores, such as Node.gitignore's `*.tgz`) and `/.gitattributes` (`-text`); every tarball flavor below writes the same pair and refuses `vendor_artifact_gitignored` when git would still drop the tarball | `package-lock.json` only (`npm-shrinkwrap.json` wins when present): every entry matching name+version gets `resolved: "file:…"` + recomputed `integrity`. `package.json` untouched | `npm ci` (integrity-verified). Plain `npm install` preserves the entry; `npm update ` re-resolves and drops it | | npm / yarn classic | (same tarball) | `yarn.lock` only: matching blocks get `resolved "file:./…#"` + `integrity` (both checksums recomputed; merged-key & `npm:`-alias blocks covered) | `yarn install --frozen-lockfile --offline` (sha1 fragment + sha512 SRI both enforced; byte-stable lock) | | npm / yarn berry (node-modules linker) | (same tarball) | root `package.json` `resolutions` + `yarn.lock` entry with `checksum: 10c0/` of the berry cache-zip (reproduced from the tarball offline). **PnP is refused** (`.pnp.*` → different artifact pipeline) | `yarn install --immutable --check-cache`, cold cache. Refused if `__metadata.cacheKey ≠ 10c0` or a non-default `compressionLevel`. Both files keep their own layout — a CRLF lock (yarn's output on Windows) is spliced in CRLF, `package.json` is re-serialized with its BOM, indent, line ending and trailing-newline shape — so vendor + `--revert` round-trip byte-exactly; a lock or `package.json` MIXING CRLF and LF is refused before any write (`vendor_yarn_berry_mixed_line_endings`) | | npm / pnpm (lockfileVersion 9) | (same tarball) | root `package.json` `pnpm.overrides` (versioned selector) **+** `pnpm-lock.yaml` surgery (overrides / importer version / packages `resolution.integrity` / snapshots) | `pnpm install --frozen-lockfile --offline`, cold store (integrity-verified; byte-stable on pnpm 9 & 10). Other lockfileVersions: 5.4/6.0 route to the legacy backend below; anything else refused | @@ -1191,7 +1191,7 @@ Every `--json` invocation emits a single JSON object that follows the **unified | `already_vendored` | `skipped` | vendor: artifact + wiring already in sync for this patch uuid. | | `unsafe_coordinates` | `failed` | vendor: purl/uuid would escape `.socket/vendor/` (tampered manifest/state); refused before any write. | | `revert_failed` | `failed` | vendor --revert: a recorded entry could not be reverted. | -| `vendor_ledger_missing` | `failed` (artifact-level: `uuid` + `details.{ecosystem,path}`, no purl) | repair (v5.0): a lockfile references `.socket/vendor///` but the vendor ledger has no entry for it; repair no longer rebuilds ledger entries from lockfiles. Recovery: restore `.socket/vendor/state.json` from version control and re-run `repair`, or `git checkout -- ` and re-vendor. | +| `vendor_ledger_missing` | `failed` (artifact-level: `uuid` + `details.{ecosystem,path}`, no purl) | repair (v5.0) and `vendor --check`: a lockfile references `.socket/vendor///` but the vendor ledger has no entry for it; repair no longer rebuilds ledger entries from lockfiles. Recovery: restore `.socket/vendor/state.json` from version control and re-run `repair`, or `git checkout -- ` and re-vendor. | | `vendor_wiring_unknown_revert_blocked` | `skipped` (beside the `failed`/`revert_failed` event) | vendor --revert: the ledger entry was reconstructed by a pre-v5 `repair` without wiring records and the live lockfile still resolves through the artifact — the revert refuses (fail-closed) instead of deleting a tarball the lock points at. Recovery: `socket-patch repair`, then restore the pre-vendor lock (or re-lock without the override) and re-run the revert. repair: an npm ledger entry whose `flavor` this release does not know (written by a newer socket-patch) is skipped, never health-checked or rebuilt, and the artifact, wiring and ledger stay as found (a lone `skipped` event; the run's exit is unaffected). Recovery: upgrade socket-patch. | | `stale_install` | `skipped` | vex (in-run `scan --mode hosted --vex`): a hosted stale-install probe found positively unpatched installed bytes, so the purl is omitted even under `--vex-no-verify` (see the gem / Python stale-install guards). | | `record_unavailable` | `skipped` | vex (manifest-less): a lockfile-wired patch has no local record (manifest, this run's hosted records or a pre-v5 redirect ledger, vendor ledger) and none could be fetched — `--offline`, transport error, 404, or a refused (paid) patch. Omitted, never attested from the `socket-patch.vendor.json` marker. | @@ -1238,8 +1238,8 @@ Every `--json` invocation emits a single JSON object that follows the **unified | `vendor_vlt_legacy_lockfile` | `skipped` (warning) | vendor (vlt): an era-A lock (vlt 0.0.0-19 … 1.0.0-rc.8): a `··` default-registry id, or default-registry ids that are URL segments equal to a scalar `options.registry` with no `·npm·` id (era B writes `·npm·` whatever the scalar). vlt 0.0.0-31 … 1.0.0-rc.5 install the vendored lock but fail to reinstall the vendored `file:` dependency if `vlt-lock.json` is deleted and re-created (the other era-A releases reinstall it; the lock does not say which release reads it). The package is still vendored; remedy: upgrade vlt. | | `vendor_vlt_reinstall_required` | `skipped` (advisory; human: `Warning: …`) | vendor / scan / get `--mode vendored` (vlt), wet and dry runs, and in-sync reruns: (a) the run rewires an optional dependency, or an importer's `node_modules/` of an optional dependency still resolves into `node_modules/.vlt/`: from vlt 0.0.0-30 a plain `vlt install` (1.2.0: also `--force`) keeps that installed upstream copy linked; the detail says to run `vlt ci` (or delete `node_modules` and run `vlt install`) to link the vendored copy, and that vlt 0.0.0-30 … 1.0.4 install no optional dependency from the lock of a project that declares only optional dependencies (upgrade to 1.0.5 or later first); (b) otherwise, an importer's link of the dependency still resolves into `node_modules/.vlt/`: the detail names the links (`node_modules/`, `/node_modules/`) and says `vlt install` (or `vlt ci`) links the vendored copy — on a warm tree after a plain `vlt install` that is true of every vendored direct dependency; (c) an importer's link resolves into the vendored dir of the patch this run replaces (a new patch uuid), which the run removes: the detail names the links and says `vlt install` (or `vlt ci`) links the new vendored copy; (d) a redownload of the payload (vendor, or `repair` after a corrupt or missing payload) could not keep vlt's links to the package's own dependencies (its old `node_modules/` held more than links): the detail says to run `vlt ci` (or delete `node_modules` and run `vlt install`), since a plain `vlt install` does not re-link them. `repair` moves those links back into the downloaded payload when they are only links. The package is vendored either way; a run whose patch fails to apply emits neither. A wet `vendor --revert` (and the revert a vendored → hosted takeover runs, whose advisory joins `redirect.warnings[]`): (a) the revert moves an `optionalDependencies` spec back from the `file:` dir, or an optional importer's `node_modules/` still resolves into the vendored uuid dir: from vlt 0.0.0-30 a plain `vlt install` keeps that link (dangling once the dir is removed), so the detail says to run `vlt ci` (or delete `node_modules` and run `vlt install`) to link the restored copy, with the same vlt 1.0.5 note; (b) otherwise, an importer's link still resolves into the vendored uuid dir: the detail names the links and says `vlt install` (or `vlt ci`) links the restored copy. A dry-run revert emits neither. | | `vendor_flavor_changed` | `failed` | vendor (npm): the purl's vendor ledger entry was written for another lockfile `flavor` than the one the router now detects (for example `npm` → `vlt` after switching package managers). Remedy: `socket-patch vendor --revert` it first, then re-vendor. Refused before any write. | -| `vendor_artifact_gitignored` | `failed` | vendor (vlt): inside a git work tree, `git check-ignore --no-index` reports the new artifact's uuid directory as ignored by a rule its own `.gitignore` cannot override (such as a root `.socket/` rule; the detail names the rule). Remedy: drop that rule for `.socket/vendor/`. Refused before any write. | -| `vendor_artifact_gitignore_unchecked` | warning | vendor (vlt): git is installed but could not answer the ignore check for the written vendored directory (it failed to start, ran past 30 s, or `rev-parse` / `check-ignore` exited with an error); the package is vendored and the detail names what failed. Remedy: make sure no ignore rule covers `.socket/` before committing. Git absent, or a project outside any work tree, raises nothing. | +| `vendor_artifact_gitignored` | `failed` | vendor (vlt and the npm-family tarball flavors: npm, pnpm, bun, yarn classic, yarn berry): inside a git work tree, `git check-ignore --no-index` reports the new artifact's uuid directory as ignored by a rule its own `.gitignore` cannot override (such as a root `.socket/` or `vendor/` rule; the detail names the rule). Remedy: drop that rule for `.socket/vendor/`. Refused before any write. A file rule such as `*.tgz` is overridden by the `/.gitignore` vendoring writes; if the written artifact still reads as ignored, the run refuses and removes the uuid dir it created. | +| `vendor_artifact_gitignore_unchecked` | warning | vendor (vlt and the npm-family tarball flavors): git is installed but could not answer the ignore check for the written vendored directory (it failed to start, ran past 30 s, or `rev-parse` / `check-ignore` exited with an error); the package is vendored and the detail names what failed. Remedy: make sure no ignore rule covers `.socket/` before committing. Git absent, or a project outside any work tree, raises nothing. | | `vendor_ledger_entry_missing` | `failed` | vendor (vlt): the only installed copy is vlt's link to a committed vendored directory, but the vendor ledger has no entry for the package; restore `.socket/vendor/state.json` from version control (v5.0: `repair` no longer re-synthesizes it). Replaces the `package_not_installed` skip. | | `vendor_variant_ambiguous` | `failed` | vendor / scan / get `--mode vendored` (pypi, gem): the package is not installed and the manifest holds several release variants of it (`?artifact_id=` / `?platform=`), none of which the vendor ledger records, so nothing says which distribution to vendor; install the package or keep one release variant. A variant the ledger records (at any patch uuid) is taken as the wired one and its siblings are left out without an event. | | `vendor_artifact_missing` | reason | The recorded artifact is missing; repair requires an online exact redownload. | @@ -1670,7 +1670,11 @@ wiring: an entry whose lockfile or config no longer references its `package-lock.json` / `npm-shrinkwrap.json` entry for the vendored `name@version` that `vendor` would rewire but that does not resolve to the vendored artifact (#588); the reason names that entry. Missing ledger entries fail with -`vendor_ledger_missing`. Offline upstream metadata is reported as the run warning +`vendor_ledger_missing`: a manifest patch with no ledger entry, and a lockfile +reference to `.socket/vendor///` that no ledger entry owns (the +artifact-level event repair emits: `uuid` plus `details.{ecosystem,path}`, no +purl), so a checkout whose ledger was ignored or dropped from the commit no +longer passes with nothing to check. Offline upstream metadata is reported as the run warning `vendor_jvm_upstream_unverified`. The check never starts an API client or writes lock/recovery files. `--check` conflicts with `--revert`. diff --git a/crates/socket-patch-cli/src/commands/vendor.rs b/crates/socket-patch-cli/src/commands/vendor.rs index 0289bf946..68e587156 100644 --- a/crates/socket-patch-cli/src/commands/vendor.rs +++ b/crates/socket-patch-cli/src/commands/vendor.rs @@ -1060,11 +1060,13 @@ async fn run_check(args: &VendorArgs) -> i32 { } env.record(event); } - for key in manifest + let mut unledgered_uuids: HashSet<&str> = HashSet::new(); + for (key, record) in manifest .patches - .keys() - .filter(|k| !state.entries.contains_key(*k)) + .iter() + .filter(|(k, _)| !state.entries.contains_key(*k)) { + unledgered_uuids.insert(record.uuid.as_str()); if !args.common.json { eprintln!("{key}: patch has no vendored ledger entry"); } @@ -1073,6 +1075,38 @@ async fn run_check(args: &VendorArgs) -> i32 { "patch has no vendored ledger entry", )); } + // A project file still wired to a vendored artifact the ledger does not + // know (the ledger was ignored or dropped from the commit along with the + // manifest) leaves every fresh install failing; the manifest keys above + // cannot see it, so the references are read from the wiring itself. + let references = + crate::commands::vendored_backend::repair::scan_vendor_references(root).await; + for (eco, uuid, rel) in references { + let ledgered = state + .entries + .values() + .any(|entry| entry.uuid == uuid && entry.ecosystem == eco); + if ledgered || unledgered_uuids.contains(uuid.as_str()) { + continue; + } + // No ledger entry means no purl to name: like repair, the event + // carries the uuid and the referenced path instead. The message + // names only the ecosystem, keeping patch identifiers out of logs. + let detail = format!( + "a lockfile references a vendored {eco} artifact under .socket/vendor/{eco}/ but \ + the vendor ledger (.socket/vendor/state.json) has no entry for it; restore \ + state.json from version control" + ); + if !args.common.json { + eprintln!("vendor_ledger_missing: {detail}"); + } + env.record( + PatchEvent::artifact(PatchAction::Failed) + .with_uuid(uuid) + .with_error("vendor_ledger_missing", detail) + .with_details(serde_json::json!({ "ecosystem": eco, "path": rel })), + ); + } if args.common.json { println!("{}", env.to_pretty_json()); } @@ -2627,15 +2661,23 @@ pub(crate) async fn vendor_records_reusing( .clone(); let refusal = match project { Some(refusal) => Some(refusal), - None => { - socket_patch_core::vendor::yarn_berry_vendor_target_preflight( - &common.cwd, - candidate, - pin, - &restore_opts, - ) - .await - } + None => match socket_patch_core::vendor::npm_tarball_gitignore_preflight( + &common.cwd, + &record.uuid, + ) + .await + { + Some(refusal) => Some(refusal), + None => { + socket_patch_core::vendor::yarn_berry_vendor_target_preflight( + &common.cwd, + candidate, + pin, + &restore_opts, + ) + .await + } + }, }; if let Some((code, detail)) = &refusal { has_errors = true; diff --git a/crates/socket-patch-cli/tests/e2e_vendor_yarn_berry_build.rs b/crates/socket-patch-cli/tests/e2e_vendor_yarn_berry_build.rs index c33839278..5f43d035f 100644 --- a/crates/socket-patch-cli/tests/e2e_vendor_yarn_berry_build.rs +++ b/crates/socket-patch-cli/tests/e2e_vendor_yarn_berry_build.rs @@ -732,3 +732,144 @@ async fn run_berry_capstone(driver: VendorDriver, yarnrc_extra: &str) { ); eprintln!("REVERT OK"); } + +// ── #831: .gitignore rules over the vendored tarball ─────────────────── + +fn git(cwd: &Path, args: &[&str]) -> Output { + let out = Command::new("git") + .args(["-c", "user.name=t", "-c", "user.email=t@t", "-c", "init.defaultBranch=main"]) + .args(args) + .current_dir(cwd) + .output() + .expect("failed to run git"); + assert!( + out.status.success(), + "git {args:?} failed:\n{}", + String::from_utf8_lossy(&out.stderr) + ); + out +} + +/// A yarn berry (node-modules linker) project inside a git work tree whose +/// `.gitignore` adds `rule`, with left-pad installed and a marker patch +/// staged. `None` when yarn berry or the registry is unavailable. +fn gitignored_berry_fixture(tmp: &Path, rule: &str) -> Option<(PathBuf, Vec)> { + if !has_corepack_pm(yarn_berry()) { + skip!( + "SKIP e2e_vendor_yarn_berry_build (gitignore): `corepack {}` unavailable", + yarn_berry() + ); + return None; + } + let proj = tmp.join("proj"); + std::fs::create_dir_all(&proj).unwrap(); + std::fs::write( + proj.join("package.json"), + format!( + r#"{{"name":"yarn-berry-gitignore","version":"0.0.0","private":true,"dependencies":{{"{DEP}":"{DEP_VERSION}"}}}}"# + ), + ) + .unwrap(); + std::fs::write( + proj.join(".yarnrc.yml"), + "nodeLinker: node-modules\nenableGlobalCache: false\n", + ) + .unwrap(); + let global = tmp.join("yarn-global"); + let install = corepack( + &proj, + yarn_berry(), + &["install"], + &[("YARN_GLOBAL_FOLDER", global.to_str().unwrap())], + ); + if !install.status.success() { + skip!( + "SKIP e2e_vendor_yarn_berry_build (gitignore): fixture `yarn install` failed:\n{}", + yarn_berry_common::yarn_output(&install) + ); + return None; + } + let orig = std::fs::read(proj.join("node_modules").join(DEP).join("index.js")).unwrap(); + let patched: Vec = [MARKER.as_bytes(), orig.as_slice()].concat(); + stage_patch( + &proj, + &format!("pkg:npm/{DEP}@{DEP_VERSION}"), + "package/index.js", + &orig, + &patched, + ); + std::fs::write( + proj.join(".gitignore"), + format!("node_modules\n.yarn/\n.pnp.*\n{rule}\n"), + ) + .unwrap(); + git(&proj, &["init", "-q"]); + Some((proj, patched)) +} + +/// #831 (yarn berry): under `*.tgz` the vendored tarball still reaches the +/// commit, so a fresh `git clone` passes `yarn install --immutable` with an +/// empty global folder and loads the patched bytes. +#[test] +fn yarn_berry_vendored_tarball_survives_a_tgz_gitignore_rule() { + let tmp = tempfile::tempdir().unwrap(); + let Some((proj, patched)) = gitignored_berry_fixture(tmp.path(), "*.tgz") else { + return; + }; + let (code, stdout, stderr) = run_socket( + &proj, + &["vendor", "--json", "--offline", "--cwd", proj.to_str().unwrap()], + ); + assert_eq!(code, 0, "vendor failed.\nstdout:\n{stdout}\nstderr:\n{stderr}"); + + git(&proj, &["add", "-A"]); + git(&proj, &["commit", "-qm", "vendored"]); + let fresh = tmp.path().join("fresh"); + git( + tmp.path(), + &["clone", "-q", proj.to_str().unwrap(), fresh.to_str().unwrap()], + ); + let fresh_global = tmp.path().join("fresh-yarn-global"); + let ci = corepack( + &fresh, + yarn_berry(), + &["install", "--immutable"], + &[ + ("YARN_GLOBAL_FOLDER", fresh_global.to_str().unwrap()), + ("YARN_ENABLE_GLOBAL_CACHE", "false"), + ], + ); + assert!( + ci.status.success(), + "a fresh clone must install from the committed tarball:\n{}", + yarn_berry_common::yarn_output(&ci) + ); + assert_eq!( + std::fs::read(fresh.join("node_modules").join(DEP).join("index.js")).unwrap(), + patched, + "the clone installs the patched bytes" + ); +} + +/// #831 (yarn berry): `.socket/` ignored refuses before any write. +#[test] +fn yarn_berry_vendor_refuses_a_gitignored_socket_dir() { + let tmp = tempfile::tempdir().unwrap(); + let Some((proj, _)) = gitignored_berry_fixture(tmp.path(), ".socket/") else { + return; + }; + let lock_before = std::fs::read(proj.join("yarn.lock")).unwrap(); + let pkg_before = std::fs::read(proj.join("package.json")).unwrap(); + let (code, stdout, stderr) = run_socket( + &proj, + &["vendor", "--json", "--offline", "--cwd", proj.to_str().unwrap()], + ); + assert_eq!(code, 1, "vendor must fail.\nstdout:\n{stdout}\nstderr:\n{stderr}"); + assert!( + stdout.contains("vendor_artifact_gitignored"), + "refusal code expected:\n{stdout}" + ); + assert_eq!(std::fs::read(proj.join("yarn.lock")).unwrap(), lock_before); + assert_eq!(std::fs::read(proj.join("package.json")).unwrap(), pkg_before); + assert!(!proj.join(format!(".socket/vendor/npm/{UUID}")).exists()); +} diff --git a/crates/socket-patch-cli/tests/e2e_vendor_yarn_classic_build.rs b/crates/socket-patch-cli/tests/e2e_vendor_yarn_classic_build.rs index cdd7e1219..3903d14a8 100644 --- a/crates/socket-patch-cli/tests/e2e_vendor_yarn_classic_build.rs +++ b/crates/socket-patch-cli/tests/e2e_vendor_yarn_classic_build.rs @@ -859,6 +859,214 @@ fn yarn_classic_detached_scan_vendored_fresh_checkout_manifestless_vex() { drop(server); } +// ── #831: .gitignore rules over the vendored tarball ─────────────────── + +fn git(cwd: &Path, args: &[&str]) -> Output { + let out = Command::new("git") + .args([ + "-c", + "user.name=t", + "-c", + "user.email=t@t", + "-c", + "init.defaultBranch=main", + ]) + .args(args) + .current_dir(cwd) + .output() + .expect("failed to run git"); + assert!( + out.status.success(), + "git {args:?} failed:\n{}", + String::from_utf8_lossy(&out.stderr) + ); + out +} + +/// A yarn classic project inside a git work tree whose `.gitignore` adds +/// `rule`, with left-pad installed and a marker patch staged. `None` when +/// yarn or the registry is unavailable (after the skip line). +fn gitignored_fixture(tmp: &Path, rule: &str) -> Option<(PathBuf, Vec)> { + if !require_yarn_classic("e2e_vendor_yarn_classic_build", |c| { + cache_env::isolate(c); + }) { + return None; + } + let proj = tmp.join("proj"); + std::fs::create_dir_all(&proj).unwrap(); + std::fs::write( + proj.join("package.json"), + format!( + r#"{{"name":"yarn-classic-gitignore","version":"0.0.0","private":true,"dependencies":{{"{DEP}":"{DEP_VERSION}"}}}}"# + ), + ) + .unwrap(); + let cache = tmp.join("yarn-cache"); + let install = corepack( + &proj, + &yarn_classic(), + &["install", "--no-progress"], + &[("YARN_CACHE_FOLDER", cache.to_str().unwrap())], + ); + if !install.status.success() { + skip!( + "fixture `yarn install` failed (registry unreachable?):\n{}", + String::from_utf8_lossy(&install.stderr) + ); + return None; + } + let orig = std::fs::read(proj.join("node_modules").join(DEP).join("index.js")).unwrap(); + let patched: Vec = [MARKER.as_bytes(), orig.as_slice()].concat(); + stage_patch( + &proj, + &format!("pkg:npm/{DEP}@{DEP_VERSION}"), + "package/index.js", + &orig, + &patched, + ); + std::fs::write(proj.join(".gitignore"), format!("node_modules\n{rule}\n")).unwrap(); + git(&proj, &["init", "-q"]); + Some((proj, patched)) +} + +/// #831: GitHub's stock Node.gitignore ignores `*.tgz`. The vendored +/// tarball must still reach the commit, so a fresh `git clone` installs the +/// patched bytes with `--frozen-lockfile --offline` and an empty cache. +#[test] +fn yarn_classic_vendored_tarball_survives_a_tgz_gitignore_rule() { + let tmp = tempfile::tempdir().unwrap(); + let Some((proj, patched)) = gitignored_fixture(tmp.path(), "*.tgz") else { + return; + }; + let (code, stdout, stderr) = run_socket( + &proj, + &[ + "vendor", + "--json", + "--offline", + "--cwd", + proj.to_str().unwrap(), + ], + ); + assert_eq!( + code, 0, + "vendor failed.\nstdout:\n{stdout}\nstderr:\n{stderr}" + ); + let env = parse_envelope(&stdout); + assert_eq!(env["summary"]["applied"], 1, "one package vendored: {env}"); + + git(&proj, &["add", "-A"]); + git(&proj, &["commit", "-qm", "vendored"]); + let tracked = String::from_utf8(git(&proj, &["ls-files", ".socket"]).stdout).unwrap(); + let tgz_rel = format!(".socket/vendor/npm/{UUID}/{DEP}-{DEP_VERSION}.tgz"); + assert!( + tracked.lines().any(|l| l == tgz_rel), + "the vendored tarball must be committed:\n{tracked}" + ); + + let fresh = tmp.path().join("fresh"); + git( + tmp.path(), + &[ + "clone", + "-q", + proj.to_str().unwrap(), + fresh.to_str().unwrap(), + ], + ); + let fresh_cache = tmp.path().join("fresh-yarn-cache"); + let ci = corepack( + &fresh, + &yarn_classic(), + &["install", "--frozen-lockfile", "--offline", "--no-progress"], + &[("YARN_CACHE_FOLDER", fresh_cache.to_str().unwrap())], + ); + assert!( + ci.status.success(), + "a fresh clone must install from the committed tarball.\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&ci.stdout), + String::from_utf8_lossy(&ci.stderr), + ); + if yarn_classic_vex::installs_file_tarballs(&yarn_classic_vex::yarn_classic_version()) { + assert_eq!( + std::fs::read(fresh.join("node_modules").join(DEP).join("index.js")).unwrap(), + patched, + "the clone installs the patched bytes" + ); + } + + // A checkout that lost the ledger and manifest (both ignored, or dropped + // from the commit) still has yarn.lock wired to the artifact: `vendor + // --check` must fail on that reference, not pass with nothing to check. + std::fs::remove_file(fresh.join(".socket/vendor/state.json")).unwrap(); + std::fs::remove_file(fresh.join(".socket/manifest.json")).unwrap(); + let (code, stdout, stderr) = run_socket( + &fresh, + &[ + "vendor", + "--check", + "--json", + "--cwd", + fresh.to_str().unwrap(), + ], + ); + assert_eq!( + code, 1, + "vendor --check must fail on an unledgered lock reference.\nstdout:\n{stdout}\nstderr:\n{stderr}" + ); + let env = parse_envelope(&stdout); + assert!( + env["events"].as_array().unwrap().iter().any(|e| { + e["errorCode"] == "vendor_ledger_missing" + && e["uuid"] == UUID + && e["details"]["ecosystem"] == "npm" + }), + "expected vendor_ledger_missing naming the referenced artifact: {env}" + ); +} + +/// #831: a rule ignoring the vendored uuid dir itself (`vendor/`, +/// `.socket/`) can't be overridden from inside it. Vendoring refuses with +/// `vendor_artifact_gitignored` and leaves `yarn.lock` untouched, instead of +/// reporting success over a tarball the commit would drop. +#[test] +fn yarn_classic_vendor_refuses_a_gitignored_vendor_dir() { + for rule in ["vendor/", ".socket/"] { + let tmp = tempfile::tempdir().unwrap(); + let Some((proj, _)) = gitignored_fixture(tmp.path(), rule) else { + return; + }; + let lock_before = std::fs::read(proj.join("yarn.lock")).unwrap(); + let (code, stdout, stderr) = run_socket( + &proj, + &[ + "vendor", + "--json", + "--offline", + "--cwd", + proj.to_str().unwrap(), + ], + ); + assert_eq!( + code, 1, + "{rule}: vendor must fail.\nstdout:\n{stdout}\nstderr:\n{stderr}" + ); + assert!( + stdout.contains("vendor_artifact_gitignored"), + "{rule}: refusal code expected:\n{stdout}" + ); + assert_eq!( + std::fs::read(proj.join("yarn.lock")).unwrap(), + lock_before, + "{rule}: yarn.lock stays untouched" + ); + assert!( + !proj.join(format!(".socket/vendor/npm/{UUID}")).exists(), + "{rule}: no artifact is written" + ); + } +} + /// #627: with `yarn.lock` a symbolic link to a lock shared with another /// checkout, `vendor` refuses as hosted mode does /// (`redirect_symlinked_file_unsupported`, exit 1) instead of renaming its diff --git a/crates/socket-patch-cli/tests/in_process_vendor.rs b/crates/socket-patch-cli/tests/in_process_vendor.rs index 3c51f2992..7827b8677 100644 --- a/crates/socket-patch-cli/tests/in_process_vendor.rs +++ b/crates/socket-patch-cli/tests/in_process_vendor.rs @@ -3836,6 +3836,59 @@ snapshots: ); } + /// Hosted → vendored in a git project that ignores `.socket/` (#831): + /// hosted mode writes nothing there, so the rule is common. The + /// gitignore refusal comes BEFORE the takeover restores the upstream + /// entry, so the hosted patch stays wired instead of the package ending + /// up patched in neither mode. + #[tokio::test] + #[serial] + async fn hosted_then_vendor_under_a_gitignored_socket_dir_keeps_the_hosted_wiring() { + if socket_patch_core::utils::process::resolve_tool("git").is_none() { + return; + } + let server = MockServer::start().await; + mock_hosted_api(&server).await; + let registry = mock_registry(&server).await; + let tmp = tempfile::tempdir().unwrap(); + let root = tmp.path(); + write_package_lock_project(root); + assert!(std::process::Command::new("git") + .arg("-C") + .arg(root) + .args(["init", "-q"]) + .status() + .unwrap() + .success()); + std::fs::write(root.join(".gitignore"), "node_modules/\n.socket/\n").unwrap(); + assert_eq!(scan_run(hosted_args(root, server.uri())).await, 0); + let hosted_lock = std::fs::read(root.join("package-lock.json")).unwrap(); + let hosted_npmrc = std::fs::read(root.join(".npmrc")).unwrap(); + assert!(String::from_utf8_lossy(&hosted_lock).contains(HOSTED_URL)); + + seed_manifest_and_blob(root); + let (code, env) = vendor_online_cli(root, ®istry, PATCH_ORIGIN, &[]); + assert_eq!(code, 1, "{env:#}"); + find_event(&env, "failed", Some("vendor_artifact_gitignored")); + assert!( + events(&env) + .iter() + .all(|e| e["errorCode"] != "vendor_takeover_reverted_redirect"), + "the takeover must not run before the refusal: {env:#}" + ); + assert_eq!( + std::fs::read(root.join("package-lock.json")).unwrap(), + hosted_lock, + "the hosted lock must be untouched" + ); + assert_eq!( + std::fs::read(root.join(".npmrc")).unwrap(), + hosted_npmrc, + "the hosted .npmrc must be untouched" + ); + assert!(!root.join(".socket/vendor/npm").exists(), "nothing is staged"); + } + /// Hosted → vendored over a linked `.socket/vendor/npm` (#664): the /// refusal comes BEFORE the takeover restores the upstream entry, so /// the hosted patch stays wired (lock and `.npmrc` byte-identical) and diff --git a/crates/socket-patch-core/src/crawlers/gradle_cache.rs b/crates/socket-patch-core/src/crawlers/gradle_cache.rs index ef295ee27..afd7c4fba 100644 --- a/crates/socket-patch-core/src/crawlers/gradle_cache.rs +++ b/crates/socket-patch-core/src/crawlers/gradle_cache.rs @@ -70,8 +70,7 @@ pub fn hash_eq(dir_name: &str, sha1_hex: &str) -> bool { /// Whether `bytes` are the pristine download Gradle stored in the hash /// directory `dir_name` (their sha1 names it). pub fn pristine(dir_name: &str, bytes: &[u8]) -> bool { - use sha1::{Digest, Sha1}; - hash_eq(dir_name, &hex::encode(Sha1::digest(bytes))) + hash_eq(dir_name, &crate::utils::digest::sha1_hex_of(bytes)) } /// Whether `path` is a version directory of a `files-2.1` tree @@ -432,8 +431,6 @@ impl DerivedIndex { /// The [`DerivedCopies`] of the jar `jar_leaf` whose pristine bytes /// hash to `pristine_sha1`. pub fn query(&self, jar_leaf: &str, pristine_sha1: &str) -> DerivedCopies { - use sha1::{Digest, Sha1}; - let instrumented = format!("instrumented-{jar_leaf}"); let mut out = DerivedCopies { incomplete: self.incomplete, @@ -460,7 +457,9 @@ impl DerivedIndex { out.stale.push(path.clone()); } else if name == jar_leaf || name == instrumented { match crate::utils::fs::read_regular_to_bytes_sync(path) { - Ok(bytes) if hash_eq(&hex::encode(Sha1::digest(&bytes)), pristine_sha1) => { + Ok(bytes) + if hash_eq(&crate::utils::digest::sha1_hex_of(&bytes), pristine_sha1) => + { out.stale.push(path.clone()) } Ok(_) => out.unknown.push(path.clone()), diff --git a/crates/socket-patch-core/src/patch/jvm_jar.rs b/crates/socket-patch-core/src/patch/jvm_jar.rs index 82d679406..f38a84403 100644 --- a/crates/socket-patch-core/src/patch/jvm_jar.rs +++ b/crates/socket-patch-core/src/patch/jvm_jar.rs @@ -25,8 +25,6 @@ use std::collections::HashMap; use std::path::{Path, PathBuf}; -use sha1::Digest as _; - use crate::crawlers::gradle_cache; use crate::hash::git_sha256::compute_git_sha256_from_bytes; use crate::manifest::schema::PatchFileInfo; @@ -353,12 +351,11 @@ fn unpatched_members( } fn sha256_hex(bytes: &[u8]) -> String { - use sha2::Digest as _; - hex::encode(sha2::Sha256::digest(bytes)) + crate::utils::digest::sha256_hex_of(bytes) } fn sha1_hex(bytes: &[u8]) -> String { - hex::encode(sha1::Sha1::digest(bytes)) + crate::utils::digest::sha1_hex_of(bytes) } /// `/jvm-originals/.jar`. diff --git a/crates/socket-patch-core/src/patch/sidecars/maven.rs b/crates/socket-patch-core/src/patch/sidecars/maven.rs index f2f5a2466..8798bfce6 100644 --- a/crates/socket-patch-core/src/patch/sidecars/maven.rs +++ b/crates/socket-patch-core/src/patch/sidecars/maven.rs @@ -17,8 +17,6 @@ use std::path::{Path, PathBuf}; -use sha1::Digest as _; - use super::{ SidecarAdvisory, SidecarAdvisoryCode, SidecarError, SidecarFile, SidecarFileAction, SidecarPayload, SidecarSeverity, @@ -44,7 +42,7 @@ impl Algo { fn digest(self, bytes: &[u8]) -> String { match self { - Algo::Sha1 => hex::encode(sha1::Sha1::digest(bytes)), + Algo::Sha1 => crate::utils::digest::sha1_hex_of(bytes), Algo::Md5 => hex::encode(md5(bytes)), } } diff --git a/crates/socket-patch-core/src/vendor/mod.rs b/crates/socket-patch-core/src/vendor/mod.rs index 2c8adc2c5..e81b22888 100644 --- a/crates/socket-patch-core/src/vendor/mod.rs +++ b/crates/socket-patch-core/src/vendor/mod.rs @@ -130,6 +130,7 @@ pub use verify::{ // The hosted→vendored takeover refuses a berry project the backend would // refuse BEFORE it reverts the hosted redirect. pub use npm_lock::npm_lock_vendor_preflight; +pub use npm_common::npm_tarball_gitignore_preflight; pub use yarn_berry_lock::{yarn_berry_vendor_preflight, yarn_berry_vendor_target_preflight}; use std::collections::{HashMap, HashSet}; diff --git a/crates/socket-patch-core/src/vendor/npm_common.rs b/crates/socket-patch-core/src/vendor/npm_common.rs index 6b8834def..96403cd64 100644 --- a/crates/socket-patch-core/src/vendor/npm_common.rs +++ b/crates/socket-patch-core/src/vendor/npm_common.rs @@ -28,6 +28,7 @@ use crate::utils::purl::{percent_decode_purl_component, strip_purl_qualifiers}; use super::common::{ already_patched_result, done, failed_result, refused, service_offline_conflict, }; +use super::npm_dir; use super::npm_pack::PackedTarball; use super::path::vendor_uuid_dir_rel; use super::reuse; @@ -177,8 +178,127 @@ pub(super) struct NpmStagedPack { /// Reuse a verified committed tarball or download its immutable service artifact. /// A dry run verifies the download without writing into the project. The /// backend wires the returned artifact only after acquisition succeeds. +/// +/// The artifact must survive the commit the vendored workflow ends with, so +/// git's ignore rules are checked on both sides of the write: a rule that +/// ignores the uuid dir itself (`.socket/`, `vendor/`) refuses +/// `vendor_artifact_gitignored` before anything is written, and once the +/// tarball is in place `/.gitignore` re-includes it against file rules +/// such as Node.gitignore's `*.tgz`, then the written paths are probed again. +/// The uuid-dir half of [`stage_patch_pack`]'s gitignore check, for callers +/// that must refuse before an earlier irreversible step: the hosted->vendored +/// takeover restores the registry entry first, so a `.socket/` rule caught +/// only at staging would leave the package patched in neither mode. +pub async fn npm_tarball_gitignore_preflight( + project_root: &Path, + uuid: &str, +) -> Option<(&'static str, String)> { + let uuid_dir_rel = vendor_uuid_dir_rel("npm", uuid)?; + let rules = npm_dir::gitignored(project_root, &[format!("{uuid_dir_rel}/")]).await?; + Some(( + npm_dir::GITIGNORED, + npm_dir::gitignored_detail(&uuid_dir_rel, &rules), + )) +} + #[allow(clippy::too_many_arguments)] pub(super) async fn stage_patch_pack( + purl: &str, + installed_dir: PackageSource<'_>, + project_root: &Path, + record: &PatchRecord, + sources: &PatchSources<'_>, + dry_run: bool, + force: bool, + warnings: &mut Vec, + service: Option<&VendorServiceConfig>, +) -> Result<(Option, ApplyResult), Box> { + let coords = guard_coordinates(purl, record)?; + if let Some(rules) = + npm_dir::gitignored(project_root, &[format!("{}/", coords.uuid_dir_rel)]).await + { + return Err(Box::new(refused( + npm_dir::GITIGNORED, + npm_dir::gitignored_detail(&coords.uuid_dir_rel, &rules), + ))); + } + let (staged, result) = acquire_patch_pack( + purl, + installed_dir, + project_root, + record, + sources, + dry_run, + force, + warnings, + service, + ) + .await?; + if let Some(staged) = &staged { + keep_pack_committable(purl, project_root, &coords, staged, warnings).await?; + } + Ok((staged, result)) +} + +/// Write `/.gitignore` and `/.gitattributes` next to the staged +/// tarball, then ask git whether it would commit them. Still ignored (a rule +/// the nested `.gitignore` cannot override) refuses and unwinds a uuid dir +/// this run created; git failing to answer is only a warning. +async fn keep_pack_committable( + purl: &str, + project_root: &Path, + coords: &NpmCoords, + staged: &NpmStagedPack, + warnings: &mut Vec, +) -> Result<(), Box> { + let unstage = |error: String| { + done_failure_unstage( + purl, + error, + project_root, + &coords.uuid_dir_rel, + staged.uuid_dir_preexisted, + ) + }; + let uuid_dir = project_root.join(&coords.uuid_dir_rel); + if let Err(e) = npm_dir::restore_uuid_metadata(&uuid_dir).await { + return Err(Box::new( + unstage(format!( + "cannot write {}/.gitignore: {e}", + coords.uuid_dir_rel + )) + .await, + )); + } + let probe = [ + staged.rel_tgz.clone(), + format!("{}/.gitignore", coords.uuid_dir_rel), + format!("{}/.gitattributes", coords.uuid_dir_rel), + ]; + match npm_dir::gitignore_probe(project_root, &probe).await { + Ok(Some(rules)) => { + if !staged.uuid_dir_preexisted { + let _ = remove_tree(&uuid_dir).await; + super::common::prune_empty_vendor_levels(&uuid_dir).await; + } + Err(Box::new(refused( + npm_dir::GITIGNORED, + npm_dir::gitignored_detail(&staged.rel_tgz, &rules), + ))) + } + Ok(None) => Ok(()), + Err(why) => { + warnings.push(VendorWarning::new( + npm_dir::GITIGNORE_UNCHECKED, + npm_dir::gitignore_unchecked_detail(&staged.rel_tgz, &why), + )); + Ok(()) + } + } +} + +#[allow(clippy::too_many_arguments)] +async fn acquire_patch_pack( purl: &str, _installed_dir: PackageSource<'_>, project_root: &Path, @@ -1312,6 +1432,116 @@ mod tests { ); } + // ──────────────── git ignore rules over the staged tarball ──────────────── + + /// A git work tree at `root` whose `.gitignore` is `rules`, or `None` + /// when git is not installed. + fn git_project(root: &Path, rules: &str) -> Option<()> { + let git = crate::utils::process::resolve_tool("git")?; + let ok = std::process::Command::new(git) + .arg("-C") + .arg(root) + .args(["init", "-q"]) + .status() + .ok()? + .success(); + assert!(ok, "git init"); + std::fs::write(root.join(".gitignore"), rules).unwrap(); + Some(()) + } + + async fn granted_service() -> wiremock::MockServer { + let tgz = build_tgz(&[("index.js", PATCHED_INDEX)]).await; + let sri = PackedTarball::from_bytes(&tgz).integrity; + let server = wiremock::MockServer::start().await; + mount_granted(&server, &sri, &tgz).await; + server + } + + /// #831: GitHub's stock Node.gitignore ignores `*.tgz`. The staged + /// tarball gets a `/.gitignore` that re-includes it, so git + /// commits it with the rewired lockfile. + #[tokio::test] + async fn a_tgz_ignore_rule_is_overridden_by_the_uuid_gitignore() { + let tmp = tempfile::tempdir().unwrap(); + if git_project(tmp.path(), "node_modules\n*.tgz\n").is_none() { + return; + } + let server = granted_service().await; + let cfg = service_cfg(&server.uri(), VendorSource::Service); + let (staged, _) = run_pipeline(tmp.path(), &patched_index_record(), Some(&cfg)) + .await + .unwrap_or_else(|e| panic!("a re-includable rule must not refuse: {e:?}")); + let staged = staged.expect("a wet run stages the tarball"); + let uuid_dir = tmp.path().join(format!(".socket/vendor/npm/{UUID}")); + assert_eq!( + std::fs::read_to_string(uuid_dir.join(".gitignore")).unwrap(), + npm_dir::UUID_GITIGNORE + ); + assert_eq!( + std::fs::read_to_string(uuid_dir.join(".gitattributes")).unwrap(), + npm_dir::UUID_GITATTRIBUTES + ); + assert_eq!( + npm_dir::gitignored(tmp.path(), std::slice::from_ref(&staged.rel_tgz)).await, + None, + "git commits the vendored tarball" + ); + } + + /// #831: a rule ignoring the uuid dir itself (`vendor/`, `.socket/`) + /// can't be overridden from inside it, so vendoring refuses before + /// writing anything, dry run included. + #[tokio::test] + async fn a_directory_ignore_rule_refuses_before_any_write() { + for rule in ["vendor/", ".socket/", ".socket/vendor/"] { + for dry_run in [false, true] { + let tmp = tempfile::tempdir().unwrap(); + if git_project(tmp.path(), &format!("node_modules\n{rule}\n")).is_none() { + return; + } + let server = granted_service().await; + let cfg = service_cfg(&server.uri(), VendorSource::Service); + let blobs = tmp.path().join(".socket/blobs"); + let sources = PatchSources::blobs_only(&blobs); + let mut warnings = Vec::new(); + let err = expect_err( + stage_patch_pack( + LP_PURL, + (&tmp.path().join("node_modules/left-pad")).into(), + tmp.path(), + &patched_index_record(), + &sources, + dry_run, + false, + &mut warnings, + Some(&cfg), + ) + .await, + ); + expect_refusal(err, npm_dir::GITIGNORED); + assert!( + !tmp.path().join(".socket/vendor").exists(), + "{rule} (dry run {dry_run}): nothing is written" + ); + } + } + } + + /// Outside a git work tree there is nothing to commit, so no rule + /// applies and the pack stages as before (metadata included). + #[tokio::test] + async fn no_work_tree_stages_without_a_refusal() { + let tmp = tempfile::tempdir().unwrap(); + std::fs::write(tmp.path().join(".gitignore"), ".socket/\n").unwrap(); + let server = granted_service().await; + let cfg = service_cfg(&server.uri(), VendorSource::Service); + let (staged, _) = run_pipeline(tmp.path(), &patched_index_record(), Some(&cfg)) + .await + .unwrap_or_else(|e| panic!("no work tree, no refusal: {e:?}")); + assert!(staged.is_some()); + } + // ───────────────────────── small helper arms ───────────────────────── /// A file squatting where `.socket/vendor` must be a directory makes the diff --git a/crates/socket-patch-core/src/vendor/npm_dir.rs b/crates/socket-patch-core/src/vendor/npm_dir.rs index 399afe22c..d11031b3a 100644 --- a/crates/socket-patch-core/src/vendor/npm_dir.rs +++ b/crates/socket-patch-core/src/vendor/npm_dir.rs @@ -514,7 +514,7 @@ pub(crate) const GITIGNORED: &str = "vendor_artifact_gitignored"; /// commit it. pub(crate) const GITIGNORE_UNCHECKED: &str = "vendor_artifact_gitignore_unchecked"; -fn gitignore_unchecked_detail(rel: &str, why: &str) -> String { +pub(crate) fn gitignore_unchecked_detail(rel: &str, why: &str) -> String { format!( "could not check whether git would commit the vendored artifact at {rel} ({why}); \ make sure no ignore rule covers .socket/ before committing it"