diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index 10a71045f..ed9fd4132 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -815,8 +815,8 @@ to **six flavors**. | pypi / pdm (pdm.lock) | (rebuilt wheel) | lock-only: the `[[package]]` gains the local-file `path` + `files[]` hash. pyproject + `content_hash` untouched. Non-fixture `[metadata] strategy` / hash-less locks refused | `pdm sync` (+ `pdm install --check`), cold cache | | pypi / pipenv (Pipfile.lock) | (rebuilt wheel) | lock-only: the `default`/`develop` entry → `{file, hashes:[sha256-of-our-wheel]}`. Pipfile + `_meta.hash` untouched. Emits `vendor_integrity_unverified` — pipenv does not hash-check file entries; the committed wheel bytes are the protection | `pipenv install --deploy` (+ `pipenv verify`), cold cache | | pypi / requirements.txt (pip / `uv pip`) | (rebuilt wheel) | pin line → `./` (markers carried over; transitive deps appended), plus `--hash=sha256:` only when the requirements tree is already in pip's hash-checking mode (any `--hash` or `--require-hashes`) | `pip install -r` / `uv pip install -r` **run from the project root** (both resolve bare paths against the CWD) | -| nuget | deterministically rebuilt `.nupkg` at `..nupkg` (the uuid dir IS a NuGet folder feed; the stale embedded signature is dropped — unsigned is accepted under NuGet's default validation) | `nuget.config` source + `packageSourceMapping` for the id (creating the mapping from scratch ALSO fans a `` out to every pre-existing source — mapping is exclusive, NU1100 otherwise) **+** `packages.lock.json` `contentHash` → `base64(sha512(nupkg))` for the entries at the patched version in every lock a project under the root restores into — the root `packages.lock.json`, member projects' locks, `packages..lock.json`, a literal `NuGetLockFilePath` (v5.0 #353/#514; an unevaluable `NuGetLockFilePath` is refused with `vendor_nuget_lock_path_unresolved`) — (`vendor_nuget_no_lockfile` warning when there is none; a lock that also resolves the id at another version is refused with `vendor_nuget_lock_other_version`, nothing written) | `dotnet restore --locked-mode`, cold cache, `--network none` (tampered nupkg fails NU1403) | -| maven | the patched `.jar` + the upstream pom (only its `` suffixed; transitives survive) + `.sha1` sidecars + an ownership marker under `.socket/vendor/maven2///-socket./` | every pom root, single-module (a reactor of one) or multi-module: the pinned `` + `` pin, `.mvn/maven.config` (`maven.repo.local.tail`) and the `socket-patch-vendor` fallback file repository (`checksumPolicy=fail`); Gradle, sbt and scala-cli roots go to the same JVM backend (ledger ecosystem `jvm`). Pre-v5 `maven_pom_repository` entries (`` to `.socket/vendor/maven/`) are revert-only: vendoring their root is refused (`vendor_jvm_shape_unsupported`, `legacy_maven_root`) | `mvn` build on a fresh checkout with a warm local repository and behind `mirrorOf external:*` (host capstone `e2e_vendor_maven_build` across the Maven matrix) | +| nuget | deterministically rebuilt `.nupkg` at `..nupkg` (the uuid dir IS a NuGet folder feed; the stale embedded signature is dropped — unsigned is accepted under NuGet's default validation), plus `/.gitignore` (`!*`: re-includes the nupkg against the project's ignores, such as VisualStudio.gitignore's `*.nupkg`); refuses `vendor_artifact_gitignored` when git would still drop it | `nuget.config` source + `packageSourceMapping` for the id (creating the mapping from scratch ALSO fans a `` out to every pre-existing source — mapping is exclusive, NU1100 otherwise) **+** `packages.lock.json` `contentHash` → `base64(sha512(nupkg))` for the entries at the patched version in every lock a project under the root restores into — the root `packages.lock.json`, member projects' locks, `packages..lock.json`, a literal `NuGetLockFilePath` (v5.0 #353/#514; an unevaluable `NuGetLockFilePath` is refused with `vendor_nuget_lock_path_unresolved`) — (`vendor_nuget_no_lockfile` warning when there is none; a lock that also resolves the id at another version is refused with `vendor_nuget_lock_other_version`, nothing written) | `dotnet restore --locked-mode`, cold cache, `--network none` (tampered nupkg fails NU1403) | +| maven | the patched `.jar` + the upstream pom (only its `` suffixed; transitives survive) + `.sha1` sidecars + an ownership marker under `.socket/vendor/maven2///-socket./`; every JVM tree root (`.socket/vendor/maven2`, Gradle's `.socket/vendor/gradle`, Coursier's `.socket/vendor/coursier`) owns a `.gitignore` (`!*`) that re-includes the jars against the project's ignores, such as Java.gitignore's `*.jar`, plus a `.gitattributes` (`-text`); a tree root git ignores itself refuses `vendor_artifact_gitignored` | every pom root, single-module (a reactor of one) or multi-module: the pinned `` + `` pin, `.mvn/maven.config` (`maven.repo.local.tail`) and the `socket-patch-vendor` fallback file repository (`checksumPolicy=fail`); Gradle, sbt and scala-cli roots go to the same JVM backend (ledger ecosystem `jvm`). Pre-v5 `maven_pom_repository` entries (`` to `.socket/vendor/maven/`) are revert-only: vendoring their root is refused (`vendor_jvm_shape_unsupported`, `legacy_maven_root`) | `mvn` build on a fresh checkout with a warm local repository and behind `mirrorOf external:*` (host capstone `e2e_vendor_maven_build` across the Maven matrix) | Ecosystems with no vendor backend (jsr) refuse per-purl with `vendor_unsupported_ecosystem`. yarn-berry **PnP** @@ -1419,8 +1419,8 @@ Every `--json` invocation emits a single JSON object that follows the **unified | `vendor_dep_manifest_unlocked` | refused | vendor (yarn classic): the patch rewrites the package's own `package.json` to depend on a descriptor (`name@range`) no `yarn.lock` block is keyed by — an added dependency, or an existing one moved to a new range. yarn 1 builds its install graph from the lock, so the rewired block would name a dependency it never resolves: online frozen installs fetch it unpinned, `--offline` installs fail and every plain `yarn install` re-saves the lock (#591). Refused after staging and before any wiring is written (the staged uuid dir is removed); the detail names the descriptors. Remedy: lock them first (for example `yarn add `), then re-run. A dry run, which stages nothing, does not foresee it. | | `redirect_yarn_classic_dep_manifest_unlocked` | `redirect.warnings[]` (warning) | scan/get `--mode hosted` (yarn classic): the served tarball's own `package.json` depends on a descriptor no `yarn.lock` block is keyed by. yarn 1 installs only what the lock names, so a pin would install the patched package without that dependency (#591). The dep is not pinned and never confirmed; the lock is left as it was. Same remedy as `vendor_dep_manifest_unlocked`. | | `redirect_yarn_classic_dep_manifest_rewritten` | `redirect.warnings[]` (warning) | scan/get `--mode hosted` (yarn classic): the served tarball's own `package.json` declares other dependencies than the pinned block's sub-maps, every descriptor already locked; the block's `dependencies:` / `optionalDependencies:` sub-maps are rewritten to match (#591). | -| `vendor_artifact_gitignored` | `failed` | vendor (vlt and the npm-family tarball flavors: npm, pnpm, bun, yarn classic, yarn berry): inside a git work tree, `git check-ignore --no-index` reports the new artifact's uuid directory as ignored by a rule its own `.gitignore` cannot override (such as a root `.socket/` or `vendor/` rule; the detail names the rule). Remedy: drop that rule for `.socket/vendor/`. Refused before any write. A file rule such as `*.tgz` is overridden by the `/.gitignore` vendoring writes; if the written artifact still reads as ignored, the run refuses and removes the uuid dir it created. | -| `vendor_artifact_gitignore_unchecked` | warning | vendor (vlt and the npm-family tarball flavors): git is installed but could not answer the ignore check for the written vendored directory (it failed to start, ran past 30 s, or `rev-parse` / `check-ignore` exited with an error); the package is vendored and the detail names what failed. Remedy: make sure no ignore rule covers `.socket/` before committing. Git absent, or a project outside any work tree, raises nothing. | +| `vendor_artifact_gitignored` | `failed` | vendor (vlt, the npm-family tarball flavors — npm, pnpm, bun, yarn classic, yarn berry — NuGet, and the JVM trees of Maven, Gradle, sbt and scala-cli): inside a git work tree, `git check-ignore --no-index` reports the new artifact's uuid directory (JVM: its tree root, such as `.socket/vendor/maven2`) as ignored by a rule its own `.gitignore` cannot override (such as a root `.socket/` or `vendor/` rule; the detail names the rule). Remedy: drop that rule for `.socket/vendor/`. Refused before any write, dry run included. A file rule such as `*.tgz`, `*.nupkg` or `*.jar` is overridden by the `!*` `.gitignore` vendoring writes into the uuid dir or tree root; if a written tarball, directory payload or nupkg still reads as ignored, the run refuses and removes the uuid dir it created. | +| `vendor_artifact_gitignore_unchecked` | warning | vendor (vlt, the npm-family tarball flavors and NuGet): git is installed but could not answer the ignore check for the written vendored directory (it failed to start, ran past 30 s, or `rev-parse` / `check-ignore` exited with an error); the package is vendored and the detail names what failed. Remedy: make sure no ignore rule covers `.socket/` before committing. Git absent, or a project outside any work tree, raises nothing. | | `vendor_ledger_entry_missing` | `failed` | vendor (vlt): the only installed copy is vlt's link to a committed vendored directory, but the vendor ledger has no entry for the package; restore `.socket/vendor/state.json` from version control (v5.0: `repair` no longer re-synthesizes it). Replaces the `package_not_installed` skip. | | `vendor_variant_ambiguous` | `failed` | vendor / scan / get `--mode vendored` (pypi, gem): the package is not installed and the manifest holds several release variants of it (`?artifact_id=` / `?platform=`), none of which the vendor ledger records, so nothing says which distribution to vendor; install the package or keep one release variant. A variant the ledger records (at any patch uuid) is taken as the wired one and its siblings are left out without an event. | | `vendor_artifact_missing` | reason | The recorded artifact is missing; repair requires an online exact redownload. | diff --git a/crates/socket-patch-cli/tests/e2e_vendor_jvm_build.rs b/crates/socket-patch-cli/tests/e2e_vendor_jvm_build.rs index 23503f7a2..06fccdb3b 100644 --- a/crates/socket-patch-cli/tests/e2e_vendor_jvm_build.rs +++ b/crates/socket-patch-cli/tests/e2e_vendor_jvm_build.rs @@ -425,6 +425,7 @@ fn maven_reactor_vendor_fresh_checkout_offline_build_and_byte_exact_revert() { let mut want_added = vec![ ".mvn/maven.config".to_string(), ".socket/vendor/maven2/.gitattributes".to_string(), + ".socket/vendor/maven2/.gitignore".to_string(), format!("{tree}/{ARTIFACT}-{SV}.jar"), format!("{tree}/{ARTIFACT}-{SV}.jar.sha1"), format!("{tree}/{ARTIFACT}-{SV}.pom"), @@ -756,6 +757,7 @@ fn gradle_multi_project_vendor_locked_offline_tamper_and_byte_exact_revert() { socket_patch_core::vendor::jvm::gradle::SCRIPT_REL.to_string(), socket_patch_core::vendor::jvm::gradle::INDEX_REL.to_string(), ".socket/vendor/gradle/.gitattributes".to_string(), + ".socket/vendor/gradle/.gitignore".to_string(), ".socket/gradle/.gitattributes".to_string(), ".socket/vendor/.gitattributes".to_string(), socket_patch_core::vendor::jvm::gradle::derived_metadata_rel(GROUP, ARTIFACT), diff --git a/crates/socket-patch-cli/tests/vendor_jvm_cli.rs b/crates/socket-patch-cli/tests/vendor_jvm_cli.rs index 7cb93fa31..90b36932e 100644 --- a/crates/socket-patch-cli/tests/vendor_jvm_cli.rs +++ b/crates/socket-patch-cli/tests/vendor_jvm_cli.rs @@ -949,6 +949,7 @@ fn gradle_vendor_429_crlf_checkout_checks_and_reverts_clean() { "proj/.socket/vendor/.gitattributes", "proj/.socket/vendor/gradle-index.tsv", "proj/.socket/vendor/gradle/.gitattributes", + "proj/.socket/vendor/gradle/.gitignore", FOO_METADATA, ]; for rel in text_files { diff --git a/crates/socket-patch-core/src/vendor/jvm/apply.rs b/crates/socket-patch-core/src/vendor/jvm/apply.rs index d66788f7d..2758a2317 100644 --- a/crates/socket-patch-core/src/vendor/jvm/apply.rs +++ b/crates/socket-patch-core/src/vendor/jvm/apply.rs @@ -95,6 +95,7 @@ fn is_owned_file(rel: &str) -> bool { [ maven_reactor::GITATTRIBUTES_REL, gradle::GITATTRIBUTES_REL, + gradle::GITIGNORE_REL, gradle::SCRIPT_GITATTRIBUTES_REL, gradle::VENDOR_GITATTRIBUTES_REL, gradle::SCRIPT_REL, diff --git a/crates/socket-patch-core/src/vendor/jvm/gradle.rs b/crates/socket-patch-core/src/vendor/jvm/gradle.rs index 95664cf9b..dcfd65308 100644 --- a/crates/socket-patch-core/src/vendor/jvm/gradle.rs +++ b/crates/socket-patch-core/src/vendor/jvm/gradle.rs @@ -46,6 +46,9 @@ pub const SCRIPT_REL: &str = ".socket/gradle/socket-patch.settings.gradle"; use super::layout::GRADLE_TREE as TREE_ROOT; /// The tree root's `.gitattributes`, shared by every Gradle patch. pub const GITATTRIBUTES_REL: &str = ".socket/vendor/gradle/.gitattributes"; +/// The tree root's `.gitignore` (`!*`): re-includes the vendored jars +/// against a user's `*.jar` rule (Java.gitignore), #620 / #1061. +pub const GITIGNORE_REL: &str = ".socket/vendor/gradle/.gitignore"; /// `.socket/gradle/`'s `.gitattributes` (`* -text`): the settings scripts /// there stay byte-exact on a `core.autocrlf` checkout (#429). Shared with /// the hosted script. @@ -511,6 +514,12 @@ pub fn plan( records.push(created_or_adopted(SCRIPT_REL, read(SCRIPT_REL).is_some())); writes.push(text_write(SCRIPT_REL, SCRIPT.as_bytes().to_vec())); records.push(owned_file(read, GITATTRIBUTES_REL, &mut writes)); + records.push(super::owned_file_with( + read, + GITIGNORE_REL, + super::coursier_tree::GITIGNORE.as_bytes(), + &mut writes, + )); records.push(owned_file(read, SCRIPT_GITATTRIBUTES_REL, &mut writes)); records.push(vendor_gitattributes(read, &mut writes)); @@ -1071,6 +1080,7 @@ pub fn unplan(read: ReadFn<'_>, c: &Coords<'_>, records: &[WiringRecord]) -> Jvm for (rel, expected) in [ (SCRIPT_REL, SCRIPT), (GITATTRIBUTES_REL, super::TREE_GITATTRIBUTES), + (GITIGNORE_REL, super::coursier_tree::GITIGNORE), (SCRIPT_GITATTRIBUTES_REL, super::TREE_GITATTRIBUTES), ] { if rel == SCRIPT_GITATTRIBUTES_REL && hosted_left { @@ -2585,6 +2595,7 @@ mod tests { (".socket/vendor/.gitattributes", false), (".socket/vendor/gradle-index.tsv", false), (".socket/vendor/gradle/.gitattributes", false), + (".socket/vendor/gradle/.gitignore", false), (".socket/vendor/gradle/com/google/code/gson/gson/2.10.1/gson-2.10.1.jar", true), (".socket/vendor/gradle/com/google/code/gson/gson/2.10.1/gson-2.10.1.pom", true), (".socket/vendor/gradle/com/google/code/gson/gson/2.10.1/socket-patch.vendor.json", true), @@ -2614,6 +2625,7 @@ mod tests { text_of(&plan, ".socket/vendor/gradle/.gitattributes"), "* -text\n" ); + assert_eq!(text_of(&plan, GITIGNORE_REL), "!*\n"); assert!(plan.warnings.is_empty(), "{:?}", plan.warnings); } diff --git a/crates/socket-patch-core/src/vendor/jvm/layout.rs b/crates/socket-patch-core/src/vendor/jvm/layout.rs index 701d0ad0a..8fd34f99a 100644 --- a/crates/socket-patch-core/src/vendor/jvm/layout.rs +++ b/crates/socket-patch-core/src/vendor/jvm/layout.rs @@ -163,6 +163,7 @@ pub const CAPTURED_FILES: &[&str] = &[ super::gradle::SCRIPT_REL, super::maven_reactor::GITATTRIBUTES_REL, super::gradle::GITATTRIBUTES_REL, + super::gradle::GITIGNORE_REL, super::gradle::SCRIPT_GITATTRIBUTES_REL, super::gradle::VENDOR_GITATTRIBUTES_REL, super::coursier_tree::INDEX_REL, @@ -543,6 +544,7 @@ mod tests { under(maven_reactor::GITATTRIBUTES_REL, MAVEN2_TREE); under(sbt::TREE_GITIGNORE_REL, MAVEN2_TREE); under(gradle::GITATTRIBUTES_REL, GRADLE_TREE); + under(gradle::GITIGNORE_REL, GRADLE_TREE); under(coursier_tree::GITIGNORE_REL, COURSIER_TREE); under(coursier_tree::GITATTRIBUTES_REL, COURSIER_TREE); under(scala_cli::GUARD_REL, COURSIER_TREE); diff --git a/crates/socket-patch-core/src/vendor/jvm/maven_reactor.rs b/crates/socket-patch-core/src/vendor/jvm/maven_reactor.rs index 920959c46..2df2ba6fd 100644 --- a/crates/socket-patch-core/src/vendor/jvm/maven_reactor.rs +++ b/crates/socket-patch-core/src/vendor/jvm/maven_reactor.rs @@ -25,6 +25,9 @@ use super::layout::MAVEN2_TREE as TREE_ROOT; pub const MAVEN_CONFIG: &str = ".mvn/maven.config"; /// The tree root's `.gitattributes`, shared by every Maven patch. pub const GITATTRIBUTES_REL: &str = ".socket/vendor/maven2/.gitattributes"; +/// The tree root's `.gitignore` (`!*`), shared with sbt: re-includes the +/// vendored jars against a user's `*.jar` rule (Java.gitignore), #1061. +pub use super::sbt::TREE_GITIGNORE_REL as GITIGNORE_REL; const OFFLINE_LINE: &str = "-Daether.offline.protocols=file"; const OFFLINE_KEY: &str = "-Daether.offline.protocols="; const TAIL_KEY: &str = "-Dmaven.repo.local.tail="; @@ -313,6 +316,12 @@ pub fn plan_with_external( )); } records.push(owned_file(read, GITATTRIBUTES_REL, &mut writes)); + records.push(super::owned_file_with( + read, + GITIGNORE_REL, + super::coursier_tree::GITIGNORE.as_bytes(), + &mut writes, + )); let (tree_dir, jar_rel, tree) = tree_writes(patch, &sv, suffixed_pom); writes.extend(tree); @@ -414,15 +423,17 @@ pub fn unplan(read: ReadFn<'_>, c: &Coords<'_>, records: &[WiringRecord]) -> Jvm before.insert(MAVEN_CONFIG.to_string(), Some(text)); } } - let created = records.iter().any(|w| { - w.kind == OWNED_FILE_KIND && w.file == GITATTRIBUTES_REL && op_of(w) == "create" - }); - if created && read(GITATTRIBUTES_REL).as_deref() == Some(TREE_GITATTRIBUTES.as_bytes()) { - before.insert( - GITATTRIBUTES_REL.to_string(), - Some(TREE_GITATTRIBUTES.to_string()), - ); - after.insert(GITATTRIBUTES_REL.to_string(), None); + for (rel, body) in [ + (GITATTRIBUTES_REL, TREE_GITATTRIBUTES), + (GITIGNORE_REL, super::coursier_tree::GITIGNORE), + ] { + let created = records + .iter() + .any(|w| w.kind == OWNED_FILE_KIND && w.file == rel && op_of(w) == "create"); + if created && read(rel).as_deref() == Some(body.as_bytes()) { + before.insert(rel.to_string(), Some(body.to_string())); + after.insert(rel.to_string(), None); + } } } JvmUnplan { diff --git a/crates/socket-patch-core/src/vendor/jvm/mod.rs b/crates/socket-patch-core/src/vendor/jvm/mod.rs index f5f62ac22..f7ac0adf9 100644 --- a/crates/socket-patch-core/src/vendor/jvm/mod.rs +++ b/crates/socket-patch-core/src/vendor/jvm/mod.rs @@ -230,6 +230,17 @@ pub enum Shape { Other, } +/// The committed vendor trees a plan for `shape` writes into. +pub(crate) fn shape_trees(shape: Shape) -> &'static [&'static str] { + match shape { + Shape::MavenReactor | Shape::Sbt => &[layout::MAVEN2_TREE], + Shape::Gradle => &[layout::GRADLE_TREE], + Shape::Mixed => &[layout::MAVEN2_TREE, layout::GRADLE_TREE], + Shape::ScalaCli => &[layout::COURSIER_TREE], + Shape::Other => &[], + } +} + /// A planned file: project-relative forward-slash path and its full new /// bytes. #[derive(Debug, Clone, PartialEq, Eq)] @@ -589,6 +600,7 @@ pub(crate) fn eol_blind(rel: &str) -> bool { gradle::SCRIPT_REL, gradle::INDEX_REL, gradle::GITATTRIBUTES_REL, + gradle::GITIGNORE_REL, gradle::SCRIPT_GITATTRIBUTES_REL, gradle::VENDOR_GITATTRIBUTES_REL, maven_reactor::GITATTRIBUTES_REL, diff --git a/crates/socket-patch-core/src/vendor/maven_repo.rs b/crates/socket-patch-core/src/vendor/maven_repo.rs index 5532b4695..63bce27af 100644 --- a/crates/socket-patch-core/src/vendor/maven_repo.rs +++ b/crates/socket-patch-core/src/vendor/maven_repo.rs @@ -552,8 +552,29 @@ pub async fn jvm_gate_preflight( } let shape = detect_shape(project_root); super::jvm::sbt_gate::for_shape(shape, project_root, &g, &a, &v) - .map(|_| ()) - .map_err(|stop| stop.code_and_detail(purl)) + .map_err(|stop| stop.code_and_detail(purl))?; + // Checked here too, so a hosted->vendored takeover keeps its pin + // instead of restoring upstream and then refusing (#1061). + match ignored_tree_root(shape, project_root).await { + Some(refusal) => Err(refusal), + None => Ok(()), + } +} + +/// The `vendor_artifact_gitignored` refusal when git ignores a tree root +/// `shape` writes into. Each tree root owns a `!*` `.gitignore` that +/// re-includes file rules such as Java.gitignore's `*.jar` (#1061), but a +/// rule ignoring the root itself (`.socket/`) can't be undone from inside. +async fn ignored_tree_root( + shape: super::jvm::Shape, + project_root: &Path, +) -> Option<(&'static str, String)> { + for tree in super::jvm::shape_trees(shape) { + if let Some(refusal) = super::npm_dir::ignored_root_refusal(project_root, tree).await { + return Some(refusal); + } + } + None } /// The committed tree bytes for `record` (jar, upstream pom, module, and @@ -686,6 +707,10 @@ async fn jvm_prelude( let gate_pass = super::jvm::sbt_gate::for_shape(shape, project_root, &group_id, &artifact_id, &version) .map_err(|stop| stop.into_outcome(purl))?; + // The tree must survive the commit the vendored workflow ends with. + if let Some((code, detail)) = ignored_tree_root(shape, project_root).await { + return Err(refused(code, detail)); + } Ok(JvmPrelude { group_id, artifact_id, @@ -1965,6 +1990,109 @@ mod tests { assert!(root.join(".socket/vendor/gradle-index.tsv").is_file()); } + /// The JVM shapes #1061 names, each as a fresh project: a single-module + /// pom, a multi-module reactor and a Gradle-only build. + async fn jvm_shape_fixture(shape: &str) -> (tempfile::TempDir, PathBuf, PathBuf, PatchRecord) { + match shape { + "pom" => fixture(Some(project_pom()), true, true).await, + "reactor" => reactor_fixture(true).await, + _ => { + let fx = fixture(None, true, true).await; + std::fs::write(fx.0.path().join("build.gradle"), "plugins { id 'java' }\n") + .unwrap(); + fx + } + } + } + + /// Every file under `.socket/` (relative, `/`-separated). + fn socket_files(root: &Path) -> Vec { + crate::vendor::test_support::tree_snapshot(root) + .into_keys() + .filter(|rel| rel.starts_with(".socket/")) + .collect() + } + + /// #1061 (and #620): GitHub's stock Java.gitignore ignores `*.jar`. + /// Vendoring a Maven, reactor or Gradle project must still leave every + /// written tree file committable (the tree roots re-include them), and + /// revert removes the re-include it created. + #[tokio::test] + #[serial_test::serial] + async fn a_jar_ignore_rule_is_overridden_by_the_tree_gitignore() { + use crate::vendor::test_support::{git_project, JAVA_GITIGNORE}; + for shape in ["pom", "reactor", "gradle"] { + let (dir, blobs, installed, record) = jvm_shape_fixture(shape).await; + let root = dir.path(); + if git_project(root, JAVA_GITIGNORE).is_none() { + return; + } + let (result, entry, _) = + unwrap_done(run_vendor(root, &blobs, &installed, &record, false).await); + assert!(result.success, "{shape}: {:?}", result.error); + let entry = entry.expect("ledger entry"); + assert!(entry.artifact.path.ends_with(".jar"), "{shape}"); + let written = socket_files(root); + assert!( + written.contains(&entry.artifact.path), + "{shape}: {written:?}" + ); + assert_eq!( + crate::vendor::npm_dir::gitignored(root, &written).await, + None, + "{shape}: git commits every vendored file" + ); + + let reverted = revert_maven(&entry, root, false).await; + assert!(reverted.success, "{shape}: {reverted:?}"); + let left = socket_files(root) + .into_iter() + .filter(|rel| rel.ends_with(".gitignore")) + .collect::>(); + assert!(left.is_empty(), "{shape}: revert leaves {left:?}"); + } + } + + /// #1061: a rule that ignores the vendor tree itself (`.socket/`) can't + /// be overridden from inside it, so every JVM shape refuses + /// `vendor_artifact_gitignored` before writing, dry run included. + #[tokio::test] + #[serial_test::serial] + async fn a_jvm_tree_directory_ignore_rule_refuses_before_any_write() { + use crate::vendor::test_support::git_project; + for shape in ["pom", "reactor", "gradle"] { + for rule in [".socket/", ".socket/vendor/"] { + for dry_run in [false, true] { + let (dir, blobs, installed, record) = jvm_shape_fixture(shape).await; + let root = dir.path(); + if git_project(root, &format!("{rule}\n")).is_none() { + return; + } + let before = crate::vendor::test_support::tree_snapshot(root); + let (code, detail) = unwrap_refused( + run_vendor(root, &blobs, &installed, &record, dry_run).await, + ); + assert_eq!( + code, "vendor_artifact_gitignored", + "{shape} {rule}: {detail}" + ); + assert!(detail.contains(rule), "{shape} {rule}: {detail}"); + assert_eq!( + crate::vendor::test_support::tree_snapshot(root), + before, + "{shape} {rule}: nothing written" + ); + // The takeover gate refuses too, before any restore. + assert_eq!( + jvm_gate_preflight(root, PURL).await.map_err(|(c, _)| c), + Err("vendor_artifact_gitignored"), + "{shape} {rule}" + ); + } + } + } + } + #[tokio::test] #[serial_test::serial] async fn refuses_unsafe_coordinates() { diff --git a/crates/socket-patch-core/src/vendor/npm_dir.rs b/crates/socket-patch-core/src/vendor/npm_dir.rs index ff5df4639..c9bb01a7c 100644 --- a/crates/socket-patch-core/src/vendor/npm_dir.rs +++ b/crates/socket-patch-core/src/vendor/npm_dir.rs @@ -505,6 +505,20 @@ fn gitignored_refusal(rel_dir: &str, rules: &str) -> VendorOutcome { refused(GITIGNORED, gitignored_detail(rel_dir, rules)) } +/// The `vendor_artifact_gitignored` refusal (code, detail) for a vendored +/// artifact root (`dir_rel`, project-relative) that git ignores as a +/// directory: a `.socket/` or `.socket/vendor/` rule no `.gitignore` inside +/// the root can override (#831, #1061). `None` when git would look inside +/// it, so the root's own `!*` `.gitignore` re-includes file rules such as +/// `*.jar`. +pub(crate) async fn ignored_root_refusal( + project_root: &Path, + dir_rel: &str, +) -> Option<(&'static str, String)> { + let rules = gitignored(project_root, &[format!("{dir_rel}/")]).await?; + Some((GITIGNORED, gitignored_detail(dir_rel, &rules))) +} + pub(crate) const GITIGNORED: &str = "vendor_artifact_gitignored"; /// The vendored dir was written, but git could not say whether it would diff --git a/crates/socket-patch-core/src/vendor/nuget_feed.rs b/crates/socket-patch-core/src/vendor/nuget_feed.rs index 620973346..52b9b6868 100644 --- a/crates/socket-patch-core/src/vendor/nuget_feed.rs +++ b/crates/socket-patch-core/src/vendor/nuget_feed.rs @@ -40,6 +40,10 @@ const CONFIG_SOURCE_WIRING_KIND: &str = "nuget_config_source"; const CONFIG_MAPPING_WIRING_KIND: &str = "nuget_config_mapping"; const LOCK_WIRING_KIND: &str = "nuget_lock_entry"; +/// `/.gitignore`, exactly: re-include the vendored nupkg against the +/// user's ignore rules (VisualStudio.gitignore's `*.nupkg`), #1061. +const UUID_GITIGNORE: &str = "!*\n"; + /// The implicit default public NuGet source, seeded as the catch-all target /// when a from-scratch `` would otherwise have no /// pre-existing source to fan `*` out to (a socket-only mapping NU1100s every @@ -175,6 +179,14 @@ async fn nuget_prelude( let nupkg_path = project_root.join(©_rel); let source_key = crate::patch::redirect::generation::hosted_pin_name(&record.uuid); + // The nupkg must survive the commit the vendored workflow ends with: a + // rule ignoring the uuid dir itself can't be undone from inside it. + if let Some((code, detail)) = + super::npm_dir::ignored_root_refusal(project_root, &uuid_dir_rel).await + { + return Err(refused(code, detail)); + } + // A patch with no files is meaningless to vendor: no-op success, no edits. if record.files.is_empty() { return Err(done( @@ -378,6 +390,10 @@ pub async fn vendor_nuget( // originals, and re-recording here would clobber them. if config_wired { if in_sync { + // A dir vendored before the re-include existed gains it now. + if !dry_run { + let _ = write_uuid_gitignore(&uuid_dir).await; + } return done( already_patched_result(purl, &nupkg_path, &record.files), None, @@ -818,13 +834,20 @@ async fn materialise_patched_nupkg( ) -> Result<(Vec, ApplyResult), Box> { match service_archive_copy(service, record, name, ".nupkg", warnings).await { Ok(bytes) => { - if let Err(e) = write_nupkg(uuid_dir, nupkg_path, &bytes).await { + let unwind = || async { if !config_wired { let _ = remove_tree(uuid_dir).await; prune_empty_vendor_levels(uuid_dir).await; } + }; + if let Err(e) = write_nupkg(uuid_dir, nupkg_path, &bytes).await { + unwind().await; return Err(Box::new(refused("vendor_prebuilt_write_failed", e))); } + if let Err(refusal) = keep_nupkg_committable(uuid_dir, nupkg_path, warnings).await { + unwind().await; + return Err(Box::new(refusal)); + } Ok(( bytes, already_patched_result(purl, nupkg_path, &record.files), @@ -834,14 +857,62 @@ async fn materialise_patched_nupkg( } } -/// Write `bytes` to `nupkg_path`, creating the uuid dir. Errors are strings. +/// Write `bytes` to `nupkg_path`, creating the uuid dir and its +/// re-including `.gitignore`. Errors are strings. async fn write_nupkg(uuid_dir: &Path, nupkg_path: &Path, bytes: &[u8]) -> Result<(), String> { tokio::fs::create_dir_all(uuid_dir) .await .map_err(|e| format!("cannot create {}: {e}", uuid_dir.display()))?; atomic_write_artifact(nupkg_path, bytes) .await - .map_err(|e| format!("cannot write {}: {e}", nupkg_path.display())) + .map_err(|e| format!("cannot write {}: {e}", nupkg_path.display()))?; + write_uuid_gitignore(uuid_dir).await +} + +/// Write `/.gitignore` ([`UUID_GITIGNORE`]) unless it already holds +/// it, in either line ending: a `core.autocrlf` checkout spells it `!*\r\n`, +/// and rewriting that to LF would dirty the tree on every re-vendor. +async fn write_uuid_gitignore(uuid_dir: &Path) -> Result<(), String> { + let path = uuid_dir.join(".gitignore"); + if read_regular_to_string(&path) + .await + .is_ok_and(|text| text.replace("\r\n", "\n") == UUID_GITIGNORE) + { + return Ok(()); + } + crate::utils::fs::atomic_write_bytes(&path, UUID_GITIGNORE.as_bytes()) + .await + .map_err(|e| format!("cannot write {}: {e}", path.display())) +} + +/// Ask git whether it would commit the written nupkg and its `.gitignore` +/// (#1061), probing from the uuid dir. Still ignored refuses +/// `vendor_artifact_gitignored` (the caller unwinds); git failing to +/// answer is only a warning. +async fn keep_nupkg_committable( + uuid_dir: &Path, + nupkg_path: &Path, + warnings: &mut Vec, +) -> Result<(), VendorOutcome> { + let leaf = nupkg_path + .file_name() + .map(|n| n.to_string_lossy().into_owned()) + .unwrap_or_default(); + let shown = nupkg_path.display().to_string(); + match super::npm_dir::gitignore_probe(uuid_dir, &[leaf, ".gitignore".to_string()]).await { + Ok(Some(rules)) => Err(refused( + super::npm_dir::GITIGNORED, + super::npm_dir::gitignored_detail(&shown, &rules), + )), + Ok(None) => Ok(()), + Err(why) => { + warnings.push(VendorWarning::new( + super::npm_dir::GITIGNORE_UNCHECKED, + super::npm_dir::gitignore_unchecked_detail(&shown, &why), + )); + Ok(()) + } + } } // ── nuget.config editing ─────────────────────────────────────────────────────── @@ -2616,6 +2687,89 @@ mod tests { ); } + /// An autocrlf checkout of the uuid `.gitignore` is not rewritten. + #[tokio::test] + async fn a_crlf_uuid_gitignore_is_left_alone() { + let tmp = tempfile::tempdir().unwrap(); + let path = tmp.path().join(".gitignore"); + std::fs::write(&path, "!*\r\n").unwrap(); + super::write_uuid_gitignore(tmp.path()).await.unwrap(); + assert_eq!(std::fs::read(&path).unwrap(), b"!*\r\n"); + std::fs::write(&path, "stale\n").unwrap(); + super::write_uuid_gitignore(tmp.path()).await.unwrap(); + assert_eq!( + std::fs::read_to_string(&path).unwrap(), + super::UUID_GITIGNORE + ); + } + + /// #1061: GitHub's stock VisualStudio.gitignore ignores `*.nupkg`. The + /// uuid dir gets a `.gitignore` that re-includes the vendored nupkg, so + /// the commit the vendored workflow ends with carries it. + #[tokio::test] + async fn a_nupkg_ignore_rule_is_overridden_by_the_uuid_gitignore() { + use crate::vendor::test_support::{git_project, VISUAL_STUDIO_GITIGNORE}; + let (dir, blobs, installed, record) = fixture(true, None).await; + let root = dir.path(); + if git_project(root, VISUAL_STUDIO_GITIGNORE).is_none() { + return; + } + assert!( + super::super::npm_dir::gitignored(root, &[copy_rel()]) + .await + .is_some(), + "precondition: the stock rules ignore the nupkg" + ); + let (result, entry, _w) = + unwrap_done(run_vendor(root, &blobs, &installed, &record, false).await); + assert!(result.success, "{:?}", result.error); + assert!(entry.is_some()); + assert_eq!( + std::fs::read_to_string(root.join(format!(".socket/vendor/nuget/{UUID}/.gitignore"))) + .unwrap(), + UUID_GITIGNORE + ); + assert_eq!( + super::super::npm_dir::gitignored(root, &[copy_rel()]).await, + None, + "git commits the vendored nupkg" + ); + } + + /// #1061: a rule ignoring the uuid dir itself can't be overridden from + /// inside it, so vendoring refuses before writing anything, dry run + /// included. + #[tokio::test] + async fn a_directory_ignore_rule_refuses_before_any_write() { + use crate::vendor::test_support::git_project; + for rule in [".socket/", ".socket/vendor/", "nuget/"] { + for dry_run in [false, true] { + let (dir, blobs, installed, record) = fixture(true, None).await; + let root = dir.path(); + if git_project(root, &format!("{rule}\n")).is_none() { + return; + } + let lock_before = tokio::fs::read(root.join(PACKAGES_LOCK)).await.unwrap(); + let (code, detail) = + unwrap_refused(run_vendor(root, &blobs, &installed, &record, dry_run).await); + assert_eq!(code, "vendor_artifact_gitignored", "{rule}: {detail}"); + assert!(detail.contains(rule), "{rule}: {detail}"); + assert!(!root.join(".socket").exists(), "{rule}: nothing written"); + assert!(!root.join("nuget.config").exists(), "{rule}: no config"); + assert_eq!( + tokio::fs::read(root.join(PACKAGES_LOCK)).await.unwrap(), + lock_before + ); + // An empty patch is refused too, never a calm success. + let mut empty = record.clone(); + empty.files.clear(); + let (code, _) = + unwrap_refused(run_vendor(root, &blobs, &installed, &empty, dry_run).await); + assert_eq!(code, "vendor_artifact_gitignored", "{rule}: empty patch"); + } + } + } + #[tokio::test] async fn refuses_unsafe_coordinates() { let (dir, blobs, installed, record) = fixture(true, None).await; diff --git a/crates/socket-patch-core/src/vendor/redownload.rs b/crates/socket-patch-core/src/vendor/redownload.rs index a9b2bd01f..76d8f1e65 100644 --- a/crates/socket-patch-core/src/vendor/redownload.rs +++ b/crates/socket-patch-core/src/vendor/redownload.rs @@ -525,7 +525,8 @@ async fn restore_maven_metadata( /// The owned files a Gradle tree needs beside its directory: the derived /// `maven-metadata.xml` (recomputed from the committed index) and the -/// `.gitattributes` the entry created, rewritten when missing. Needs no +/// `.gitattributes` / tree-root `.gitignore` the entry created, rewritten +/// when missing. Needs no /// download, so `repair` also runs it for a healthy entry. pub async fn restore_jvm_owned_files(root: &Path, entry: &VendorEntry) -> Result<(), String> { use super::jvm::gradle; @@ -556,6 +557,17 @@ pub async fn restore_jvm_owned_files(root: &Path, entry: &VendorEntry) -> Result wanted.push((rel.to_string(), "* -text\n".to_string())); } } + for rel in [ + gradle::GITIGNORE_REL, + super::jvm::maven_reactor::GITIGNORE_REL, + ] { + if created(rel) { + wanted.push(( + rel.to_string(), + super::jvm::coursier_tree::GITIGNORE.to_string(), + )); + } + } if created(gradle::VENDOR_GITATTRIBUTES_REL) { wanted.push(( gradle::VENDOR_GITATTRIBUTES_REL.to_string(), diff --git a/crates/socket-patch-core/src/vendor/test_support.rs b/crates/socket-patch-core/src/vendor/test_support.rs index 567ab7b10..11821eee6 100644 --- a/crates/socket-patch-core/src/vendor/test_support.rs +++ b/crates/socket-patch-core/src/vendor/test_support.rs @@ -245,6 +245,36 @@ pub(crate) async fn persist(root: &Path, key: &str, mut entry: VendorEntry) { save_state(root, &state).await.unwrap(); } +/// Make `root` a git work tree whose `.gitignore` is `rules`. `None` when +/// git is not installed (the caller skips: no git, nothing to commit). +pub(crate) fn git_project(root: &Path, rules: &str) -> Option<()> { + let git = crate::utils::process::resolve_tool("git")?; + let ok = std::process::Command::new(git) + .arg("-C") + .arg(root) + .args(["init", "-q"]) + .status() + .ok()? + .success(); + assert!(ok, "git init"); + std::fs::write(root.join(".gitignore"), rules).unwrap(); + Some(()) +} + +/// GitHub's stock `Java.gitignore` (github/gitignore), verbatim. +pub(crate) const JAVA_GITIGNORE: &str = "# Compiled class file\n*.class\n\n# Log file\n*.log\n\n\ +# BlueJ files\n*.ctxt\n\n# Mobile Tools for Java (J2ME)\n.mtj.tmp/\n\n# Package Files #\n*.jar\n\ +*.war\n*.nar\n*.ear\n*.zip\n*.tar.gz\n*.rar\n\n\ +# virtual machine crash logs, see http://www.java.com/en/download/help/error_hotspot.xml\n\ +hs_err_pid*\nreplay_pid*\n"; + +/// The package rules of GitHub's stock `VisualStudio.gitignore`. +pub(crate) const VISUAL_STUDIO_GITIGNORE: &str = "[Bb]in/\n[Oo]bj/\n[Ll]og/\n\ +# NuGet Packages\n*.nupkg\n# NuGet Symbol Packages\n*.snupkg\n\ +# The packages folder can be ignored because of Package Restore\n**/[Pp]ackages/*\n\ +# except build/, which is used as an MSBuild target.\n!**/[Pp]ackages/build/\n\ +# NuGet v3's project.json files produces more ignorable files\n*.nuget.props\n*.nuget.targets\n"; + pub(crate) fn has_warning(warnings: &[VendorWarning], code: &str) -> bool { warnings.iter().any(|w| w.code == code) } diff --git a/docs/ecosystems.md b/docs/ecosystems.md index f0d064b3c..ee78e0a24 100644 --- a/docs/ecosystems.md +++ b/docs/ecosystems.md @@ -21,7 +21,7 @@ The backticked slug in each row is the value `-e`/`--ecosystems` accepts (e.g. | Go (`golang`) | ✅ `go.mod` `replace` → `.socket/go-patches/` — see [Go: directory replaces and go.sum](#go-directory-replaces-and-gosum) | ✅ `replace` → the committed vendor tree | ✅ (free tier) fork-style `replace` → `patch.socket.dev/gopatch/` + committed `go.sum` pin; see [Go notes](#go-directory-replaces-and-gosum). Paid hosted patches are unsupported; `redirect_golang_unsupported` names the vendored remedy | | Maven (`maven`) — Maven and Gradle | ✅ in place in every copy the build consumes: each `~/.m2` copy it reads and each Gradle `files-2.1` copy; `~/.m2` `.sha1`/`.md5` sidecars are rewritten, Gradle copies get advisories; jar-member records swap in the patch service's whole jar — prefer vendored / hosted, see [Maven & NuGet caveats](#maven--nuget-caveats) and [Gradle](#gradle) | ✅ suffixed Maven repository (`-socket.` pin + `.mvn/maven.config` + fallback file repository) for every pom root, single-module or reactor, or Gradle 6.8+ same-GAV repository with settings wiring and SHA-256 checks (a root with both `pom.xml` and a Gradle build wires both); see [JVM vendoring](design/maven-vendoring.md) and [Gradle](#gradle) | ✅ fail-closed by a Socket-only `-socket.` suffix: pom projects get a pinned `` (`${property}` versions are refused); Gradle 6.8+ builds get an owned settings script, lock-entry rewrites and a resolution tripwire — see [Maven & NuGet caveats](#maven--nuget-caveats) and [Gradle](#gradle) | | sbt / Mill / scala-cli (`maven`) | ✅ Coursier caches (sbt 1.3+, sbt 2, Mill, scala-cli) and Ivy caches (sbt 0.13–1.2, `useCoursier := false`) patched in place, Coursier checksum sidecars resynced — see [Scala build tools](#scala-build-tools-sbt-mill-scala-cli) | ✅ sbt 0.13.18+: generated `socket-patch-vendor.sbt` over the committed suffixed `.socket/vendor/maven2` tree; scala-cli directory builds: owned `socket-patch.scala` + same-GAV `.socket/vendor/coursier` tree (Linux / macOS); Mill: not wired (agent or hosted guidance) | ✅ sbt 0.13.18+: one generated `socket-patch.sbt`, gated on sbt's own `sbt update` records; Mill / scala-cli: paste-able snippets only (`redirect_mill_manual_snippet`, `redirect_scala_cli_manual_snippet`) | -| NuGet (`nuget`) | ✅ in-place patching deletes `.nupkg.metadata` and advises on the `.nupkg.sha512` tamper-evidence sidecar — prefer vendored / hosted, see [Maven & NuGet caveats](#maven--nuget-caveats) | ✅ committed folder feed + `packageSourceMapping` + `packages.lock.json` contentHash pin | ✅ `nuget.config` source + source-mapping, `packages.lock.json` contentHash rewrite. See the locked-mode note in [Maven & NuGet caveats](#maven--nuget-caveats) | +| NuGet (`nuget`) | ✅ in-place patching deletes `.nupkg.metadata` and advises on the `.nupkg.sha512` tamper-evidence sidecar — prefer vendored / hosted, see [Maven & NuGet caveats](#maven--nuget-caveats) | ✅ committed folder feed (its `.gitignore` re-includes the nupkg against `*.nupkg` rules) + `packageSourceMapping` + `packages.lock.json` contentHash pin | ✅ `nuget.config` source + source-mapping, `packages.lock.json` contentHash rewrite. See the locked-mode note in [Maven & NuGet caveats](#maven--nuget-caveats) | | Composer (`composer`) | ✅ in place (`vendor/`) | ✅ `composer.lock` `dist: path` rewrite | ✅ `composer.lock` dist url + shasum rewrite; the entry's `source` and `dist.mirrors` are removed. See [composer-compatibility.md](testing/composer-compatibility.md) | | Deno (`deno`) | ✅ in place (the only mode for Deno) | ❌ refused (`vendor_unsupported_ecosystem`) | ❌ not supported | @@ -900,7 +900,10 @@ Classifier jars a build declares are vendored too, and a derived `maven-metadata keeps ranges on the vendored version. Existing pgp-only verification entries, and the classifier jars the tree serves, get a checksum. Refusals use `vendor_jvm_shape_unsupported` / `vendor_jvm_upstream_unavailable`, and partial wiring uses `vendor_jvm_degraded` -(VEX withheld). Each detail starts with `reason: :`. +(VEX withheld). Each detail starts with `reason: :`. The tree root owns a +`.gitignore` (`!*`) so a `*.jar` rule (GitHub's stock Java.gitignore) cannot drop the +vendored jars from the commit; a rule that ignores `.socket/` itself is refused +(`vendor_artifact_gitignored`) before anything is written. ### VEX