Skip to content

Commit efddc0d

Browse files
Print one JSON envelope from every command (v5.0) (#1369)
* Report artifact GC in one JSON shape and count every rollback leg (#1257, #1066) GC was reported four ways. repair and remove buried the sweep in artifact-level event details with no byte count, while rollback and scan --prune printed a hand-built `gc` object. The contract documented summary.bytesFreed, summary.bytesDownloaded and events[].bytes, but no command emitted any of them, so its GC jq recipe returned null. - json_envelope::GcReport {removedBlobs, removedDiffArchives, removedPackageArchives, bytesFreed} is built from the three sweep passes and serialized identically everywhere: the envelope's new top-level `gc` (repair, remove), rollback's `gc` and scan's `gc`. The hand-written json! blocks are gone. - summary.bytesFreed is always present and mirrors gc.bytesFreed. events[].bytes is set on the GC carrier event and on --update's downloaded event. summary.bytesDownloaded is dropped from the contract. - repair's GC carrier event no longer bumps summary.removed/verified, matching remove: summary counters count patch entries, and the sweep totals live in `gc`. - remove's human output now names the diff/package archives it sweeps. - rollback --json: rolledBack and failed now span the agent, vendored and hosted legs (#1066). A run where something failed and nothing was rolled back, already original or not installed now reports status "error" with error.code rollback_failed instead of partial_failure. Exit codes are unchanged. - CLI_CONTRACT.md: the envelope and PatchEvent schemas, the PatchAction "Emitted by" column, the per-command action matrix and the GC jq recipe now match the emitters. New unit tests pin the documented summary/gc/PatchEvent key sets against what json_envelope serializes. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Print scan's GC dry-run preview in the shared gc shape The --dry-run preview of scan --prune/--sync used its own vocabulary (prunableManifestEntries, orphanBlobs, orphanDiffArchives, orphanPackageArchives, revertableVendoredEntries, vendorOrphanDirs, bytesReclaimable). It now prints the same keys as the wet pass and every other GC-running command, counting what the pass would remove, and leaves out only the keys a real pass alone can fill (keptVendoredEntries, failedVendoredEntries, skipped, warnings). v5.0 MAJOR. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Lay the envelope foundation every --json command will share - Envelope::extra (flattened) carries a command's own payload keys beside the shared ones; set_extra refuses a shared key. - Status gains notInstalled, noMatch, noPackages and selectionRequired (get's outcomes) so no command needs a snake_case status. - PatchAction gains rolledBack; summary.rebuilt and summary.rolledBack are always present. - manifest_load_error is the one mapping from a manifest load failure to manifest_invalid / manifest_unreadable (#931); list and remove use it, and remove reports a manifest that vanished mid-run as manifest_not_found instead of manifest_invalid. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Print rollback --json as the unified envelope Every rollback outcome is now a PatchEvent: agent restores are rolledBack (verified on a dry run, installed copy in details.path), already-original and not-installed entries are skipped with already_original / package_not_installed, failures are failed with the blocking file's code. Vendored and hosted legs carry details.mode; drift-keeps are failed vendor_revert_kept (they still exit 1). Manifest entries the run drops are removed (verified on a dry run) with details.manifest. GC goes through set_gc; a requested GC that could not run adds the gc_skipped warning. Every error path prints a full envelope; manifest load failures use manifest_load_error. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Map manifest load failures to one error code on every envelope command (#931) apply, apply --check, repair, vendor, vendor --check and vex now report an unparseable manifest as manifest_invalid and an unreadable one as manifest_unreadable through json_envelope::manifest_load_error, like list and remove. Gone: apply_failed / repair_failed for a manifest load failure, and vendor's undocumented invalid_manifest. vex keeps exit 2. apply's run_locked reports a manifest that vanished mid-run as manifest_not_found. Exit codes are unchanged. remove's GC carrier now matches repair's (details.count + details.checked, bytes); the per-kind totals are only in gc. The in-place rollback leg is reported as rolledBack events (summary.rolledBack) instead of the carrier's details.rolledBack. Sweep failures reach --json as cleanup_failed warnings. Vendored- and hosted-leg events of remove and repair carry details.mode; repair's download carrier key is now details.downloadMode. Sidecar file actions and severities serialize with the envelope's camelCase enum convention (wire-identical for today's one-word values). Adds a regression matrix test over list, remove, apply, apply --check, repair, vendor, vendor --check and vex. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Assert the rollback envelope in the rollback test suites Adds tests/common/rollback_json.rs, per-leg views projected from events, and moves the rollback-centric suites onto summary/events. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Move every rollback --json assertion onto the envelope Covers the mixed suites (apply multicopy, dispatch, global scope, json error shape, hosted/vendored takeovers) and the ignored e2e suites, which were fixed by reading. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Document the unified manifest-load codes and remove/repair event shapes CLI_CONTRACT.md: manifest_invalid / manifest_unreadable now list every command that loads the manifest (v5.0, MAJOR, #931; vex keeps exit 2); the top-level error table gains the codes apply, remove, vendor, vex and --update already emitted; the remove/repair matrix rows, the rolledBack action row and new notes describe the shared GC carrier (details.count + checked), details.mode on vendored/hosted-leg events, repair's details.downloadMode, and the sidecars[] value-tag casing. apply --check prints a noManifest envelope when the manifest vanishes between the existence probe and the read, instead of printing nothing. Tests pin details.mode on remove's hosted/vendored legs and repair's vendored phase. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Expect rollback_failed when every Bun global-store package is refused All packages refused and nothing rolled back is a failed run since #1066. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Document rollback's unified envelope in CLI_CONTRACT.md The rollback contract's JSON section now maps each outcome to its event, the action matrix and migration status list rollback as an envelope command, and the errorCode / EnvelopeError tables carry its codes. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Move scan and get --json onto the shared envelope scan and get now print one serialized Envelope per run (command, status from the Status enum, dryRun, events, summary, warnings, gc, vex), built by one emitter each (scan::emit_scan, get::emit): - The agent download engine records per-patch events into the caller's envelope (downloaded / updated + oldUuid with the patch metadata in details, skipped already_in_manifest, failed with download_failed / patch_no_applicable_files / blob_write_failed or the refusal code), then applied / failed events for the nested apply; it never prints, so scan --mode agent --json can no longer put two JSON documents on stdout when the lock is held or the manifest is unreadable. - Hosted runs record applied (verified on --dry-run) / skipped events tagged details.mode hosted; redirect shrinks to {mode, rewrittenFiles} and its warnings move to the top level. - Vendored runs merge the vendor engine's events, warnings and sidecars into the outer envelope (details.mode vendored) instead of nesting a vendor envelope; the dry-run preview becomes verified / skipped events. - scan --prune records pruned manifest entries as removed events (details.manifest), reverted vendored entries as removed, drift-kept and failed reverts as skipped, the sweep as the envelope's gc, and a pass that could not take the lock as a gc_skipped warning. - get's statuses become notFound / noPackages / noMatch / notInstalled / paidRequired / selectionRequired; selection_required options use publishedAt; every error prints a full envelope. - Exit parity (#1062): one fetch_details decision for both outputs, no human-only fetched == 0 failure, hosted --json uses the policy-gated prune, and the human vendored dry run previews the --prune GC. - A manifest that exists but cannot be loaded fails agent-mode scan with manifest_invalid / manifest_unreadable and is a warning elsewhere. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Record the envelope rollback keys in the Python backtest harnesses Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Document the scan/get envelope and add shape and parity tests CLI_CONTRACT.md: scan and get join the envelope (migration status, action matrix, PatchAction rows, error codes, a new "scan and get JSON" section replacing the patches[] entry shape, jq recipes), and the scan mode / get paragraphs describe events, the redirect payload and top-level warnings instead of the nested blocks. Tests: shared envelope invariants in tests/common/envelope.rs, a legacy redirect rebuild for the in-memory engine parity harness, scan envelope tests (agent events, dry run, lock held prints one document, manifest load errors) and the #1062 human/JSON parity tests, get envelope tests. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Update scan and get tests for the envelope shape Every test that read the legacy scan/get JSON (top-level counters, patches[], apply / download / vendor / redirect blocks, nested and string warnings, snake_case statuses and actions, gc sub-keys) now reads the envelope: events (details.mode for the hosted and vendored legs), summary, top-level warnings, redirect.rewrittenFiles and the scan payload keys. The in-memory hosted engine parity tests compare the engine's redirect block against one rebuilt from the disk run's envelope. Env-gated e2e files were updated by reading. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Silence an unused binding in the yarn berry hosted test Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Restore per-caller bun.lockb rollback status The shared bun.lockb rollback helper expected status "error" for every caller, so the hosted -> vendored -> hosted takeover leg went red: there a manifest record makes the agent leg report the copy already original, and the run is correctly a partial_failure. A branch rewrite had dropped the earlier fix for this. Each caller now states which outcome it expects, so the takeover leg checks partial_failure and the hosted-only alias/transitive shapes keep checking rollback_failed. Assisted-by: Claude Code:claude-opus-5-5 * Read scan's unified dry-run GC key in the #1062 prune preview test main's new test read gc.prunableManifestEntries, which this branch renamed to prunedManifestEntries for the dry-run preview. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Read the v5 envelope in e2e suites, compat harnesses and the scan bench The envelope retired scan's legacy counters and nested blocks, but the ignored e2e suites, the Python compatibility harnesses and the scan benchmark still read them, so every compat workflow failed. - e2e (hosted production, gem, vlt, bun.lockb, pypi/yarn hosted): hosted pins are the details.mode "hosted" applied/verified events, skips the hosted skipped events, warnings the top-level warnings[] (omitted when empty), prune reverts the vendor_reverted events, hosted rollback restores rolledBack events. - scripts/backtest-{bun,uv,vlt,pdm,pipenv,poetry}.py: per-leg event counts replace redirect.redirected / vendor.summary.applied / apply.applied, and codes come from error.code, event errorCodes and warnings[]; the harness unit tests use envelope fixtures. - socket-patch-bench: validation accepts both shapes, since the base and head binaries straddle the change (packagesWithPatches and totalPatches derive from packages[], redirected from the hosted events). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Normalize CRLF in the contract tests; count every rollback leg in bun.lockb e2e - json_envelope's contract tests read CLI_CONTRACT.md through one LF-normalized copy, so a Windows checkout (CRLF) no longer misses the ```jsonc fence (review thread on #1273; full-scope test windows). - e2e_bun_lockb's shared-bundled case also has an agent copy that fails (hash_mismatch on a bundled copy the patch never touched); with #1066's all-leg counter, `failed` is the hosted refusal plus those agent failures. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Carry the vendor engine's error into the scan/get envelope The vendor engine marks a hard error and still returns Ok for an unreadable vendor ledger, a refused group commit and vendor_commit_failed. merge_vendor_envelope dropped venv.error, so a corrupt .socket/vendor/state.json under scan/get --mode vendored --json printed a bare partialFailure with no code. The merge now marks the outer envelope with the engine's {code, message}; a caller aborting with its own error still overrides it. Unit tests pin both arms. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Read the unified dry-run GC key in main's Cargo shared-cache prune test main's #1278 e2e (sync_keeps_entry_whose_shared_cache_copy_is_still_patched) reads the preview's prunableManifestEntries, which the one GC shape renamed to prunedManifestEntries; the contract text it brought gets the same rename. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Read the cargo-cache keep under the one GC shape's preview key #1305's sync_keeps_entry_whose_shared_cache_copy_is_still_patched asserted the scan --sync --dry-run preview via gc.prunableManifestEntries, a key #1273 retires: under the one GC shape a dry run reports would-be prunes as gc.prunedManifestEntries. The keep logic itself merged intact (preview and wet pass both keep the still-patched itoa entry, and the wet pass reports cargo_cache_patch_kept in gc.warnings[] with the purl and the rollback remedy), so only the test read the wrong key. Assert the same ryu-only list under the new key. CLI_CONTRACT.md's scan --prune paragraph still named the retired preview keys (prunableManifestEntries, revertableVendoredEntries, vendorOrphanDirs); point it at the unified keys and note that the preview, like every warnings[] entry, carries no cargo_cache_patch_kept. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
1 parent 10874cc commit efddc0d

180 files changed

Lines changed: 8025 additions & 5142 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎crates/socket-patch-bench/README.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,8 @@ sample:
6666
- exit code 0 and one JSON document on stdout, `status: success`;
6767
- `scannedPackages`, `lockfileOnlyPackages`, `packagesWithPatches` and
6868
`totalPatches` equal what the generated project contains;
69-
- hosted runs: `redirect.redirected` is every patch, `rewrittenFiles` is
69+
- hosted runs: every patch is pinned (`redirect.redirected` before v5, the
70+
hosted `applied` events since), `rewrittenFiles` is
7071
exactly the expected set, nothing is skipped, every warning code is one
7172
the scenario expects, and each reported file really changed on disk;
7273
- dry runs: nothing on disk changed;

‎crates/socket-patch-bench/src/engine.rs‎

Lines changed: 73 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -270,6 +270,69 @@ fn warning_codes(v: &Value) -> Vec<String> {
270270
.unwrap_or_default()
271271
}
272272

273+
// The base and head binaries may straddle the v5 JSON envelope change
274+
// (`scan --json`'s legacy counters became events), so every reading below
275+
// accepts both shapes: the legacy key when present, else the envelope's.
276+
277+
/// How many discovered packages have patches: legacy `packagesWithPatches`,
278+
/// else the length of the `packages` discovery array.
279+
fn packages_with_patches(v: &Value) -> Value {
280+
match &v["packagesWithPatches"] {
281+
Value::Null => v["packages"]
282+
.as_array()
283+
.map_or(Value::Null, |a| a.len().into()),
284+
n => n.clone(),
285+
}
286+
}
287+
288+
/// How many patches discovery found: legacy `totalPatches`, else the sum of
289+
/// `packages[].patches[]`.
290+
fn total_patches(v: &Value) -> Value {
291+
match &v["totalPatches"] {
292+
Value::Null => v["packages"].as_array().map_or(Value::Null, |a| {
293+
a.iter()
294+
.map(|p| p["patches"].as_array().map_or(0, Vec::len))
295+
.sum::<usize>()
296+
.into()
297+
}),
298+
n => n.clone(),
299+
}
300+
}
301+
302+
/// The hosted events (`details.mode: "hosted"`) of a v5 envelope.
303+
fn hosted_events(v: &Value) -> impl Iterator<Item = &Value> {
304+
v["events"]
305+
.as_array()
306+
.into_iter()
307+
.flatten()
308+
.filter(|e| e["details"]["mode"] == "hosted")
309+
}
310+
311+
/// How many packages a hosted run pinned (or would pin, on a dry run):
312+
/// legacy `redirect.redirected`, else the hosted `applied` / `verified`
313+
/// events.
314+
fn hosted_pins(v: &Value) -> Value {
315+
match &v["redirect"]["redirected"] {
316+
Value::Null if v["events"].is_array() => hosted_events(v)
317+
.filter(|e| e["action"] == "applied" || e["action"] == "verified")
318+
.count()
319+
.into(),
320+
n => n.clone(),
321+
}
322+
}
323+
324+
/// The packages a hosted run skipped: legacy `redirect.skipped[]`, else
325+
/// the hosted `skipped` events.
326+
fn hosted_skips(v: &Value) -> Vec<Value> {
327+
match v["redirect"]["skipped"].as_array() {
328+
Some(a) => a.clone(),
329+
None => hosted_events(v)
330+
.filter(|e| e["action"] == "skipped")
331+
.cloned()
332+
.collect(),
333+
}
334+
}
335+
273336
/// Check that a run did the work the fixture calls for. Any mismatch makes
274337
/// the sample meaningless, so it is an error, not a footnote.
275338
fn validate(
@@ -326,12 +389,12 @@ fn validate(
326389
),
327390
(
328391
"packagesWithPatches",
329-
v["packagesWithPatches"].clone(),
392+
packages_with_patches(&v),
330393
want_patched.len().into(),
331394
),
332395
(
333396
"totalPatches",
334-
v["totalPatches"].clone(),
397+
total_patches(&v),
335398
p.fixture.patches.len().into(),
336399
),
337400
];
@@ -342,6 +405,7 @@ fn validate(
342405
}
343406
let mut codes = warning_codes(&v["warnings"]);
344407
let redirect = &v["redirect"];
408+
// Pre-v5 binaries nest the hosted warnings under `redirect`.
345409
codes.extend(warning_codes(&redirect["warnings"]));
346410
let unexpected: Vec<&String> = codes
347411
.iter()
@@ -354,11 +418,11 @@ fn validate(
354418
+ &serde_json::to_string(&redirect["warnings"]).unwrap_or_default()
355419
));
356420
}
357-
let skipped = redirect["skipped"].as_array().map(Vec::len).unwrap_or(0);
358-
if skipped > 0 {
421+
let skipped = hosted_skips(&v);
422+
if !skipped.is_empty() {
359423
return Err(format!(
360424
"redirect skipped packages: {}",
361-
redirect["skipped"]
425+
Value::Array(skipped)
362426
));
363427
}
364428
let rewritten: Vec<String> = {
@@ -377,10 +441,11 @@ fn validate(
377441
want_rewritten.sort();
378442
match kind {
379443
Kind::Hosted | Kind::DryRun => {
380-
if redirect["redirected"] != e.redirected {
444+
let redirected = hosted_pins(&v);
445+
if redirected != e.redirected {
381446
return Err(format!(
382-
"redirect.redirected: got {}, want {}",
383-
redirect["redirected"], e.redirected
447+
"redirect.redirected: got {redirected}, want {}",
448+
e.redirected
384449
));
385450
}
386451
if rewritten != want_rewritten {

‎crates/socket-patch-cli/CLI_CONTRACT.md‎

Lines changed: 263 additions & 253 deletions
Large diffs are not rendered by default.

0 commit comments

Comments
 (0)