fix(sbom, vex, build): keep artifacts with the image in every repository - #282
Draft
reyreavman wants to merge 6 commits into
Draft
fix(sbom, vex, build): keep artifacts with the image in every repository#282reyreavman wants to merge 6 commits into
reyreavman wants to merge 6 commits into
Conversation
werf cleanup deletes stages from the final repo and then collects orphaned sha256-* artifact indexes there, but no e2e run exercised that path against a live SBOM. The existing final-repo test stops right after the build, and the cleanup case in the VEX suite asserts deletion in the stages repo, so a regression that dropped an in-use SBOM from the final repo would go unnoticed. Build a project with --final-repo, read the SBOM back from the final repo and from the stages repo, run cleanup twice with the built commit reachable from origin so retention policies hold the stage, and read the SBOM again after each run. Address the stages repo by its stage tag rather than by the digest from the build report, because copying a stage into the final repo may change the manifest digest. Keep the bare remote outside the work tree: the fixture adds the whole project directory to the image, so an in-tree remote breaks giterminism. Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
Collaborator
Author
Verification
Review focus
Follow-up
|
…final repo Two gaps around --final-repo had no coverage. A project where one image is the base of another (fromImage) or imports files from another exercises the base/import SBOM lookup during convergence of the dependent image. If that lookup resolves against a repository that does not hold the base SBOM, the build fails with advice to rebuild with SBOM generation enabled, which cannot help. Cover both dependency kinds: build with --final-repo against a clean registry, read the merged SBOM from the final repo, and check a rebuild serves both SBOMs from cache instead of regenerating them. For a multi-platform image the SBOMs belong on the platform manifest digests, never on the index digest, and the registry-level index copy into the final repo preserves the platform manifest digests. Assert each platform manifest carries its SBOM in both the stages repo and the final repo, the index digest carries none in either, and sbom get resolves a platform SBOM from the final repo through the index digest reported to the user. The stapel fixture builds for both platforms on the Docker backend with WERF_EXPERIMENTAL_STAPEL_ARM, following the multi-platform signing suite. Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
…ookup Rebuild the dependent-images fixtures from scratch instead of the trusted builder base. An image derived from the builder base inherits the io.deckhouse.internal.builder label, and with WERF_E2E_ALLOW_LOCAL_BUILDER_IMAGES a failed base SBOM lookup silently degrades into ErrSbomNotRequired while the asserted base packages surface through the filesystem scan — the previous entries passed without exercising the lookup at all. The scratch fixtures leave no escape: the fromImage entries now fail on current main, reproducing the broken base SBOM lookup under --final-repo, and need no builder base or external environment beyond the registry. Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
Add specs/020-sbom-artifact-locality: SBOM, VEX and other attached artifacts live alongside the image in every registry-backed repository that holds the image. The spec states the contract over pairs of a repository and a subject digest, fixes the convergence target to the repository the image was built in, requires every stage copy to carry attached artifacts with repair on repeat, defines the digest-preservation rule for backend-mediated copies, and records the root cause of the current breakage: two unintended regressions inside PR #225 that first killed the propagation step and then moved the single remaining copy into the final repo. Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
With --final-repo the SBOM and VEX of an image existed in exactly one copy, in the final repo: publishFinalImage overwrote the image's content tag descriptor with the final repo one, artifact convergence resolved its target through that descriptor, and the propagation step into the final repo had been dead since the descriptor stopped being stored where the step read it. Reading the SBOM from the stages repo returned nothing, a project where one final image is built from another failed with advice to enable SBOM generation that was already enabled, and for a multi-platform image the artifacts were copied onto the index digest — where no retrieval path looks and where the second platform's copy displaced the first. Split the two roles of the content tag descriptor: contentTagDesc keeps describing the image in the repository it was built in and is the single target for artifact convergence and for the base and import SBOM lookups, while the new finalContentTagDesc carries the published final repo descriptor for the build report. Replace the per-platform SBOM propagation with one step that runs after both SBOM and VEX convergence and carries every attached artifact kind: for a single-platform image onto the published final descriptor, for a multi-platform image onto the platform manifest digests preserved by the registry-level index copy, with image-level artifacts following the index digest. The step runs on every build and the copies are idempotent, so a repository holding the image without its artifacts is repaired by the next run. Extend the same contract to the remaining copy paths: stage copies between repositories carry the artifacts of every manifest an index references and repair a destination that already holds the manifest, werf stages copy and bundle copy carry the artifacts of the images they transfer, and the backend-mediated secondary-to-primary copy carries artifacts only when the digest survived the copy — a statement about the source digest is not a statement about the destination digest — leaving a warning about what was left behind otherwise. Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
Collaborator
Author
Verification
Review focus
Follow-up
|
Final repo cleanup had its protection inverted: a final stage whose stage ID was absent from the stages repo was marked protected, while nothing protected the ones still present, so every final stage whose counterpart survived cleanup landed in the deletion set. One werf cleanup run against --final-repo emptied the final repo of every live image while keeping whatever the stages repo no longer knew about. Nothing caught it because no test ran cleanup against a final repo. Restore the mirror semantics: a final stage whose stage ID is still present in the stages repo after cleanup is protected (reason "found in repo"), the rest are deleted. Rewrite the unit test that pinned the inverted behavior and add the deletion case. Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
With
--final-repo, the SBOM and VEX of an image existed in exactly one copy, in the final repo: reading them from the stages repo returned nothing, a project where one final image is built from another (fromImage/import) failed with advice to enable SBOM generation that was already enabled, and for a multi-platform image the artifacts were unreachable in the final repo altogether. This PR adds the artifact locality contract (specs/020-sbom-artifact-locality), e2e tests that falsify each breakage, and the fix: artifacts live alongside the image in every registry-backed repository that holds the image.Repro of the dependent-build failure from a clean checkout: a two-image
werf.yaml(basefrom scratch,appwithfromImage: base),build.sbom.enable: true, thenwerf build --repo <r> --final-repo <rf>against a clean registry fails withthe image ... must have an SBOM artifact attached; to generate an SBOM for the image, rebuild it with SBOM generation enabled.What
Convergence and lookups
--final-repo; publishing elsewhere can no longer move where artifacts are written.fromImageandimport) with--final-repoon a clean registry succeeds, and the dependent image's SBOM contains the base image's components.--final-repois used;werf stage imagestill prints the stages repo reference.Placement and propagation
--final-repo,werf sbom getandwerf attest ls/get/verifyreturn the artifacts against both the stages repo and the final repo.werf sbom get --repo <final-repo> --digest <index-digest> --platform <p>returns that platform's SBOM. Previously both platform SBOMs were copied onto the index digest, where the second displaced the first.Copy paths
werf stages copyand bundle copy carry the attached artifacts of the images they transfer.Cleanup
Verification status
fromImage+import, both docker backends) and multi-platform placement — red on main, green with the fix.Why
Two unintended regressions inside PR #225 produced the single-copy state.
55dc5ea4b(content-based tag) stopped storing the final repo descriptor where SBOM propagation read it, silently killing the copy into the final repo.a891ed411(nil-panic fix) then routed convergence through a shared accessor that prefers the published descriptor, moving the single remaining copy from the stages repo into the final repo. The root defect is one descriptor answering two questions: "the image as published" (what the report needs) and "the image being described" (what convergence and the base/import SBOM lookups need). The fix splits them:contentTagDescalways describes the image in its build repository, the newfinalContentTagDesccarries the published final repo descriptor.The alternative — teaching cleanup to protect the single final-repo copy, as the original review card suggested — was rejected: cleanup was never the problem (it already protects live images), and a single copy stays fragile no matter how careful the cleanup is. The contract and its rationale live in
specs/020-sbom-artifact-locality/spec.md.