OCPNODE-2018: Add OCI Referrers API support for signature discovery

[saschagrunert]

Ref #221

Add support for the OCI Distribution Spec 1.1 Referrers API for both reading and writing sigstore signatures.

Reading: Query GET /v2/<name>/referrers/<digest> with pagination support, fall back to the referrers tag schema for registries without native API support, filter by sigstore artifact types, and deduplicate signatures found across both the referrers and cosign tag sources.

Writing: Push each sigstore signature as an individual OCI artifact manifest with a subject field pointing to the target image. Uses an empty config (application/vnd.oci.empty.v1+json) and the signature payload as a single layer. Deduplicates by precomputing the artifact manifest digest and checking against existing referrers before uploading. Maintains the referrers tag schema index for registries without native API support. The referrers write is best-effort so a failure does not block the established cosign tag path.

Both paths are gated on the existing use-sigstore-attachments flag in registries.d configuration.

Consumer PRs for CI testing:

• CRI-O: WIP: OCPNODE-2018: Vendor OCI Referrers API support and add integration tests cri-o/cri-o#10269
• Skopeo: WIP: OCPNODE-2018: Vendor OCI Referrers API support and add system tests skopeo#2971

[mtrmac]

Thanks! I do think we need this support.

A literally 2-minute look.

Once we add this, I think we need to add write support as well; e.g. skopeo copy needs to not break validity of signed images.

That might need a registries.d configuration option to choose which destination to use. (And then we need to be rather careful about “amplification”, we should not end up with repeated copies of the same image adding an unbounded number of signature objects, e.g. if each copy read from old+new APIs and wrote all signatures to new, we might be adding the “old” signatures to the new API on every copy.)

[mtrmac]

Generally, yes — but IIRC the OpenShift integrated registry has a pretty small limit (because the signatures are stored in the API server) that is not difficult to hit. Is that going to be a problem?

[saschagrunert]

Good point. Changed to use the cosign tag path only as fallback: if the Referrers API (or its tag schema fallback) finds signatures, the legacy cosign tag is skipped entirely. This avoids duplicates and the extra network request.

[mtrmac]

That’s not obviously correct… what happens if, for a digest with referrer signatures, an old version of skopeo copy --sign-by-sigstore… adds new signatures, to the cosign path?

I haven’t spent much time thinking about this but I think we will need to actually deduplicate individual objects. (And for writes… I don’t know, that’s harder. Leave all existing signatures as they are? Implement an option to write everything to referrers? (to write everything to cosign-v2 tags, as well?)?)

[saschagrunert]

Changed the approach: both referrers and the cosign tag path are now always checked, and signatures are deduplicated by content digest afterward (deduplicateSigstoreSignatures hashes each signature's serialized blob). This handles the case where an old skopeo adds new signatures to the cosign path while referrers already exist.

For the write side, I agree that's a separate concern. I'd suggest handling it in a follow-up: leave existing signatures as they are for now, and add a registries.d option for choosing the write destination later.

[mtrmac]

For the write side, I agree that's a separate concern.

To be clearer I think the missing write support is a blocker for merging this; not something that can wait.

[saschagrunert]

Okay, write support has been added now. Signatures are pushed as individual OCI artifact manifests with subject, empty config, and the payload as a single layer. Dedup by precomputed manifest digest, tag schema index maintained for non-native registries. The referrers write is best-effort to avoid blocking the cosign tag path.

[harche]

Thanks for this. Could you add unit tests for the cases called out inline, including Referrers pagination via Link?