fix(docker): make Tentacle images OCI-compliant (FD-492) by todthomson · Pull Request #1259 · OctopusDeploy/OctopusTentacle

todthomson · 2026-06-18T07:32:38Z

Linear: FD-492 — Image octopusdeploy/tentacle not OCI Compliant

Background

Customer FD-492 (Zendesk 209584, ARUP Laboratories) reported that the octopusdeploy/tentacle Docker image is not OCI compliant: Podman rejects it when it is used as a base image (FROM octopusdeploy/tentacle …).

The root cause is not Dockerfile content and not the Debian 11→12 base-image upgrade (#1218 / EFT-3311) — that PR does not touch the manifest format, and building the old vs new image locally shows no difference. The actual problem is inconsistent/mixed manifest media types produced by the build/push pipeline. Docker tolerates this; strict OCI clients like Podman do not.

What Octopus Server actually does (verified in the OctopusDeploy repo)

The triage note claimed Server "fixed the same issue in Jan 2026." I checked the Server repo history directly and that framing is not accurate. There is no oci-mediatypes/--provenance flag combination anywhere in Server's history (git log -S finds zero hits). Instead, Server builds its Linux image single-architecture (linux/amd64) with the classic Docker builder (DockerTasks.DockerBuild + DockerTasks.DockerPush, nuke-build/Build.DockerImages.cs), producing one self-consistent Docker schema2 manifest with no manifest list and no attestation index. Server avoids the problem by never producing a multi-arch/attestation manifest, rather than by fixing one with flags.

Why this PR uses buildx flags instead of copying Server

The Tentacle Kubernetes agent image is genuinely multi-arch (linux/arm64,linux/amd64) and therefore must be built with buildx and a manifest list — it can't adopt Server's single-arch classic-build approach without dropping arm64. For a multi-arch buildx image, the correct equivalent remedy is to force a single OCI media type and drop the attestation index.

Manifest evidence (verified against the published registry)

octopusdeploy/tentacle:latest (the customer's image) — confirmed two distinct OCI violations:

The OCI index mixes manifest formats:

Platform	Manifest mediaType
linux/amd64	`application/vnd.oci.image.manifest.v1+json` (OCI — buildx)
windows/amd64	`application/vnd.docker.distribution.manifest.v2+json` (Docker schema2 — classic `docker build`/`push`)

A strict OCI consumer rejects an OCI index that references a Docker-schema2 manifest.

Even the linux/amd64 manifest mixes layer media types — application/vnd.docker.image.rootfs.diff.tar.gzip ×4 (inherited debian base layers) + application/vnd.oci.image.layer.v1.tar+gzip ×8 (new layers). This is the literal "mixes two layer format standards", normalized by oci-mediatypes=true.

kubernetes-agent-tentacle:latest (this repo's image) — current published index contains 4 manifests: arm64, amd64, plus two unknown/unknown attestation-manifest entries (buildx provenance). Its per-platform layers are already uniformly OCI, so here the defect is purely the attestation index — removed by disabling default attestations (BUILDX_NO_DEFAULT_ATTESTATIONS=1). (Per-platform layer types verified all-OCI.)

Changes

Build the Kubernetes agent image with consistent OCI media types (build/Build.Pack.cs)
- On push, use --output type=image,oci-mediatypes=true,push=true and BUILDX_NO_DEFAULT_ATTESTATIONS=1 so the whole manifest uses a single OCI media type with no attestation index.
- The local --load path (used by tests/dev) is unchanged.
Modernize image labels across all four Dockerfiles
- Replace the deprecated org.label-schema.* namespace (retired 2016) with the standard org.opencontainers.image.* annotation keys.
- Use the SPDX identifier Apache-2.0 for the license.

⚠️ Follow-up required (out of scope for this repo)

The customer-reported image, octopusdeploy/tentacle (docker/linux/Dockerfile), is amd64-only on Linux but published in a multi-platform tag alongside the Windows image, and is built in the TeamCity pipeline, not in this repo's NUKE build. Closing FD-492 for that image requires, in TeamCity: (a) build/push the Linux image with oci-mediatypes=true so its layers are uniformly OCI, and (b) assemble the multi-platform tag so the Windows manifest is also OCI (otherwise the OCI index keeps referencing a Docker-schema2 manifest). This PR fixes the Kubernetes agent image (built here) and the shared label metadata.

Testing notes

dotnet build build/_build.csproj succeeds.
Verify the pushed manifest with docker buildx imagetools inspect --raw <tag>: the index should contain only the two platform manifests (no unknown/unknown attestation manifests), and every layer mediaType should be application/vnd.oci.image.layer.v1.tar+gzip. Plus a podman pull / FROM smoke test.

🤖 Generated with Claude Code

Note (review follow-up): the attestation switch is standardised on BUILDX_NO_DEFAULT_ATTESTATIONS=1 to match the docker buildx bake step in OctopusDeploy/TeamCity-Configuration#112 (identical across buildx build and bake, covers all default attestations, and avoids a post-PATH CLI flag).

OCI annotations (review follow-up): in addition to the Dockerfile LABELs (which set org.opencontainers.image.* on the image config), the k8s agent build now also stamps those annotations onto the image index and each platform manifest via the buildx exporter annotation-index.* / annotation.* options — the spec-preferred location registries/OCI tooling read. Verified locally on a multi-arch build. The linux octopusdeploy/tentacle :latest index is assembled by a separate docker manifest create step (TeamCity-Configuration), so its index annotations are deferred to the manifest-step follow-up (switching to docker buildx imagetools create --annotation).

CLAassistant · 2026-06-24T06:34:29Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ todthomson
❌ Test

Test seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

todthomson · 2026-06-24T09:58:27Z

Code review

Reviewed at current state (5 commits; 5 files). Substantive change is Build.Pack.cs (k8s agent image); the 4 Dockerfiles are label modernization + keep-in-sync comments.

What it does

Build.Pack.cs (k8s agent image): on push, emits type=image,oci-mediatypes=true,push=true with OCI org.opencontainers.image.* annotations on the index + each platform manifest, and sets BUILDX_NO_DEFAULT_ATTESTATIONS=1. Load path unchanged. A single buildDate is reused for the BUILD_DATE build-arg and the created annotation.
4 Dockerfiles: org.label-schema.* → org.opencontainers.image.* (SPDX Apache-2.0), plus keep-in-sync pointer comments.

Correctness — solid (compile-verified, behaviour verified locally)

oci-mediatypes=true + annotation-index.*/annotation.* → annotations confirmed on both the index and per-platform manifests (incl. values with spaces).
BUILDX_NO_DEFAULT_ATTESTATIONS=1 via AddProcessEnvironmentVariable → no attestation index; cleaner than a post-PATH --provenance=false, and aligned with Improve wording to align with octo CLI docs for space parameter #112/Support username/password again #113.
buildDate scoping correct; compiles 0/0.
Works with the k8s TeamCity step (which --uses a docker-container builder; type=image,push=true requires exactly that).

Issues / notes

--output CSV comma fragility — now guarded. Annotations are folded into the comma-separated --output option string, so a comma in a value would silently corrupt it. Safe today (no value has a comma), but the description was the realistic future candidate. Addressed: the build now throws a clear error if any annotation value contains a comma. (NUKE's DockerBuildxBuild has no --annotation setter, so the --output CSV is the pragmatic mechanism; the guard future-proofs it.)
(info) Annotations only on the push path — expected; the load/dev path doesn't need them. The config created LABEL is still set on both paths via the build-arg.
(info) Metadata duplication — mitigated via the keep-in-sync comments.

Conventions / tests

Comments are excellent — they explain the why (media-type mixing, the env-var choice, the annotation rationale) and point at the sibling LABELs.
No automated tests — inherent for a buildx-invocation change; covered by post-merge imagetools inspect verification (expect 2 platform manifests, no attestation entries, all-OCI layers, annotations present).

Verdict

Clean, correct, minimal, compile-verified. No blockers. The one substantive review point (--output CSV comma fragility) is now guarded; everything else is informational.

Strict OCI clients such as Podman reject the Tentacle Kubernetes agent image when it is used as a base image: buildx emits a manifest that mixes Docker and OCI layer media types and wraps it in an OCI index containing provenance attestation manifests. - Build the Kubernetes agent image (which must be multi-arch, so it has to use buildx) with `type=image,oci-mediatypes=true,push=true` and `--provenance=false`, so the whole manifest uses a single, consistent OCI media type with no attestation index. - Replace the deprecated `org.label-schema.*` labels with the standard `org.opencontainers.image.*` annotation keys across all Dockerfiles, and use the SPDX `Apache-2.0` license identifier. Note: the customer-reported `octopusdeploy/tentacle` (docker/linux) image is amd64-only and is built in the TeamCity pipeline, not in this repo. Like Octopus Server's Linux image, the simplest fix there is a classic single- arch `docker build` (no buildx manifest list / provenance); if that pipeline uses buildx, apply the same flags as above. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…IONS (FD-492) Use BUILDX_NO_DEFAULT_ATTESTATIONS=1 instead of --provenance=false so the k8s agent buildx build here matches the docker buildx bake step in the TeamCity Linux image build (OctopusDeploy/TeamCity-Configuration#112). The env var is identical across buildx build and bake, covers all default attestations (not just provenance), and avoids passing a CLI flag after the positional build path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…/manifests (FD-492) The Dockerfile LABELs set org.opencontainers.image.* on the image config; this also stamps the same metadata as OCI annotations on the image index and each platform manifest (via the buildx image-exporter annotation-index.* / annotation.* options), which is the spec-preferred location that registries and OCI tooling read. Verified locally that the annotations land on both the index and the per-platform manifests. Note: the linux octopusdeploy/tentacle :latest index is assembled by a separate 'docker manifest create' step (TeamCity-Configuration); annotating that index is deferred to the manifest-step follow-up that switches it to 'docker buildx imagetools create --annotation'. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Review nit (E14): the org.opencontainers.image.* metadata is duplicated across the Dockerfile LABELs, the k8s annotations in Build.Pack.cs, and the TeamCity manifest step. Add short keep-in-sync comments pointing at the sibling definitions so the duplication doesn't silently drift. Comment-only. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…FD-492) Review nit (E16): capture DateTime.UtcNow once and reuse it for both the BUILD_DATE build arg (feeding the Dockerfile created LABEL) and the created OCI annotation, so the image config and manifest annotation report the same instant instead of two UtcNow calls milliseconds apart. Compile-verified. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Review follow-up: the annotation values are folded into the comma-separated buildx --output option list, so a comma in a value would silently corrupt it. Throw a clear error if any value contains a comma. Compile-verified. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

LukeButters

I guess this makes sense, I didn't know about OCI until today.

matt-richardson · 2026-06-25T03:45:50Z

+                // set the same metadata on the image *config*; annotations are the spec-preferred
+                // location that registries and OCI tooling read from the manifest/index.
+                // Keep these values in sync with the org.opencontainers.image.* LABELs in
+                // docker/kubernetes-agent-tentacle/Dockerfile.


Claude is over commenting - can we trim it down to things that future people will need when looking at the code.

Reasoning for "why are we making the changes" are much better as comments on a PR IMHO.

APErebus · 2026-06-25T05:58:57Z

+                // annotation-index.* lands on the image index, annotation.* on each platform manifest.
+                // Guard: annotation values are folded into the comma-separated buildx --output option
+                // list, so a comma in a value would corrupt it (the keys are fixed and comma-free).
+                var output = "type=image,oci-mediatypes=true,push=true";
+                foreach (var (key, value) in annotations)
+                {
+                    if (value.Contains(','))
+                        throw new InvalidOperationException(
+                            $"OCI annotation '{key}' value must not contain a comma; it would break the buildx --output option list: '{value}'");
+
+                    output += $",annotation-index.{key}={value},annotation.{key}={value}";
+                }


We are controlling the annotations above and FullSemVer and buildDate will never have comma's, so this is a completely unnecessary guard + comment

todthomson force-pushed the eft/tod/fix/FD-492-OCI-compliance-etc branch 2 times, most recently from a909c62 to 70a0178 Compare June 23, 2026 01:40

todthomson marked this pull request as ready for review June 23, 2026 09:08

todthomson requested review from a team as code owners June 23, 2026 09:08

todthomson marked this pull request as draft June 24, 2026 02:23

todthomson marked this pull request as ready for review June 24, 2026 10:00

todthomson and others added 6 commits June 24, 2026 20:57

todthomson force-pushed the eft/tod/fix/FD-492-OCI-compliance-etc branch from 9e376e1 to 9cae415 Compare June 24, 2026 10:57

bec-callow-oct approved these changes Jun 25, 2026

View reviewed changes

LukeButters approved these changes Jun 25, 2026

View reviewed changes

matt-richardson reviewed Jun 25, 2026

View reviewed changes

APErebus reviewed Jun 25, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(docker): make Tentacle images OCI-compliant (FD-492)#1259

fix(docker): make Tentacle images OCI-compliant (FD-492)#1259
todthomson wants to merge 6 commits into
mainfrom
eft/tod/fix/FD-492-OCI-compliance-etc

todthomson commented Jun 18, 2026 •

edited

Loading

Uh oh!

CLAassistant commented Jun 24, 2026

Uh oh!

todthomson commented Jun 24, 2026

Uh oh!

LukeButters left a comment

Uh oh!

matt-richardson Jun 25, 2026

Uh oh!

APErebus Jun 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Uh oh!

Conversation

todthomson commented Jun 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Background

What Octopus Server actually does (verified in the OctopusDeploy repo)

Why this PR uses buildx flags instead of copying Server

Manifest evidence (verified against the published registry)

Changes

⚠️ Follow-up required (out of scope for this repo)

Testing notes

Uh oh!

CLAassistant commented Jun 24, 2026

Uh oh!

todthomson commented Jun 24, 2026

Code review

What it does

Correctness — solid (compile-verified, behaviour verified locally)

Issues / notes

Conventions / tests

Verdict

Uh oh!

LukeButters left a comment

Choose a reason for hiding this comment

Uh oh!

matt-richardson Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

APErebus Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

todthomson commented Jun 18, 2026 •

edited

Loading