From df8165ec4154b08a2823ab0b85c731b5bda53797 Mon Sep 17 00:00:00 2001 From: Nikolai Emil Damm Date: Thu, 11 Jun 2026 00:46:35 +0200 Subject: [PATCH] fix(cluster-policies): mutate pod security contexts on CREATE only The add-security-context ClusterPolicy matched Pods without an operations scope, so Kyverno also applied the securityContext mutation on every pod UPDATE. Pod spec is immutable: for any pod created while the policy/webhook was not yet active (exactly what fresh-cluster bring-up ordering produces), every later update gets the mutation bolted on and the apiserver rejects the whole request with HTTP 422 'pod updates may not change fields other than image...'. That bricked OpenBao's Kubernetes service registration in CI: the label-state updates (openbao-active/sealed/initialized) 422'd forever, the openbao-active Service never gained endpoints, the entire vault seeding chain timed out, and every system test since the 2026-06-10 active-service cutover failed. Probe evidence in #1990's run: the 422 diff shows the webhook's own securityContext injection, not the label patch. Prod was unaffected only because its pods happened to be recreated while the policy was live (mutation already in the spec -> no-op on update). Scope both rules to operations: [CREATE]. Pods created before the policy stay unmutated until their next natural recreation, which is strictly better than being permanently un-updatable. Co-Authored-By: Claude Fable 5 --- .../best-practices/add-security-context.yaml | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/k8s/bases/infrastructure/cluster-policies/best-practices/add-security-context.yaml b/k8s/bases/infrastructure/cluster-policies/best-practices/add-security-context.yaml index 0d8f3e91d..30c58d3fc 100644 --- a/k8s/bases/infrastructure/cluster-policies/best-practices/add-security-context.yaml +++ b/k8s/bases/infrastructure/cluster-policies/best-practices/add-security-context.yaml @@ -31,6 +31,16 @@ spec: - resources: kinds: - Pod + # CREATE only. Without this, Kyverno also mutates pod UPDATEs; + # pod spec is immutable, so for any pod created while the + # policy/webhook wasn't active yet (fresh-cluster bring-up + # ordering) EVERY later update — including OpenBao's + # service-registration label patches — is rejected with + # HTTP 422 "pod updates may not change fields other than + # image...". That froze openbao-active at zero endpoints and + # broke every system test from 2026-06-10 (#1990 probe output). + operations: + - CREATE exclude: any: - resources: @@ -59,6 +69,9 @@ spec: - resources: kinds: - Pod + # CREATE only — same immutability rationale as above. + operations: + - CREATE exclude: any: - resources: