feat(plane-enterprise): hooks for shared secrets, storage and the read replica (3.5.0) - #284
Merged
pratapalakshmi merged 4 commits intoAug 14, 2026
Conversation
… connectors Both of these could previously only be externalized through a whole-Secret hook, which made the one thing people actually need impossible. AI/LLM provider keys lived in the Plane AI Secret alongside PLANE_PI_DATABASE_URL, FOLLOWER_POSTGRES_URI, AMQP_URL, CELERY_BROKER_URL and the signing keys. Adopting pi_api_env_existingSecret therefore meant owning all of those too — and they are per-environment, while a provider account is not. So "one provider key shared by every environment" had no expression. ai_providers_existingSecret carries nothing environment-specific, so one Secret can be mirrored into as many namespaces as share the account, while a customer can point the same values key at a per-environment Secret. The chart only takes a name; the granularity is the secret store's business. The suppression had to cover the empty-string branches, not just the populated ones. The chart rendered COHERE_API_KEY: "" and CUSTOM_LLM_API_KEY: "" whenever those providers were disabled, and envFrom resolves later-source-wins — so a shared Secret mounted earlier would have been silently overwritten with empty strings. There is now an assertion for exactly this, and it fails on the pre-change render. Silo connector credentials had the same shape of problem: the silo Secret also carries DATABASE_URL, AMQP_URL and REDIS_URL. silo_connectors_existingSecret takes the whole connector surface, client ids and app names included, so it has a single owner rather than a half-external state. It is mounted on all nine workloads that mount silo-secrets, not just silo: the Django auth adapter reads GITHUB_CLIENT_ID and GITHUB_CLIENT_SECRET from that Secret on the api family, so scoping it to silo alone would have dropped those variables there. There is an assertion for that too. Also adds hack/resolve-env.py and hack/assert-secrets.py. The first resolves what each container's environment actually becomes — envFrom in list order, then explicit env on top — because a chart diff cannot show whether a key that moved between Secrets is still reachable. The second carries the assertions this work turns on, including a no-DSN check: a connection string with an embedded password is a rotation dead-end, since nothing can track a rotated password baked into a string. Verified byte-identical resolved environments across all workloads when no new hook is set. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…nset AWS region
services.postgres.read_replica.remote_url was the last password-bearing connection
string in this chart with no discrete-parts alternative, so a managed rotation could
never reach the replica. external_secrets.database.readReplica now supplies
POSTGRES_READ_REPLICA_{USER,PASSWORD} from a mirrored Secret with the endpoint in
values, and DATABASE_READ_REPLICA_URL is blanked in that mode so a stale composed URL
cannot shadow the rotated credential.
No application change was needed: Django takes the replica's parts straight into its
DATABASES dict, so nothing composes a URL at all. secretName defaults to the primary's,
since a replica normally accepts the same credential.
The fail guard that demanded remote_url now accepts the mirror as a source too, and
says what the alternatives are rather than naming only one of them — otherwise
enabling a replica keylessly was impossible by construction.
AWS_REGION and AWS_S3_ENDPOINT_URL are now omitted when unset rather than rendered
empty. The keys next to them were already treated this way because an empty
AWS_ACCESS_KEY_ID derails boto3's credential chain; the region is subtler. When a pod
identity is attached and OpenSearch has no basic auth, the API signs OpenSearch
requests with SigV4 and reads the region as `os.environ.get("AWS_REGION") or default` —
an empty string is present, so the default never applies, and every signed request goes
out with an empty region and is rejected. Absent is the only safe encoding of "unset".
The companion app change makes the same value safe on older charts.
Verified with hack/: the resolved environment of every container is byte-identical when
no new hook is set; a fully externalized render (database, rabbitmq, redis, replica,
external object storage) now contains zero connection strings with embedded passwords;
and in the keyless case AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_REGION are
absent from every container rather than empty.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…y marker (3.5.0) external_secrets.storage covers the object-storage credentials that a pod identity cannot replace: an S3-compatible backend with no workload identity, or GCS with a service-account key. That key is the highest-value blob in a GCS deployment — the whole service-account JSON, private key included — and it had no hook at all, so the only way to keep it out of values.yaml was to take over the entire doc-store Secret and own USE_MINIO, the bucket name and the size limit along with it. The mirror is wired through the three composite credential helpers rather than per-workload, since those already reach every workload that mounts the doc-store Secret. It is inert while the bundled MinIO is deployed, which supplies its own credentials. serviceAccount.cloudIdentity declares that an identity is attached out of band. It changes no rendered output, and it cannot be inferred: an EKS Pod Identity association is an API object keyed on cluster, namespace and service account, and nothing about it shows up in the pod spec. What it buys is three warnings for the interactions an attached identity creates — a static env.aws_access_key silently beating the identity in the SDK chain, the same for Plane AI's Bedrock keys, and the one that is genuinely surprising: a remote OpenSearch with no credentials will be authenticated with SigV4, because both the API and Plane AI read "AWS credentials are present" as "sign requests" — and an identity puts credentials in every container on the ServiceAccount. Correct for Amazon OpenSearch Service, wrong for any other. The warning names both fixes. Verified with hack/: the resolved environment of every container is byte-identical when no new hook is set; ten configurations render valid YAML including all hooks at once and the legacy groups; and the acceptance case — every hook in use, requireExplicitSecrets on, all signing keys blanked — renders with no credential pattern, no connection string carrying a password, and AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY / AWS_REGION absent from every container. What remains in the chart's own Secrets is empty URL placeholders and non-secret configuration. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
2 tasks
…ons.yml
None of the credential mirrors, shared key groups or ServiceAccount fields were
reachable from the Rancher UI, which is how most operators configure this chart — a
hook nobody can find is a hook nobody uses.
Three new groups. Credential Mirrors covers external_secrets.{database,rabbitmq,redis,
storage,opensearch}, kept apart from the existing whole-Secret replacements because
they answer a different question: a mirror is read as parts and never composed into a
DSN, so a managed rotation lands with nothing to recompute. Shared Secrets covers the
four key groups. Service Account covers create/name/annotations/cloudIdentity plus
env.requireExplicitSecrets.
The descriptions carry the operational traps rather than restating the field names:
Amazon MQ needs rabbitmq_ssl and port 5671 because the parts path has no URL scheme to
carry TLS; ElastiCache puts its auth token in the password key; the signing keys must
never be shared or rotated; EKS Pod Identity needs no annotation because it binds by
ServiceAccount name.
pratapalakshmi
merged commit Aug 14, 2026
bd99749
into
feat/silo-live-pi-external-secrets
1 check passed
pratapalakshmi
added a commit
that referenced
this pull request
Aug 14, 2026
…e and Plane AI (#279) * feat(plane-enterprise): extend discrete credential parts to silo, live and Plane AI Phase 1 wired the Django services to consume a rotating credential as discrete parts. Silo, live and Plane AI still needed a composed DSN, so a deployment backed by a managed rotation secret had to maintain hand-built URLs for them alongside the secret itself. With the companion app release those services read parts too, and this wires them up. Also fixes the shared-key defects a cross-service audit turned up. Per-workload least privilege. plane.infraCredsEnv is split into postgresCredsEnv / rabbitmqCredsEnv / redisCredsEnv (+ the existing opensearchCredsEnv) and recomposed from them, so the eleven existing callers render byte-identically. Live then takes Redis and nothing else; silo takes Postgres, RabbitMQ and Redis but not OpenSearch. Plane AI gets its own plane.piInfraCredsEnv because it reads different variable names: PLANE_PI_POSTGRES_* for its own database and FOLLOWER_POSTGRES_* for its read path into the main Plane database, both from the same external_secrets.database Secret — one managed instance hosting two databases is the shape this chart provisions. It deliberately receives no RABBITMQ_*: Plane AI resolves an AMQP broker ahead of a Redis one, and Redis is its broker here, so emitting RabbitMQ parts would silently move its queue. An explicit env.pi_envs.follower_postgres_uri still wins, as the escape hatch for a follower with genuinely separate credentials. Composed DSNs are blanked in silo/live/pi-api Secrets whenever the matching external_secrets block is set, for the same reason as the API: each app prefers a URL when one is present, so a stale composed URL would shadow the rotated password. Audit fixes: RUNNER_HMAC_SECRET_KEY was delivered only to the runner, which has no code that reads it, while the API — the only service that verifies the signature — never received it, so runner HMAC auth could not succeed. It is now mounted on the seven api-family workloads that already mount silo-secrets, for the same reason: the verifying side lives there. It was also the one shared-key render bypassing plane.secretValue, so requireExplicitSecrets did not cover it, and runner.deployment was the only workload missing appKeysSecretRef. Both fixed. Making the runner actually sign its requests is a product decision and stays out of scope. AES_SALT was advertised as an app-keys member but had no values key and was never rendered — api, silo and pi worked only because all three share the same hardcoded default. env.silo_envs.aes_salt now renders into all three Secrets, and only when non-empty, so that default still applies to existing instances. app_keys_existingSecret combined with a legacy *_existingSecret carrying the same key diverges per service: app-keys is mounted first, so a duplicate wins on the workloads mounting that group and loses elsewhere, leaving two services disagreeing on a key that has to match. The chart cannot see inside an operator-supplied Secret, so NOTES.txt now warns when both are set and lists the keys that must not be duplicated. The public-default warning covered four of the six app keys; SILO_HMAC_SECRET_KEY — the API's inbound-auth key for the whole silo surface — and CURSOR_WEBHOOK_SECRET were missing. Both added. values.yaml now documents which keys the app-keys Secret must contain, SECRET_KEY foremost: omitting it gives each API pod a different random signing key, silently. Version gating. The "these services read connection URLs" and "redis needs parts support" warnings are gated on planeVersion (v3.2.0 and v3.1.0 respectively) rather than always firing. planeVersion is operator-supplied and need not be semver, so an unparsable value warns instead of failing the render — semverCompare would error. Verified additive over Phase 1: 28 containers, no env var lost anywhere; the only change on default values is RUNNER_HMAC_SECRET_KEY reaching the api family, which is the fix. helm lint clean and ten configurations render valid YAML, including runner with and without app-keys, the legacy groups alone, and the four version gates. Requires the companion app release for silo/live/pi parts support; the chart warns when planeVersion predates it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(plane-enterprise): hooks for shared secrets, storage and the read replica (3.5.0) (#284) * feat(plane-enterprise): dedicated hooks for AI provider keys and silo connectors Both of these could previously only be externalized through a whole-Secret hook, which made the one thing people actually need impossible. AI/LLM provider keys lived in the Plane AI Secret alongside PLANE_PI_DATABASE_URL, FOLLOWER_POSTGRES_URI, AMQP_URL, CELERY_BROKER_URL and the signing keys. Adopting pi_api_env_existingSecret therefore meant owning all of those too — and they are per-environment, while a provider account is not. So "one provider key shared by every environment" had no expression. ai_providers_existingSecret carries nothing environment-specific, so one Secret can be mirrored into as many namespaces as share the account, while a customer can point the same values key at a per-environment Secret. The chart only takes a name; the granularity is the secret store's business. The suppression had to cover the empty-string branches, not just the populated ones. The chart rendered COHERE_API_KEY: "" and CUSTOM_LLM_API_KEY: "" whenever those providers were disabled, and envFrom resolves later-source-wins — so a shared Secret mounted earlier would have been silently overwritten with empty strings. There is now an assertion for exactly this, and it fails on the pre-change render. Silo connector credentials had the same shape of problem: the silo Secret also carries DATABASE_URL, AMQP_URL and REDIS_URL. silo_connectors_existingSecret takes the whole connector surface, client ids and app names included, so it has a single owner rather than a half-external state. It is mounted on all nine workloads that mount silo-secrets, not just silo: the Django auth adapter reads GITHUB_CLIENT_ID and GITHUB_CLIENT_SECRET from that Secret on the api family, so scoping it to silo alone would have dropped those variables there. There is an assertion for that too. Also adds hack/resolve-env.py and hack/assert-secrets.py. The first resolves what each container's environment actually becomes — envFrom in list order, then explicit env on top — because a chart diff cannot show whether a key that moved between Secrets is still reachable. The second carries the assertions this work turns on, including a no-DSN check: a connection string with an embedded password is a rotation dead-end, since nothing can track a rotated password baked into a string. Verified byte-identical resolved environments across all workloads when no new hook is set. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(plane-enterprise): read-replica credential mirror, and omit an unset AWS region services.postgres.read_replica.remote_url was the last password-bearing connection string in this chart with no discrete-parts alternative, so a managed rotation could never reach the replica. external_secrets.database.readReplica now supplies POSTGRES_READ_REPLICA_{USER,PASSWORD} from a mirrored Secret with the endpoint in values, and DATABASE_READ_REPLICA_URL is blanked in that mode so a stale composed URL cannot shadow the rotated credential. No application change was needed: Django takes the replica's parts straight into its DATABASES dict, so nothing composes a URL at all. secretName defaults to the primary's, since a replica normally accepts the same credential. The fail guard that demanded remote_url now accepts the mirror as a source too, and says what the alternatives are rather than naming only one of them — otherwise enabling a replica keylessly was impossible by construction. AWS_REGION and AWS_S3_ENDPOINT_URL are now omitted when unset rather than rendered empty. The keys next to them were already treated this way because an empty AWS_ACCESS_KEY_ID derails boto3's credential chain; the region is subtler. When a pod identity is attached and OpenSearch has no basic auth, the API signs OpenSearch requests with SigV4 and reads the region as `os.environ.get("AWS_REGION") or default` — an empty string is present, so the default never applies, and every signed request goes out with an empty region and is rejected. Absent is the only safe encoding of "unset". The companion app change makes the same value safe on older charts. Verified with hack/: the resolved environment of every container is byte-identical when no new hook is set; a fully externalized render (database, rabbitmq, redis, replica, external object storage) now contains zero connection strings with embedded passwords; and in the keyless case AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_REGION are absent from every container rather than empty. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(plane-enterprise): storage credential mirror and a cloud-identity marker (3.5.0) external_secrets.storage covers the object-storage credentials that a pod identity cannot replace: an S3-compatible backend with no workload identity, or GCS with a service-account key. That key is the highest-value blob in a GCS deployment — the whole service-account JSON, private key included — and it had no hook at all, so the only way to keep it out of values.yaml was to take over the entire doc-store Secret and own USE_MINIO, the bucket name and the size limit along with it. The mirror is wired through the three composite credential helpers rather than per-workload, since those already reach every workload that mounts the doc-store Secret. It is inert while the bundled MinIO is deployed, which supplies its own credentials. serviceAccount.cloudIdentity declares that an identity is attached out of band. It changes no rendered output, and it cannot be inferred: an EKS Pod Identity association is an API object keyed on cluster, namespace and service account, and nothing about it shows up in the pod spec. What it buys is three warnings for the interactions an attached identity creates — a static env.aws_access_key silently beating the identity in the SDK chain, the same for Plane AI's Bedrock keys, and the one that is genuinely surprising: a remote OpenSearch with no credentials will be authenticated with SigV4, because both the API and Plane AI read "AWS credentials are present" as "sign requests" — and an identity puts credentials in every container on the ServiceAccount. Correct for Amazon OpenSearch Service, wrong for any other. The warning names both fixes. Verified with hack/: the resolved environment of every container is byte-identical when no new hook is set; ten configurations render valid YAML including all hooks at once and the legacy groups; and the acceptance case — every hook in use, requireExplicitSecrets on, all signing keys blanked — renders with no credential pattern, no connection string carrying a password, and AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY / AWS_REGION absent from every container. What remains in the chart's own Secrets is empty URL placeholders and non-secret configuration. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(plane-enterprise): expose the 3.3.0–3.5.0 secret hooks in questions.yml None of the credential mirrors, shared key groups or ServiceAccount fields were reachable from the Rancher UI, which is how most operators configure this chart — a hook nobody can find is a hook nobody uses. Three new groups. Credential Mirrors covers external_secrets.{database,rabbitmq,redis, storage,opensearch}, kept apart from the existing whole-Secret replacements because they answer a different question: a mirror is read as parts and never composed into a DSN, so a managed rotation lands with nothing to recompute. Shared Secrets covers the four key groups. Service Account covers create/name/annotations/cloudIdentity plus env.requireExplicitSecrets. The descriptions carry the operational traps rather than restating the field names: Amazon MQ needs rabbitmq_ssl and port 5671 because the parts path has no URL scheme to carry TLS; ElastiCache puts its auth token in the password key; the signing keys must never be shared or rotated; EKS Pod Identity needs no annotation because it binds by ServiceAccount name. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
Author
|
Absorbed into #278, which now carries the whole plane-enterprise side as one PR. Verified rather than assumed: the Closing so nobody reviews the same work twice. plane-ce stays separate in #285 — it is a different chart and touches no file in this one. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Surveying the real deployments turned up ~95 cleartext secrets in committed values, and the charts were the reason it could not be fixed at the deployment layer. Three gaps mattered most, and this closes them.
AI/LLM provider keys — the headline gap
They lived in the Plane AI Secret next to
PLANE_PI_DATABASE_URL,FOLLOWER_POSTGRES_URI,AMQP_URL,CELERY_BROKER_URLand the signing keys. Sopi_api_env_existingSecretwas the only route, and adopting it meant owning all of those too — which are per-environment, while a provider account is not. "One provider key shared by every environment" had no expression at all, and in practice the same Anthropic/OpenAI/Groq/Cohere keys are duplicated across deployments.ai_providers_existingSecretcarries nothing environment-specific:OPENAI_API_KEY,CLAUDE_API_KEY,GROQ_API_KEY,COHERE_API_KEY,CUSTOM_LLM_API_KEY,BR_AWS_*, andAI_OPENAI_API_KEY— the last of which the chart has no values key for at all, so this is its only route. Mounted on the four Plane AI workloads and on live.The suppression had to cover the empty-string branches. The chart rendered
COHERE_API_KEY: ""andCUSTOM_LLM_API_KEY: ""whenever those providers were disabled, andenvFromresolves later-source-wins — so a shared Secret mounted earlier would have been silently overwritten with empty strings. There is an assertion for exactly this, and it fails on the pre-change render.Silo connector credentials
Same shape: the silo Secret also carries
DATABASE_URL/AMQP_URL/REDIS_URL.silo_connectors_existingSecrettakes the whole connector surface, client ids and app names included, so it has one owner rather than a half-external state.Mounted on all nine workloads that mount silo-secrets, not just silo — the Django auth adapter reads
GITHUB_CLIENT_ID/_SECRETfrom that Secret on the api family, so scoping it to silo would have dropped those variables there. Asserted.The read replica — the last password-bearing DSN
services.postgres.read_replica.remote_urlembeds the password in a string, so a managed rotation could never reach it.external_secrets.database.readReplicasuppliesPOSTGRES_READ_REPLICA_{USER,PASSWORD}from a mirrored Secret with the endpoint in values, andDATABASE_READ_REPLICA_URLis blanked in that mode so a stale URL cannot shadow the rotated credential.No application change was needed — Django takes the replica's parts straight into its
DATABASESdict, so nothing composes a URL.secretNamedefaults to the primary's, since a replica normally accepts the same credential. Thefailguard that demandedremote_urlnow accepts the mirror too, and names both options.Object storage and cloud identity
external_secrets.storagecovers what a pod identity cannot: an S3-compatible backend with no workload identity, or GCS with a service-account key — the whole JSON including the private key, previously unhookable without taking over the entire doc-store Secret.AWS_REGIONandAWS_S3_ENDPOINT_URLare now omitted when unset rather than rendered empty. The keys beside them were already handled this way because an emptyAWS_ACCESS_KEY_IDderails boto3's chain; the region is subtler — see the companion app PR.serviceAccount.cloudIdentitydeclares an out-of-band identity. It changes no output and cannot be inferred (a Pod Identity association is an EKS API object keyed on cluster+namespace+SA). It buys three warnings, the important one being that a remote OpenSearch with no credentials will be authenticated with SigV4 once an identity is attached.Type of Change
Test Scenarios
Adds
hack/resolve-env.pyandhack/assert-secrets.py. The first resolves what each container's environment actually becomes —envFromin list order, then explicitenvon top, for containers and initContainers — because a chart diff cannot tell you whether a key that moved between Secrets is still reachable. The second carries the assertions this work turns on.ai_providers_existingSecretset, no rendered Secret contains any AI key including empty-valued. Control case confirms it fails otherwise.secretRefprecedes the chart's own Plane AI Secret.GITHUB_CLIENT_IDstill resolves on the api container under the connectors hook.(postgresql|amqp|redis)://user:pass@. This is the assertion that proves rotation is possible, and it is what would have caught the read-replica gap.AWS_ACCESS_KEY_ID,AWS_SECRET_ACCESS_KEY,AWS_REGIONabsent from every container — not empty.requireExplicitSecrets: trueand all signing keys blanked renders with no credential pattern and no DSN. What remains in the chart's Secrets is empty URL placeholders and non-secret config.helm lintclean; ten configurations render valid YAML including all hooks at once and the legacy groups.Not covered: no live cluster apply. Wants a staging run with ESO + Reloader.
References
Requires makeplane/plane-ee#8968 for
OPENSEARCH_AUTH_MODE(referenced by the SigV4 warning). Follow-ups: the same contract for plane-ce, plane-cloud and the private charts;questions.ymlstill does not expose any of the 3.3.0–3.5.0 hooks.🤖 Generated with Claude Code