fix(security): redact all credential-bearing URL components in status dbServer - #846
Open
kparkinson-ld wants to merge 4 commits into
Open
kparkinson-ld wants to merge 4 commits into
kparkinson-ld wants to merge 4 commits into
Conversation
…rver (SEC-9489, SEC-9498)
Contributor
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
…onents
Follow-ups to the RedactURL rewrite in this PR.
Opaque URLs were not redacted at all. RedactURL only touched User, RawQuery and
Fragment, but a non-hierarchical "scheme:rest" URL keeps everything -- userinfo
included -- in URL.Opaque. Since ct.OptURLAbsolute accepts these (url.IsAbs is
true for an opaque URL), a REDIS_URL missing its "//" published the credential
verbatim on the unauthenticated status resource:
redis:very-secret-password@redishost:6379
-> "dbServer":"redis:very-secret-password@redishost:6379"
Note the redaction stops at the last "@" rather than replacing the whole opaque
body: url.Parse also reports a bare "host:port" as scheme plus opaque, so
blanking it would turn "consul.example.com:8500" into "consul.example.com:xxxxx"
and destroy the port.
Two cases were over-redacted, both claiming a credential had been removed when
none was present:
- An empty userinfo. url.Parse returns a non-nil but empty Userinfo for
"scheme://@host", so "redis://@host:6379" became "redis://xxxxx@host:6379".
- ForceQuery. A trailing "?" means RawQuery is empty by definition, so there is
nothing to redact; "https://host/path?" became "https://host/path?xxxxx",
longer than the input. This was already inconsistent with the empty-fragment
case immediately below it, which was handled correctly.
Consul's dbServer was the one DataStoreEnvironmentInfo.DBServer assignment still
bypassing the redactor, despite the field's new doc comment requiring it. The
store still receives the real address; only the displayed copy is redacted.
The doc comment and docs/endpoints.md claimed that "any credentials in the URL"
are redacted. Both now say that components are replaced only when present, and
that the path is deliberately preserved -- so a credential embedded in a path
segment is not covered.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Member
|
@kparkinson-ld I've made some changes to this PR. LMK if you think these modifications are fine. |
Contributor
|
@keelerm84 reviewed
Verified locally on the merged branch: |
Contributor
Author
Yep this looks good 👍 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Centralizes URL redaction so no datastore credential can reach the unauthenticated
GET /statusresponse or the logs.Closes SEC-9489 and SEC-9498.
util.RedactURLnow masks the entire userinfo section (not just the password), so a secret in the username position —redis://secret@host, which redigo authenticates with — is no longer disclosed.?password=/?token=style credentials.ConfigureDataStorenow runs its endpoint URL through the same routine; previously it was assigned todbSerververbatim.dbServerremains useful for diagnostics.Requirements
Implementation details
Root cause.
util.RedactURLonly replaced the userinfo password component.redis://secret@hostyieldsUser != nilbutPassword()returns("", false), so the string passed through byte-for-byte intoDataStoreEnvironmentInfo.DBServer→environments.<env>.dataStoreStatus.dbServeron/status, which has no auth middleware. Query-string credentials were never examined, and a parse failure returned the raw input (fail-open). Separately, the DynamoDB branch never called the redaction routine at all, so a credential-bearing custom endpoint URL (basic-auth proxy, token query param) was served verbatim — including passwords in the standard position that the Redis path would have masked.Behavior change.
dbServerand the corresponding info log messages now render as, e.g.:redis://username:pw@host:3000→redis://xxxxx@host:3000(previouslyredis://username:xxxxx@host:3000, so the username is no longer shown)redis://token@host:3000→redis://xxxxx@host:3000(previously unredacted)https://u:p@ddb-gw.internal:8000/path?token=t#f→https://xxxxx@ddb-gw.internal:8000/path?xxxxx#xxxxxScope. Only the URL redaction path changed. Consul's
dbServer(a bare host, no URL), the proxy-server log line inhttpconfig(uses the stdlibURL.Redacted()), and every other status field are untouched.Testing.
internal/utilcovers username-position secrets, query/fragment masking, the empty-string case, and fail-closed parse failure;internal/sdkscovers the Redis and DynamoDBdbServervalues plus the log output.make lintclean,make test(including-race) passes.Link to Devin session: https://app.devin.ai/sessions/404615ce21fa469ab7e80a26bd488f6e
Open in Devin Desktop: https://app.devin.ai/desktop/session/404615ce21fa469ab7e80a26bd488f6e?variant=devin
Requested by: @kparkinson-ld
Note
Overview
Hardens credential handling for the unauthenticated
GET /statusdataStoreStatus.dbServerfield and matching datastore startup logs.util.RedactURLnow masks the entire userinfo (not only the password), plus query and fragment, usingxxxxx, while keeping scheme/host/port/path for diagnostics. Unparseable URLs fail closed toxxxxxinstead of echoing the raw string; opaque andhost:port-style inputs get special handling so Consul-style hosts are not broken.Datastore wiring: Consul host strings and DynamoDB endpoint URLs now go through the same redaction as Redis when populating
DataStoreEnvironmentInfo.DBServerand info logs; actual SDK connection config still uses the real values.Docs describe the redaction rules and the limitation that secrets in URL paths are not masked.
Reviewed by Cursor Bugbot for commit a9d990a. Bugbot is set up for automated code reviews on this repo. Configure here.