Skip to content

fix(security): redact all credential-bearing URL components in status dbServer - #846

Open
kparkinson-ld wants to merge 4 commits into
v8from
devin/1788215050-redact-status-urls
Open

kparkinson-ld wants to merge 4 commits into
v8from
devin/1788215050-redact-status-urls

Conversation

@kparkinson-ld

@kparkinson-ld kparkinson-ld commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Centralizes URL redaction so no datastore credential can reach the unauthenticated GET /status response or the logs.

Closes SEC-9489 and SEC-9498.

  • util.RedactURL now masks the entire userinfo section (not just the password), so a secret in the username position — redis://secret@host, which redigo authenticates with — is no longer disclosed.
  • Query and fragment components are masked too, covering ?password=/?token= style credentials.
  • Unparseable input now fails closed (returns the placeholder) instead of passing the raw string through.
  • The DynamoDB branch of ConfigureDataStore now runs its endpoint URL through the same routine; previously it was assigned to dbServer verbatim.
  • Scheme, host, port, and path are preserved, so dbServer remains useful for diagnostics.

Requirements

  • I have added test coverage for new or changed functionality
  • I have followed the repository's pull request submission guidelines
  • I have validated my changes against all supported platform versions
Implementation details

Root cause. util.RedactURL only replaced the userinfo password component. redis://secret@host yields User != nil but Password() returns ("", false), so the string passed through byte-for-byte into DataStoreEnvironmentInfo.DBServerenvironments.<env>.dataStoreStatus.dbServer on /status, which has no auth middleware. Query-string credentials were never examined, and a parse failure returned the raw input (fail-open). Separately, the DynamoDB branch never called the redaction routine at all, so a credential-bearing custom endpoint URL (basic-auth proxy, token query param) was served verbatim — including passwords in the standard position that the Redis path would have masked.

Behavior change. dbServer and the corresponding info log messages now render as, e.g.:

  • redis://username:pw@host:3000redis://xxxxx@host:3000 (previously redis://username:xxxxx@host:3000, so the username is no longer shown)
  • redis://token@host:3000redis://xxxxx@host:3000 (previously unredacted)
  • https://u:p@ddb-gw.internal:8000/path?token=t#fhttps://xxxxx@ddb-gw.internal:8000/path?xxxxx#xxxxx

Scope. Only the URL redaction path changed. Consul's dbServer (a bare host, no URL), the proxy-server log line in httpconfig (uses the stdlib URL.Redacted()), and every other status field are untouched.

Testing. internal/util covers username-position secrets, query/fragment masking, the empty-string case, and fail-closed parse failure; internal/sdks covers the Redis and DynamoDB dbServer values plus the log output. make lint clean, make test (including -race) passes.

Link to Devin session: https://app.devin.ai/sessions/404615ce21fa469ab7e80a26bd488f6e
Open in Devin Desktop: https://app.devin.ai/desktop/session/404615ce21fa469ab7e80a26bd488f6e?variant=devin
Requested by: @kparkinson-ld


Note

Overview
Hardens credential handling for the unauthenticated GET /status dataStoreStatus.dbServer field and matching datastore startup logs.

util.RedactURL now masks the entire userinfo (not only the password), plus query and fragment, using xxxxx, while keeping scheme/host/port/path for diagnostics. Unparseable URLs fail closed to xxxxx instead of echoing the raw string; opaque and host:port-style inputs get special handling so Consul-style hosts are not broken.

Datastore wiring: Consul host strings and DynamoDB endpoint URLs now go through the same redaction as Redis when populating DataStoreEnvironmentInfo.DBServer and info logs; actual SDK connection config still uses the real values.

Docs describe the redaction rules and the limitation that secrets in URL paths are not masked.

Reviewed by Cursor Bugbot for commit a9d990a. Bugbot is set up for automated code reviews on this repo. Configure here.

@devin-ai-integration

Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@devin-ai-integration devin-ai-integration Bot added the devin-pr Pull request created by Devin AI label Aug 31, 2026
@devin-ai-integration
devin-ai-integration Bot requested a review from a team August 31, 2026 22:25
kparkinson-ld and others added 3 commits September 1, 2026 17:07
…onents

Follow-ups to the RedactURL rewrite in this PR.

Opaque URLs were not redacted at all. RedactURL only touched User, RawQuery and
Fragment, but a non-hierarchical "scheme:rest" URL keeps everything -- userinfo
included -- in URL.Opaque. Since ct.OptURLAbsolute accepts these (url.IsAbs is
true for an opaque URL), a REDIS_URL missing its "//" published the credential
verbatim on the unauthenticated status resource:

    redis:very-secret-password@redishost:6379
      -> "dbServer":"redis:very-secret-password@redishost:6379"

Note the redaction stops at the last "@" rather than replacing the whole opaque
body: url.Parse also reports a bare "host:port" as scheme plus opaque, so
blanking it would turn "consul.example.com:8500" into "consul.example.com:xxxxx"
and destroy the port.

Two cases were over-redacted, both claiming a credential had been removed when
none was present:

- An empty userinfo. url.Parse returns a non-nil but empty Userinfo for
  "scheme://@host", so "redis://@host:6379" became "redis://xxxxx@host:6379".
- ForceQuery. A trailing "?" means RawQuery is empty by definition, so there is
  nothing to redact; "https://host/path?" became "https://host/path?xxxxx",
  longer than the input. This was already inconsistent with the empty-fragment
  case immediately below it, which was handled correctly.

Consul's dbServer was the one DataStoreEnvironmentInfo.DBServer assignment still
bypassing the redactor, despite the field's new doc comment requiring it. The
store still receives the real address; only the displayed copy is redacted.

The doc comment and docs/endpoints.md claimed that "any credentials in the URL"
are redacted. Both now say that components are replaced only when present, and
that the path is deliberately preserved -- so a credential embedded in a path
segment is not covered.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@keelerm84

Copy link
Copy Markdown
Member

@kparkinson-ld I've made some changes to this PR. LMK if you think these modifications are fine.

@devin-ai-integration

Copy link
Copy Markdown
Contributor

@keelerm84 reviewed ed473b5 — all four changes look right to me, and two of them are real gaps I missed:

  • Opaque URLs: correct catch. redis:secret@host:6379 parses as scheme + Opaque, and ct.OptURLAbsolute accepts it, so it was published verbatim. Redacting up to the last @ rather than the whole opaque body is the right call given url.Parse also reports bare host:port that way — the consul.example.com:8500 and my-host assertions pin that down.
  • Consul dbServer: agreed, it was the one DBServer assignment still bypassing the redactor after I added the doc comment requiring it, and a plain host round-trips unchanged so there's no diagnostic loss.
  • Empty userinfo / ForceQuery: both were mine over-redacting and claiming a credential was removed where none existed. Dropping the ForceQuery branch also makes it consistent with the empty-fragment case right below.
  • Docs: the "any credentials in the URL" wording was overclaiming; calling out the preserved path as a documented limitation is more honest.

Verified locally on the merged branch: go test ./internal/util/... ./internal/sdks/... passes and make lint is clean. No further changes from me — good to go from my side.

@kparkinson-ld

Copy link
Copy Markdown
Contributor Author

@kparkinson-ld I've made some changes to this PR. LMK if you think these modifications are fine.

Yep this looks good 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

devin-pr Pull request created by Devin AI exempt

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants