Skip to content

fix: resolve rate-limit proxy IP collapse and JSON log injection#289

Open
luis5tb wants to merge 2 commits into
RHEcosystemAppEng:mainfrom
luis5tb:rate-limit-proxy-ip
Open

fix: resolve rate-limit proxy IP collapse and JSON log injection#289
luis5tb wants to merge 2 commits into
RHEcosystemAppEng:mainfrom
luis5tb:rate-limit-proxy-ip

Conversation

@luis5tb

@luis5tb luis5tb commented Jun 9, 2026

Copy link
Copy Markdown
Collaborator

Rate limiter: enable uvicorn proxy header trust (X-Forwarded-For) so each client gets its own bucket behind GCLB/OpenShift Routes instead of all tenants sharing a single proxy-IP bucket. Reorder Auth before RateLimit middleware on the agent service so tenant-based bucketing (order/user/client) is active — the multi-dimensional principal resolution was dead code when rate limiting ran before auth.

JSON logging (CWE-117): replace %-format JSON templates with pythonjsonlogger JsonFormatter so user-controlled content in log messages cannot break JSON structure or inject fake fields.

@luis5tb luis5tb force-pushed the rate-limit-proxy-ip branch 3 times, most recently from 07196d5 to de2da8a Compare June 9, 2026 06:41
@IlonaShishov

Copy link
Copy Markdown
Collaborator

Code Review — Issues

1. Security trade-off of reordering: unauthenticated floods no longer throttled at IP level before auth

The old comment explicitly noted: "runs before auth so unauthenticated floods are throttled at the IP level before any auth processing." By moving rate-limit after auth, every unauthenticated request now passes through the full JWT validation pipeline before being throttled. This means:

  • An attacker sending floods of invalid tokens will consume JWT validation resources (SSO introspection calls, crypto verification) before hitting rate limits
  • The proxy IP collapse was a real bug, but the original ordering was a deliberate defense-in-depth choice

Consider whether a two-layer approach is better: a lightweight IP-based pre-auth throttle (e.g., connection-level via Cloud Armor / nginx) plus the current tenant-based post-auth throttle. The CLAUDE.md notes Cloud Armor is already in the Cloud Run deployment — does it cover this gap?

2. forwarded_allow_ips: str = "*" trusts all proxies by default

The * default means any X-Forwarded-For header is trusted, even from direct clients not behind a load balancer. This is fine when ingress is restricted to the load balancer (Cloud Run, OpenShift Routes), but in Podman/dev mode where there's direct port binding, an attacker can spoof X-Forwarded-For to evade IP-based rate limiting.

The field description says "safe when ingress is restricted to the load balancer" — consider defaulting to "" (disabled) or the GCLB IP ranges, and only enabling in deployment configs.

3. Marketplace handler uses os.getenv instead of Pydantic settings

The agent service added proxy_headers and forwarded_allow_ips to the Settings model with validation, but the marketplace handler reads them via raw os.getenv:

proxy_headers=os.getenv("PROXY_HEADERS", "true").lower() == "true",
forwarded_allow_ips=os.getenv("FORWARDED_ALLOW_IPS", "*"),

This bypasses the Pydantic validation. It's consistent with how the handler already reads LOG_LEVEL/LOG_FORMAT (also raw os.getenv), so it follows precedent, but it's a divergence worth noting.

4. Duplicate if log_format == "json" check in marketplace handler

if log_format == "json":
    from pythonjsonlogger.json import JsonFormatter
    handler.setFormatter(...)

if log_format == "json":           # <-- second check, same condition
    logging.basicConfig(level=log_level, handlers=[handler])
else:
    logging.basicConfig(...)

These two blocks should be a single if/else. The same pattern appears in the agent's main.py.

5. Stderr → stdout change in marketplace handler

The diff adds import sys and StreamHandler(sys.stdout). The old code used logging.basicConfig() which defaults to stderr. This silently changes marketplace handler logs from stderr to stdout. If intentional, worth a note in the PR description.

luis5tb added a commit to luis5tb/google-lightspeed-agent that referenced this pull request Jun 9, 2026
- Add Cloud Armor IP-based throttle rule at priority 100 for pre-auth
  flood protection and document middleware order trade-off in app.py
- Change forwarded_allow_ips default to empty (safe-by-default for dev),
  add explicit FORWARDED_ALLOW_IPS=* in Cloud Run and OpenShift configs
- Fix duplicate if log_format check in both main.py and marketplace
  __main__.py by consolidating into single if/else blocks
- Add stdout comment on marketplace handler StreamHandler

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
luis5tb added a commit to luis5tb/google-lightspeed-agent that referenced this pull request Jun 10, 2026
- Add Cloud Armor IP-based throttle rule at priority 100 for pre-auth
  flood protection and document middleware order trade-off in app.py
- Change forwarded_allow_ips default to empty (safe-by-default for dev),
  add explicit FORWARDED_ALLOW_IPS=* in Cloud Run and OpenShift configs
- Fix duplicate if log_format check in both main.py and marketplace
  __main__.py by consolidating into single if/else blocks
- Add stdout comment on marketplace handler StreamHandler

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@luis5tb luis5tb force-pushed the rate-limit-proxy-ip branch from ec7e8fc to 9d6296b Compare June 10, 2026 05:35
luis5tb added a commit to luis5tb/google-lightspeed-agent that referenced this pull request Jun 11, 2026
- Add Cloud Armor IP-based throttle rule at priority 100 for pre-auth
  flood protection and document middleware order trade-off in app.py
- Change forwarded_allow_ips default to empty (safe-by-default for dev),
  add explicit FORWARDED_ALLOW_IPS=* in Cloud Run and OpenShift configs
- Fix duplicate if log_format check in both main.py and marketplace
  __main__.py by consolidating into single if/else blocks
- Add stdout comment on marketplace handler StreamHandler

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@luis5tb luis5tb force-pushed the rate-limit-proxy-ip branch from 9d6296b to 5b17225 Compare June 11, 2026 08:00

@IlonaShishov IlonaShishov left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

luis5tb and others added 2 commits June 12, 2026 14:23
Rate limiter: enable uvicorn proxy header trust (X-Forwarded-For) so
each client gets its own bucket behind GCLB/OpenShift Routes instead of
all tenants sharing a single proxy-IP bucket. Reorder Auth before
RateLimit middleware on the agent service so tenant-based bucketing
(order/user/client) is active — the multi-dimensional principal
resolution was dead code when rate limiting ran before auth.

JSON logging (CWE-117): replace %-format JSON templates with
pythonjsonlogger JsonFormatter so user-controlled content in log
messages cannot break JSON structure or inject fake fields.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add Cloud Armor IP-based throttle rule at priority 100 for pre-auth
  flood protection and document middleware order trade-off in app.py
- Change forwarded_allow_ips default to empty (safe-by-default for dev),
  add explicit FORWARDED_ALLOW_IPS=* in Cloud Run and OpenShift configs
- Fix duplicate if log_format check in both main.py and marketplace
  __main__.py by consolidating into single if/else blocks
- Add stdout comment on marketplace handler StreamHandler

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@luis5tb luis5tb force-pushed the rate-limit-proxy-ip branch from 5b17225 to 38a693f Compare June 12, 2026 14:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants