feat(http): standardize the 401 response, with a proxy-config note

CLAUDE.md

@@ -114,6 +114,10 @@ For HTTP transport, the wire protocol maps to separate endpoints: `POST /vgi/{me
 
 **Authentication**: `AuthContext` (frozen dataclass) carries `domain`, `authenticated`, `principal`, and `claims`. For HTTP transport, `make_wsgi_app(authenticate=...)` installs `_AuthMiddleware` that calls the callback on each request and populates `CallContext.auth`. Pipe transport gets anonymous auth by default. Methods can call `ctx.auth.require_authenticated()` to gate access.
 
+**Unauthorized responses (HTTP)**: Every 401 follows `docs/unauthorized-spec.md`. `vgi_rpc/http/_unauthorized.py` defines `AuthReason` (closed set: `missing_credential`, `invalid_credential`, `expired_credential`, `insufficient_scope`, `proxy_required`, `unauthorized`), `AuthFailure` (a `ValueError` carrying a reason, so `chain_authenticate` still treats it as "try the next credential"), and `AuthenticationError` (an `RpcError` subclass the client raises, with `reason` / `detail` / `proxy_hint`). `_AuthMiddleware` classifies the callback's exception onto `req.context.vgi_auth_reason`; `_make_error_serializer` in `http/server/_errors.py` renders **JSON** (`{error, reason, detail, proxy_hint?}`) unless `Accept` contains `text/html`, in which case it renders the styled page. Both carry `VGI-Auth-Reason` and `Cache-Control: no-store`.
+
+The reason code names the *stage*, never a verifier's diagnosis — every proxy-proof outcome collapses onto `proxy_required`, which is what keeps `docs/proxy-proof-spec.md` §6's uniform-rejection rule intact. **The proxy note** (`VGI-Auth-Proxy-Required: true` + `proxy_hint`) is derived from *server configuration*, not from what failed on a given request, so it is identical on every 401 and discloses nothing: it exists for the deployment where the reverse proxy isn't forwarding the header and *everything* 401s. Header dependencies are auto-discovered — `mtls_authenticate*` and `proxy_proof_gate` (require mode only) call `declare_proxy_headers` on themselves, and `chain_authenticate` / `require_all` propagate; `make_wsgi_app(proxy_auth_headers=[...])` states them directly for a custom authenticator. Cross-language conformance group: `TestUnauthorized`, capability-gated on `VGI-Auth-Reason`.
+
 **Server identity**: Each `RpcServer` gets a `server_id` (auto-generated 12-char hex or caller-supplied). This ID is attached to all log and error batches as `vgi_rpc.server_id` metadata for distributed tracing. `RpcServer` also accepts `enable_describe=True` to register the synthetic `__describe__` introspection method.
 
 **Protocol identity**: `RpcServer` computes `protocol_hash` (SHA-256) over the canonical `__describe__` payload at construction. Three answer-different questions:

README.md

@@ -505,6 +505,7 @@ with http_connect(MyService, "http://localhost:8080") as proxy:
 | `enable_health_endpoint` | `True` | JSON health check at `GET {prefix}/health` (bypasses auth) |
 | `repo_url` | `None` | Source-repository link surfaced on the landing page |
 | `oauth_resource_metadata` | `None` | `OAuthResourceMetadata` for RFC 9728 discovery |
+| `proxy_auth_headers` | `None` | Headers a trusted reverse proxy must inject for auth to succeed. Adds a "check the proxy configuration" note to every [401](docs/unauthorized-spec.md). The built-in mTLS and proxy-proof authenticators are discovered automatically; this is for a custom `authenticate` |
 | `enable_sticky` | `False` | Enable HTTP [sticky sessions](docs/sticky-sessions-spec.md) — `ctx.open_session(state)` binds a Python object to the worker for the session token's lifetime |
 | `sticky_default_ttl` | `300.0` | Default session TTL in seconds for sticky sessions; overridable per-call via `ctx.open_session(state, ttl=...)` |
 | `sticky_echo_headers` | `None` | Headers the client replays on every subsequent request in the session (e.g. Fly.io routing) |

docs/WIRE_PROTOCOL.md

@@ -551,7 +551,7 @@ Response body: IPC stream (result_schema, 0..N log batches, 1 result/error batch
 
 HTTP 200: Success (even when the response contains an error batch)
 HTTP 400: Protocol error (bad IPC, missing metadata, param validation failure)
-HTTP 401: Authentication failure (plain-text body, NOT Arrow IPC)
+HTTP 401: Authentication failure (JSON envelope or HTML page, NOT Arrow IPC)
 HTTP 404: Unknown method
 HTTP 415: Wrong Content-Type
 HTTP 500: Server implementation error
@@ -698,7 +698,7 @@ is inside the ciphertext, so it cannot be tampered with independently.
 When the server has an `authenticate` callback configured:
 
 - The callback receives the HTTP request and returns an `AuthContext`.
-- On failure (`ValueError` or `PermissionError`), the server returns **HTTP 401** with a **plain-text** body (NOT Arrow IPC), because no method has been resolved yet and no output schema is available.
+- On failure (`ValueError` or `PermissionError`), the server returns **HTTP 401**. The body is NOT Arrow IPC — no method has been resolved yet, so no output schema is available. Its shape is the standardized envelope of `docs/unauthorized-spec.md`: a JSON object carrying a `reason` code from a closed set, mirrored on a `VGI-Auth-Reason` header, or the styled HTML page when the request's `Accept` asks for `text/html`.
 - Other exceptions from the callback propagate as HTTP 500.
 - Clients MUST detect 401 responses before attempting to parse Arrow IPC.
 
@@ -710,8 +710,9 @@ per-worker secret.
 
 - It is a **precondition ANDed with** the `authenticate` callback above, never an alternative
   credential — the caller's `Authorization` header is untouched and still carries the end user.
-- Failure maps to the same **HTTP 401 + plain-text body** as any other authenticate failure. The
-  body MUST NOT echo the reason or the claimed key id.
+- Failure maps to the same **HTTP 401** as any other authenticate failure, carrying the
+  `proxy_required` reason code. The body MUST NOT echo the verifier's reason or the claimed key
+  id — every proof outcome collapses onto that one code.
 - `OPTIONS`, `/.well-known/`, and `{prefix}/health` are exempt in all modes, so load-balancer
   probes and capability discovery keep working.
 - A worker requiring proofs advertises `VGI-Proxy-Proof-Required: true` on every response.
@@ -965,8 +966,8 @@ IPC Stream (error):
 | Type error in implementation | 400 Bad Request |
 
 > **Note**: Even for HTTP 400/500 responses, the response body is a valid
-> Arrow IPC stream containing an error batch, except for 401 (plain text)
-> and 415 (Falcon default response).
+> Arrow IPC stream containing an error batch, except for 401 (JSON or HTML,
+> per `docs/unauthorized-spec.md`) and 415 (Falcon default response).
 
 ---
 

docs/index.md

@@ -37,6 +37,7 @@ Define RPC interfaces as Python [`Protocol`](https://docs.python.org/3/library/t
 - **IPC validation** — configurable batch validation levels for untrusted data
 - **Large batch support** — transparent [externalization to S3/GCS](api/external.md) for oversized data
 - **HTTP response caps** — bound on-wire body size and external upload volume per call via [`max_response_bytes` and `max_externalized_response_bytes`](hosting.md)
+- **Standardized 401s** — every HTTP rejection carries a reason code from a closed set, plus a note when the cause is likely a missing reverse-proxy configuration — see [Unauthorized Responses](unauthorized-spec.md)
 - **Sticky sessions** — opt-in HTTP affinity binding stateful objects (cursors, model handles) to a client across calls — see [Sticky Sessions](sticky-sessions-spec.md)
 - **Per-call I/O statistics** — [`CallStatistics`](api/core.md#callstatistics) tracks batches, rows, and bytes for usage accounting (access log + [OTel](api/otel.md) spans)
 - **Wire protocol debug logging** — enable `vgi_rpc.wire` at DEBUG for full wire-level visibility — see [Logging](api/logging.md#wire-protocol-debugging)

docs/porting-guide.md

@@ -196,6 +196,18 @@ ignore the budget.
 - **HTTP state-token format.** Tokens are AEAD-sealed (XChaCha20-Poly1305 or ChaCha20-Poly1305, depending on what's available natively in the target language). Each port is free to choose its own plaintext encoding — Python uses length-prefixed Arrow IPC, Go uses gob, TypeScript uses JSON+BigInt, Java uses CBOR, Rust uses length-prefixed bytes — because tokens are not expected to round-trip across language ports. The behavioral contract is per-port: round-trip integrity, cross-principal replay protection (via AEAD AAD or per-principal key derivation), and TTL enforcement after authenticity. See `vgi_rpc/http/server/_state_token.py` for the Python reference.
 - **Per-process server identity.** `server_id` is generated once per process lifetime, NOT per call. The same string must appear in every log record from the same instance.
 
+## HTTP unauthorized responses
+
+Every 401 a conformant HTTP server emits carries a coarse reason code from a closed set, on a `VGI-Auth-Reason` header and in a JSON body, plus a static note on services whose authentication depends on a reverse proxy. The full spec lives at [`docs/unauthorized-spec.md`](unauthorized-spec.md). The canonical `TestUnauthorized` group is capability-gated on `VGI-Auth-Reason`, so a port that has not adopted this skips cleanly.
+
+The pieces worth calling out:
+
+1. **Negotiate on `Accept`, and default to JSON.** `*/*` — what every RPC client sends — MUST resolve to the JSON envelope. A port may serve JSON to browsers too; it may never serve HTML to a client that did not ask for `text/html`. Getting this backwards is the pre-existing bug this spec was written to fix: the Python client used to paste an entire HTML page into an exception message.
+2. **Keep the reason set closed.** Six codes, listed in §3 of the spec. A failure that maps to none of them is `unauthorized`. A client switching on the code needs the set not to grow under it in a language it does not control.
+3. **The code names the stage, not the diagnosis.** Every proxy-proof outcome collapses onto `proxy_required` — this is what keeps the uniform-rejection rule of the proxy-proof spec intact while still telling an operator which layer refused.
+4. **Derive the proxy note from configuration, never from the request.** Emit it on every 401 from a proxy-dependent service, identically. That is what makes it safe to show, and it is still correct in the case it exists for: a proxy that is not forwarding the header 401s *everything*.
+5. **Carry declarations through composition.** If your chain / require-all helpers do not propagate an authenticator's proxy-header dependency, wrapping one silently drops the note — which is exactly the deployment where you needed it.
+
 ## HTTP proxy proof
 
 Proxy proof is an **opt-in additive feature**: a worker can refuse any request that did not arrive through a trusted proxy, which is verified by recomputing an HMAC over a timestamp, a nonce and the worker's own identifier. The full spec lives at [`docs/proxy-proof-spec.md`](proxy-proof-spec.md). A port may:

docs/proxy-proof-spec.md

@@ -170,6 +170,8 @@ Steps 1–4 involve no MAC computation and MUST be performed first.
 
 **Rejection is uniform.** The response body MUST NOT contain the verifier's message, the reason code, or any echo of `kid` — an attacker controls that field. Detail goes to logs and metrics only. In `require` mode a failure maps to **HTTP 401**, matching every existing authenticate-callback failure in the framework.
 
+That 401 follows [`docs/unauthorized-spec.md`](unauthorized-spec.md), which is compatible with the paragraph above rather than an exception to it. Every outcome in the table above — `no_proof`, `malformed`, `unknown_kid`, `expired`, `not_yet_valid`, `bad_mac`, `replayed` — collapses onto the single reason code `proxy_required`, with an identical detail string. The code names the stage that refused the request, which the response already said by rejecting it at all; it never names which check was tripped. A `require`-mode worker also carries that spec's proxy-configuration note, again identical on every 401 and derived from configuration rather than from the request, so it cannot be used to probe anything.
+
 ## 7. Modes
 
 | Mode | Behavior |