[github-actions]

agentgateway/agentgateway#4156 — llm: improvements to stream_options

Docs versions: standalone/main, standalone/latest, kubernetes/main, kubernetes/latest (first shipped in prerelease v1.7.0-rc.1)

What changes for users: Chat Completions streams now include a final usage chunk before data: [DONE], even when the client sets stream_options.include_usage to false. Clients that parse stream chunks must handle the final chunk with choices: [] and request-level usage.

What the docs now say: The standalone Chat Completions pages now explain the streamed usage chunk and the stream_options.include_usage override. The prompt-guard and guardrail candidates do not own Chat Completions stream output, and the Kubernetes streaming page was not in the authorized target list for this run.

Not verified: No cluster or standalone process was configured. The configuration is unrun.

How agentgateway-4156 was drafted, and what was not verified

• Code PR: llm: improvements to stream_options agentgateway#4156
• Code issue: none linked
• Product: agentgateway (upstream) → agentgateway/website
• Docs version: standalone/main, standalone/latest, kubernetes/main, kubernetes/latest. First shipped in v1.7.0-rc.1
• Category: scoped-edit / pr

Plan, and what changed it

• Planned: Document the stream_options.include_usage behavior on the pages named by the code-path targets for every listed tree.
• Learned: The dossier diff and changed-functions.md scope the behavior to Chat Completions streams, and the prompt-guard and guardrail candidates cover guard behavior rather than stream output.
• Did: Updated the standalone Chat Completions pages, left the Kubernetes guardrail pages unchanged, and wrote a behavior-change release note for all four trees.

Why a documentation change is necessary

The pull request changes process_completions_request so that every streamed Chat Completions request sets stream_options.include_usage to true, including requests that explicitly set it to false. The StreamOptions doc comment in the diff says the changed behavior adds a final usage chunk before data: [DONE], with an empty choices array and request-level usage. The Chat Completions page already documents response usage conventions, so readers need the streamed-response exception there.

What changed on disk

• content/docs/standalone/main/documentation/llm/api-types/completions.md: Added an anchored section that explains usage chunks in streamed Chat Completions replies.
• content/docs/standalone/latest/documentation/llm/api-types/completions.md: Added the same section for the latest tree.

How to verify this change

1. Run a Chat Completions request through agentgateway with "stream": true and "stream_options": {"include_usage": false}.
2. Confirm that the streamed response includes a final data chunk before data: [DONE] with choices: [] and a populated usage object.
3. Confirm that earlier streamed chunks carry usage: null.

Proposed release note

Wrote proposed-release-note.md as a behavior-change entry for standalone/main, standalone/latest, kubernetes/main, and kubernetes/latest.

What was not verified

• Configuration was not applied to a cluster or run through a standalone agentgateway process.
• Internal links are unchecked because Hugo was not run.
• The Kubernetes stream-output page that contains data: [DONE], content/docs/kubernetes/main/documentation/llm/streaming.md, was not edited because the page was not in the authorized targets.

How the pages were chosen (triage report)

Draft a documentation change

agentgateway/agentgateway#4156 — llm: improvements to stream_options

• Product: agentgateway (upstream) → agentgateway/website
• Docs version: standalone/main, standalone/latest, kubernetes/main, kubernetes/latest
• Summary (read from the body): Default it to true in all cases
• Not tested: no cluster was available, so every command in this diff is unrun

A declared version this draft does not reach

This pull request is planned for more than one documentation version,
and nothing in the diff publishes to:

• kubernetes/latest
• kubernetes/main

A file under assets/ reaches a version tree only where a page in that
tree reuses it, so editing a shared asset does not by itself cover every
mode a product ships. Either that version still needs writing, or the
change genuinely does not apply there and the reason belongs in this body.

Release notes

This pull request also adds its release-note entries:

• content/docs/kubernetes/latest/release-notes/release-notes.md: Chat Completions streams always include usage (breaking_change)
• content/docs/kubernetes/main/release-notes/release-notes.md: Chat Completions streams always include usage (breaking_change)
• content/docs/standalone/latest/release-notes/release-notes.md: Chat Completions streams always include usage (breaking_change)
• content/docs/standalone/main/release-notes/release-notes.md: Chat Completions streams always include usage (breaking_change)

Check these release-note links and names. Each note is on its page, but a link may not reach the drafted content there, or a name may belong to the other mode:

• content/docs/kubernetes/latest/release-notes/release-notes.md: llm: improvements to stream_options agentgateway#4156's note names stream_options.include_usage, which content/docs/standalone/latest mentions and content/docs/kubernetes/latest does not. Check that this tree's readers can set it, or word the note for this mode.
• content/docs/kubernetes/main/release-notes/release-notes.md: llm: improvements to stream_options agentgateway#4156's note names stream_options.include_usage, which content/docs/standalone/main mentions and content/docs/kubernetes/main does not. Check that this tree's readers can set it, or word the note for this mode.
• content/docs/standalone/latest/release-notes/release-notes.md: llm: improvements to stream_options agentgateway#4156's note names . Update stream parsers that assumed streamed chunks before , which content/docs/kubernetes/latest mentions and content/docs/standalone/latest does not. Check that this tree's readers can set it, or word the note for this mode.
• content/docs/standalone/main/release-notes/release-notes.md: llm: improvements to stream_options agentgateway#4156's note names . Update stream parsers that assumed streamed chunks before , which content/docs/kubernetes/main mentions and content/docs/standalone/main does not. Check that this tree's readers can set it, or word the note for this mode.

Draft branch: pr-tracker-draft-agentgateway-4156

[cloudflare-workers-and-pages]

Deploying agentproxy with    Cloudflare Pages

Latest commit:	13efe56
Status:	✅  Deploy successful!
Preview URL:	https://d4154bb7.agentproxy.pages.dev
Branch Preview URL:	https://pr-tracker-draft-agentgatewa-gws5.agentproxy.pages.dev

View logs