Skip to content

fix(gemini): surface model-generated images instead of dropping them - #41468

Open
IngTian wants to merge 1 commit into
anomalyco:devfrom
IngTian:fix/gemini-inline-image-output
Open

fix(gemini): surface model-generated images instead of dropping them#41468
IngTian wants to merge 1 commit into
anomalyco:devfrom
IngTian:fix/gemini-inline-image-output

Conversation

@IngTian

@IngTian IngTian commented Aug 10, 2026

Copy link
Copy Markdown

Issue for this PR

Closes #41467

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Gemini image models return generated images as inlineData, but nothing carries them into the session, so the image is dropped and the user sees only the model's text. The request is still billed.

Two gaps, so three small changes:

  1. LLMEvent.File — there was no event to carry media. Media arrives whole rather than as deltas, so there's no start/delta/end triple.
  2. Gemini decoder emits it for inlineData, closing any open reasoning block first so media doesn't land inside one.
  3. SessionProcessor persists it as a file part, closing the current text block first to keep ordering. It reuses the same image normalization tool attachments already use, so an oversized generated image can't blow up the next request; if normalization is unavailable it keeps the original rather than dropping the image.

Also: toModelMessages replays assistant file parts and the Gemini request builder accepts assistant media, so a follow-up turn ("make it brighter") can edit the generated image instead of starting over. Under stripMedia (compaction/overflow) it degrades to a text placeholder like other media.

Note responseModalities: ["TEXT", "IMAGE"] must still be set in the model's provider options, or Vertex returns text only.

How did you verify your code works?

Tests: packages/llm 299 pass, packages/opencode session suite 399 pass / 0 fail, typecheck clean on both.

New coverage — a Gemini decoder test asserting a file event is emitted for inlineData alongside the text, a processor test asserting the image is persisted as a file part, and two toModelMessages tests for replay and stripMedia. I checked the processor test fails without the fix, so it's real coverage rather than a tautology.

I also updated the existing rejects unsupported assistant media content test, which asserted the old behavior ("only support text, reasoning, and tool-call content for now") — that's the placeholder this PR fills in.

Not verified: live Vertex traffic. Decoder behavior is covered by a synthetic SSE fixture rather than a real provider response.

Screenshots / recordings

Not a UI change — this makes the file part exist; existing clients render it.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

Gemini image models (gemini-*-image, "Nano Banana") return generated
images as `inlineData` parts. The Gemini stream decoder only handled
`text` and `functionCall`, so image parts fell through the loop and were
discarded. The user is billed for the image output tokens and sees only
the model's text ("Here is the image you asked for") with no image.

There was no `file` event in the LLM event union to carry the media, so
this adds one and threads it through:

- `LLMEvent.File` — a complete media block. Media arrives whole rather
  than as deltas, so there is no start/delta/end triple.
- Gemini decoder emits it for `inlineData`, closing any open reasoning
  block first so media does not land inside it.
- `SessionProcessor` persists it as a `file` part, closing the current
  text block first to preserve ordering, and reuses the same image
  normalization tool attachments use so an oversized generated image
  cannot blow up the next request. If normalization is unavailable it
  keeps the original rather than dropping the image.
- `toModelMessages` replays assistant `file` parts, and the Gemini
  request builder now accepts assistant media, so follow-up turns
  ("make it brighter") can edit a generated image instead of starting
  over. Under `stripMedia` (compaction/overflow) it degrades to a text
  placeholder like other media.

Note that `responseModalities: ["TEXT", "IMAGE"]` must still be set in
the model's provider options for Gemini to return images at all; without
it the API returns text only.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@IngTian

IngTian commented Aug 10, 2026

Copy link
Copy Markdown
Author

Two notes for whoever picks this up.

Relationship to #40126. That PR fixes the same gap on v2 and is further along — if v2 is where you want this, close mine in its favour. I opened this one because dev is where releases are cut (1.18.15 is the current one) and the branches have diverged, so #40126 will not reach dev users: packages/opencode doesn't exist on v2, and v2 currently sits 1267 commits behind with no merges from dev in its recent history. Happy to be wrong about that if you're planning a sync.

Pairs with #41463. On dev these two are sequential, not alternatives. #41463 stops Vertex rejecting the request outright ("model does not support function calling"); this PR makes the returned image visible. With only #41463 you get a billed request whose image is silently dropped; with only this one the request still 400s unless the user hand-writes an agent with permission: {"*": deny}.

@github-actions

Copy link
Copy Markdown
Contributor

The following comment was made by an LLM, it may be inaccurate:

I found a potentially related PR:

PR #40126: feat(session): support Gemini image generation
#40126

This PR appears to be related as it also addresses Gemini image generation support. You should check whether this earlier PR covers similar ground or if there's any overlap in approach between the two PRs.

IngTian added a commit to IngTian/openchamber that referenced this pull request Aug 10, 2026
The image-drop bug now has a dev-targeted PR (anomalyco/opencode#41468, issue
#41467) rather than only the v2-branch one, so both halves of the upstream fix
are traceable from here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Gemini image models: generated images are discarded, never reach the session

1 participant