Skip to content

feat: support Gemini URL Context - #4349

Merged
dgageot merged 3 commits into
mainfrom
feat/gemini-url-context
Sep 18, 2026
Merged

dgageot merged 3 commits into
mainfrom
feat/gemini-url-context

Conversation

@dgageot

@dgageot dgageot commented Sep 17, 2026

Copy link
Copy Markdown
Member

Gemini exposes a url_context built-in tool that lets the model fetch and reason over public URLs at inference time. This adds opt-in support for it behind a provider_opts.url_context: true flag, consistent with the existing google_search, google_maps, and code_execution toggles.

The change wires url_context into builtInTools() in the Gemini client, extends diagnostics and error classification to cover responses that reference the new tool, and adds test coverage for direct, gateway, and Vertex routing paths — including disabled and malformed cases, and scenarios that mix built-in tools with custom function declarations.

One pre-existing SDK restriction is documented but not fixed here: on Vertex AI, mixing built-in tools (including url_context) with custom function-call tools is unsupported by the underlying SDK. The combination is tested and the error is classified correctly, but callers must avoid it until the SDK lifts the restriction. The schema, docs for the Google provider, and a runnable examples/gemini_url_context.yaml are all updated.

Rebased onto main (2e0650eb3), preserving the embedding schema additions. Example-loading tests now use a local DMR HTTP stub instead of depending on a running Docker Model Runner, and the URL Context docs pass Markdown lint.

Validation: task build, task test, task lint, markdownlint-cli2@0.22.1, and race tests for pkg/model/provider/gemini, pkg/config, and pkg/teamloader all pass locally. No live paid API calls were made.

@aheritier aheritier added area/docs Documentation changes area/providers For features/issues/fixes related to LLM providers (Bedrock, LiteLLM, Qwen, custom, etc.) area/providers/gemini Google Gemini provider support kind/feat PR adds a new feature (maps to feat:). Use on PRs only. labels Sep 17, 2026
@dgageot
dgageot marked this pull request as ready for review September 17, 2026 17:05
@dgageot
dgageot requested a review from a team as a code owner September 17, 2026 17:05
@dgageot
dgageot force-pushed the feat/gemini-url-context branch from 78e95db to 45e5f17 Compare September 18, 2026 08:18
@aheritier aheritier added the status/needs-rebase PR has merge conflicts or is out of date with main label Sep 18, 2026
@aheritier

Copy link
Copy Markdown
Collaborator

👋 This PR has merge conflicts with the base branch. Please rebase or merge the latest base branch and resolve them. I've moved it to draft and added status/needs-rebase; it'll be picked back up automatically once the conflicts are cleared.

@dgageot
dgageot force-pushed the feat/gemini-url-context branch from 45e5f17 to 81df0da Compare September 18, 2026 10:10
gtardif
gtardif previously approved these changes Sep 18, 2026
@dgageot
dgageot added this pull request to the merge queue Sep 18, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Sep 18, 2026
@dgageot
dgageot force-pushed the feat/gemini-url-context branch from 81df0da to 0307821 Compare September 18, 2026 11:10
@dgageot dgageot removed the status/needs-rebase PR has merge conflicts or is out of date with main label Sep 18, 2026
aheritier
aheritier previously approved these changes Sep 18, 2026
@dgageot
dgageot added this pull request to the merge queue Sep 18, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Sep 18, 2026
Built-in toggle via config, diagnostics/classification wiring,
tests, schema update, docs and example.

Assisted-By: Docker AI Agent
Signed-off-by: David Gageot <david.gageot@docker.com>
Signed-off-by: David Gageot <david@gageot.net>

Assisted-By: Claude
…overy

Signed-off-by: David Gageot <david@gageot.net>

Assisted-By: Claude
@dgageot
dgageot added this pull request to the merge queue Sep 18, 2026
Merged via the queue into main with commit 47ef6ba Sep 18, 2026
21 checks passed
@dgageot
dgageot deleted the feat/gemini-url-context branch September 18, 2026 12:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/docs Documentation changes area/providers/gemini Google Gemini provider support area/providers For features/issues/fixes related to LLM providers (Bedrock, LiteLLM, Qwen, custom, etc.) kind/feat PR adds a new feature (maps to feat:). Use on PRs only.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants