Skip to content

Anthropic buildMessageParam silently drops image-URL, PDF, and hosted-file inputs - #615

Open
PratikDhanave (PratikDhanave) wants to merge 5 commits into
microsoft:mainfrom
PratikDhanaveFork:anthropic-buildmessageparam-image-pdf-file
Open

Anthropic buildMessageParam silently drops image-URL, PDF, and hosted-file inputs#615
PratikDhanave (PratikDhanave) wants to merge 5 commits into
microsoft:mainfrom
PratikDhanaveFork:anthropic-buildmessageparam-image-pdf-file

Conversation

@PratikDhanave

Copy link
Copy Markdown
Contributor

Problem

buildMessageParam in provider/anthropicprovider/agent.go only handled base64 image DataContent. Every other multimodal content type fell through the content switch and was silently dropped before the request reached the Anthropic Messages API:

  • *message.URIContent (including image URLs) — no case at all
  • non-image *message.DataContent (application/pdf) — the single if TopLevelMediaType() == "image" skipped it
  • *message.HostedFileContent — no case at all

This diverges from the OpenAI chat provider (provider/openaiprovider/chat.go), which maps URIContent, DataContent, and HostedFileContent, and from the .NET/Python multimodal content mapping where a user message's image/PDF parts are always forwarded. A caller attaching a PDF or an image URL to an Anthropic run got a request with those parts missing and no error.

Change

Extend the content switch in buildMessageParam:

  • base64 application/pdf DataContent -> anthropic.NewDocumentBlock(Base64PDFSourceParam{Data})
  • image URIContent -> anthropic.NewImageBlock(URLImageSourceParam{URL})
  • application/pdf URIContent -> anthropic.NewDocumentBlock(URLPDFSourceParam{URL})

The base64-image path is unchanged. HostedFileContent is left with an explanatory comment rather than a silent skip: the stable Messages API (anthropic.MessageNewParams) in anthropic-sdk-go v1.58.1 exposes no file-id image/document source — only the Beta API has BetaFileImageSourceParam/BetaFileDocumentSourceParam — so a hosted file reference cannot be forwarded here yet.

This brings Anthropic multimodal input handling in line with the OpenAI provider and the cross-SDK (.NET/Python) semantics of forwarding image and document parts on user messages.

Testing

Added TestBuildMessageParam_ImageURLAndPDFAreForwarded to the canonical agent_test.go, using the existing httptest-server harness. It runs a message carrying an image URIContent and an application/pdf DataContent, captures the outgoing request body, and asserts it contains an image block with a url source and a document block with a base64 application/pdf source. The test fails before the fix (both blocks absent) and passes after.

go build ./..., go vet ./provider/anthropicprovider/..., and go test ./provider/anthropicprovider/... all pass.

Copilot AI review requested due to automatic review settings July 23, 2026 05:42
@PratikDhanave
PratikDhanave (PratikDhanave) requested a review from a team as a code owner July 23, 2026 05:42

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes Anthropic provider multimodal input handling so image URLs and PDF inputs are forwarded to the Anthropic Messages API instead of being dropped during request construction.

Changes:

  • Extend buildMessageParam to map URIContent images and PDFs, plus base64 application/pdf DataContent, into Anthropic image/document content blocks.
  • Add a regression test that asserts outgoing requests include both an image-url block and a base64-PDF document block.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
provider/anthropicprovider/agent.go Adds URI/image and PDF (base64 + URL) content block mapping in buildMessageParam; documents HostedFile limitation.
provider/anthropicprovider/agent_test.go Adds a regression test validating image URL + base64 PDF are present in the outgoing Anthropic request body.
Comments suppressed due to low confidence (1)

provider/anthropicprovider/agent.go:500

  • Same as above for URIContent: checking MediaType with exact equality will miss valid values that include parameters (e.g. "application/pdf; ..."), causing the PDF URL to be silently dropped.
			case c.MediaType == "application/pdf":

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread provider/anthropicprovider/agent.go Outdated
mediaType = "image/jpeg"
}
content = append(content, anthropic.NewImageBlockBase64(mediaType, c.Data))
case c.MediaType == "application/pdf":

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — fixed in 90f1ec2: both PDF checks now go through a new isPDFMediaType helper that parses the media type with mime.ParseMediaType and compares the base type to "application/pdf" case-insensitively, so parameterized values like "application/PDF; charset=binary" are recognized instead of dropped. Added a test covering the parameterized case.

Comment thread provider/anthropicprovider/agent.go Outdated
Comment on lines +503 to +507
case *message.HostedFileContent:
// The stable Anthropic Messages API used here (anthropic.MessageNewParams)
// has no file-id image/document source in anthropic-sdk-go v1.58.1; only
// the Beta API exposes BetaFileImageSourceParam/BetaFileDocumentSourceParam.
// A hosted file reference therefore cannot be forwarded yet.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — fixed in 90f1ec2: buildMessageParam now returns an explicit error for HostedFileContent (naming the offending file id and pointing callers to DataContent/URIContent) instead of silently dropping it behind a comment. Added a test asserting the request fails rather than being sent.

@github-actions

This comment has been minimized.

@github-actions github-actions Bot added the parity-approved Go API consistency review found no parity issues label Jul 23, 2026
@PratikDhanave
PratikDhanave (PratikDhanave) force-pushed the anthropic-buildmessageparam-image-pdf-file branch from 64761f4 to e7e5dfa Compare July 24, 2026 01:42
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

buildMessageParam only handled base64 image DataContent and dropped every
other multimodal content type on the floor: URIContent (including image
URLs), application/pdf DataContent, and HostedFileContent all fell through
the switch and were silently discarded before reaching the API.

Extend the switch to match the OpenAI chat provider's multimodal mapping:
base64 PDF DataContent -> NewDocumentBlock(Base64PDFSourceParam), image
URIContent -> NewImageBlock(URLImageSourceParam), and PDF URIContent ->
NewDocumentBlock(URLPDFSourceParam). The stable Messages API in
anthropic-sdk-go v1.58.1 has no file-id image/document source (only the Beta
API does), so HostedFileContent is documented as not-yet-forwardable rather
than silently ignored.
@PratikDhanave
PratikDhanave (PratikDhanave) force-pushed the anthropic-buildmessageparam-image-pdf-file branch from 90f1ec2 to 7f3d1ac Compare July 24, 2026 09:36
# Conflicts:
#	provider/anthropicprovider/agent_test.go
@github-actions

This comment has been minimized.

# Conflicts:
#	provider/anthropicprovider/agent.go
#	provider/anthropicprovider/agent_test.go
@github-actions github-actions Bot added area:provider Changes files in the provider area area:provider/anthropic Changes files in the provider / anthropic area size:large At most 300 changed lines across at most 10 files pending-auto-risk Automatic risk classification is in progress labels Aug 20, 2026
@github-actions

This comment has been minimized.

@github-actions github-actions Bot added failed-auto-risk Automatic risk classification was inconclusive or failed and removed pending-auto-risk Automatic risk classification is in progress labels Aug 20, 2026
# Conflicts:
#	provider/anthropicprovider/agent.go
#	provider/anthropicprovider/agent_test.go
@github-actions github-actions Bot added the pending-auto-risk Automatic risk classification is in progress label Aug 22, 2026
@github-actions github-actions Bot added risk:medium Contained production impact requiring normal review depth and removed failed-auto-risk Automatic risk classification was inconclusive or failed pending-auto-risk Automatic risk classification is in progress labels Aug 22, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Parity Review — PR #615

Scope: provider/anthropicprovider/agent.go — internal buildMessageParam function. No exported Go APIs were changed. The public-api-change label is not warranted.

Cross-SDK comparison

Content type Go (before) Go (after this PR) Python (_chat_client.py) .NET
DataContent image (base64) ✅ forwarded as image/base64 block ✅ unchanged ✅ forwarded delegates to SDK
DataContent application/pdf ❌ silently dropped ✅ forwarded as document/base64 block ⚠️ debug-logged and ignored delegates to SDK
URIContent image URL ❌ silently dropped ✅ forwarded as image/url block ✅ forwarded delegates to SDK
URIContent application/pdf URL ❌ silently dropped ✅ forwarded as document/url block ⚠️ debug-logged and ignored delegates to SDK
HostedFileContent ❌ silently dropped ✅ explicit error returned ❌ no case (silent drop) delegates to SDK

Assessment

This PR is a correctness fix, not a feature addition. Silently dropping multimodal content without an error is a bug in all SDKs, and Go is patching its own instance of that bug.

Two minor divergences exist relative to Python upstream:

  1. PDF document forwarding — Go now forwards application/pdf DataContent and URIContent as Anthropic document blocks. Python currently logs and ignores these. Go is ahead of Python here, but in a clearly correct direction aligned with Anthropic Messages API capabilities. No upstream feature gate governs this; it is a provider-level implementation choice.

  2. HostedFileContent error vs. silent drop — Go returns an explicit error (anthropic: hosted file references ... are not supported); Python has no case and silently drops. The explicit error is safer for callers and does not break parity — it improves observability.

Neither divergence represents a semantic regression or a missed feature gate. The PR description's claim that this matches Python semantics is slightly imprecise for PDF forwarding (Python ignores PDF today), but the Go behavior is correct.

Result: ✅ Parity approved. The PR fixes a bug in Go's multimodal content mapping and leaves Go in a correct and slightly more capable state than Python's current implementation. No cross-repo consistency issues require action. The parity-approved label remains appropriate.

Generated by Go API Consistency Review Agent · sonnet46 · 28 AIC · ⌖ 5.84 AIC · ⊞ 6K ·

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:provider/anthropic Changes files in the provider / anthropic area area:provider Changes files in the provider area parity-approved Go API consistency review found no parity issues risk:medium Contained production impact requiring normal review depth size:large At most 300 changed lines across at most 10 files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants