Anthropic buildMessageParam silently drops image-URL, PDF, and hosted-file inputs - #615
Conversation
There was a problem hiding this comment.
Pull request overview
This PR fixes Anthropic provider multimodal input handling so image URLs and PDF inputs are forwarded to the Anthropic Messages API instead of being dropped during request construction.
Changes:
- Extend
buildMessageParamto mapURIContentimages and PDFs, plus base64application/pdfDataContent, into Anthropic image/document content blocks. - Add a regression test that asserts outgoing requests include both an image-url block and a base64-PDF document block.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| provider/anthropicprovider/agent.go | Adds URI/image and PDF (base64 + URL) content block mapping in buildMessageParam; documents HostedFile limitation. |
| provider/anthropicprovider/agent_test.go | Adds a regression test validating image URL + base64 PDF are present in the outgoing Anthropic request body. |
Comments suppressed due to low confidence (1)
provider/anthropicprovider/agent.go:500
- Same as above for URIContent: checking MediaType with exact equality will miss valid values that include parameters (e.g. "application/pdf; ..."), causing the PDF URL to be silently dropped.
case c.MediaType == "application/pdf":
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| mediaType = "image/jpeg" | ||
| } | ||
| content = append(content, anthropic.NewImageBlockBase64(mediaType, c.Data)) | ||
| case c.MediaType == "application/pdf": |
There was a problem hiding this comment.
Good catch — fixed in 90f1ec2: both PDF checks now go through a new isPDFMediaType helper that parses the media type with mime.ParseMediaType and compares the base type to "application/pdf" case-insensitively, so parameterized values like "application/PDF; charset=binary" are recognized instead of dropped. Added a test covering the parameterized case.
| case *message.HostedFileContent: | ||
| // The stable Anthropic Messages API used here (anthropic.MessageNewParams) | ||
| // has no file-id image/document source in anthropic-sdk-go v1.58.1; only | ||
| // the Beta API exposes BetaFileImageSourceParam/BetaFileDocumentSourceParam. | ||
| // A hosted file reference therefore cannot be forwarded yet. |
There was a problem hiding this comment.
Good catch — fixed in 90f1ec2: buildMessageParam now returns an explicit error for HostedFileContent (naming the offending file id and pointing callers to DataContent/URIContent) instead of silently dropping it behind a comment. Added a test asserting the request fails rather than being sent.
ff998a3 to
4f62ed7
Compare
4f62ed7 to
64761f4
Compare
This comment has been minimized.
This comment has been minimized.
64761f4 to
e7e5dfa
Compare
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
buildMessageParam only handled base64 image DataContent and dropped every other multimodal content type on the floor: URIContent (including image URLs), application/pdf DataContent, and HostedFileContent all fell through the switch and were silently discarded before reaching the API. Extend the switch to match the OpenAI chat provider's multimodal mapping: base64 PDF DataContent -> NewDocumentBlock(Base64PDFSourceParam), image URIContent -> NewImageBlock(URLImageSourceParam), and PDF URIContent -> NewDocumentBlock(URLPDFSourceParam). The stable Messages API in anthropic-sdk-go v1.58.1 has no file-id image/document source (only the Beta API does), so HostedFileContent is documented as not-yet-forwardable rather than silently ignored.
90f1ec2 to
7f3d1ac
Compare
# Conflicts: # provider/anthropicprovider/agent_test.go
This comment has been minimized.
This comment has been minimized.
# Conflicts: # provider/anthropicprovider/agent.go # provider/anthropicprovider/agent_test.go
This comment has been minimized.
This comment has been minimized.
# Conflicts: # provider/anthropicprovider/agent.go # provider/anthropicprovider/agent_test.go
Parity Review — PR #615Scope: Cross-SDK comparison
AssessmentThis PR is a correctness fix, not a feature addition. Silently dropping multimodal content without an error is a bug in all SDKs, and Go is patching its own instance of that bug. Two minor divergences exist relative to Python upstream:
Neither divergence represents a semantic regression or a missed feature gate. The PR description's claim that this matches Python semantics is slightly imprecise for PDF forwarding (Python ignores PDF today), but the Go behavior is correct. Result: ✅ Parity approved. The PR fixes a bug in Go's multimodal content mapping and leaves Go in a correct and slightly more capable state than Python's current implementation. No cross-repo consistency issues require action. The
|
Problem
buildMessageParaminprovider/anthropicprovider/agent.goonly handled base64 imageDataContent. Every other multimodal content type fell through the contentswitchand was silently dropped before the request reached the Anthropic Messages API:*message.URIContent(including image URLs) — no case at all*message.DataContent(application/pdf) — the singleif TopLevelMediaType() == "image"skipped it*message.HostedFileContent— no case at allThis diverges from the OpenAI chat provider (
provider/openaiprovider/chat.go), which mapsURIContent,DataContent, andHostedFileContent, and from the .NET/Python multimodal content mapping where a user message's image/PDF parts are always forwarded. A caller attaching a PDF or an image URL to an Anthropic run got a request with those parts missing and no error.Change
Extend the content switch in
buildMessageParam:application/pdfDataContent->anthropic.NewDocumentBlock(Base64PDFSourceParam{Data})URIContent->anthropic.NewImageBlock(URLImageSourceParam{URL})application/pdfURIContent->anthropic.NewDocumentBlock(URLPDFSourceParam{URL})The base64-image path is unchanged.
HostedFileContentis left with an explanatory comment rather than a silent skip: the stable Messages API (anthropic.MessageNewParams) in anthropic-sdk-go v1.58.1 exposes no file-id image/document source — only the Beta API hasBetaFileImageSourceParam/BetaFileDocumentSourceParam— so a hosted file reference cannot be forwarded here yet.This brings Anthropic multimodal input handling in line with the OpenAI provider and the cross-SDK (.NET/Python) semantics of forwarding image and document parts on user messages.
Testing
Added
TestBuildMessageParam_ImageURLAndPDFAreForwardedto the canonicalagent_test.go, using the existing httptest-server harness. It runs a message carrying an imageURIContentand anapplication/pdfDataContent, captures the outgoing request body, and asserts it contains animageblock with aurlsource and adocumentblock with a base64application/pdfsource. The test fails before the fix (both blocks absent) and passes after.go build ./...,go vet ./provider/anthropicprovider/..., andgo test ./provider/anthropicprovider/...all pass.