Non Deterministic PDF file processing

[alok992-tech]

when a user uploads a PDF file sometime CLI adds the entire content into base64 content and send to LLM and some case send as PDF as attachments. Incase of attachment LLM is able to invoke the PDF skills but other cases it does not invoke skills as it does not recognize this as a PDF file

[github-actions]

Investigation Findings

What I Investigated

I looked at the following code paths:

• python/copilot/session.py — BlobAttachment, FileAttachment, and Attachment type definitions
• nodejs/src/generated/session-events.ts, go/generated_session_events.go, dotnet/src/Generated/SessionEvents.cs — the UserMessageData schema generated from the protocol definition
• docs/features/image-input.md and all SDK READMEs — existing documentation on attachments

What I Found

The behavior described is not a bug — it is intentional, model-dependent behavior built into the Copilot CLI runtime. The SDK protocol exposes two fields on UserMessageData that reveal exactly what's happening:

• supportedNativeDocumentMimeTypes — "Normalized document MIME types that were sent natively instead of through tagged_files XML." When the active model supports PDF natively (e.g., certain Gemini or Claude variants), the PDF is sent as a native document attachment directly to the LLM.
• nativeDocumentPathFallbackPaths — "Path-backed native document attachments that stayed on the tagged_files path flow because native upload would exceed the request size limit." When the file is too large for native upload, the runtime falls back to extracting the text via the skill-pdf pipeline instead.
• source: "skill-pdf" — messages injected by the PDF skill (text extraction path) carry this source tag and are filtered from the user-visible timeline.

So the two paths are:

1. Native attachment path (model supports PDF natively + file within size limit) → LLM receives the PDF directly → PDF skills are accessible.
2. skill-pdf / tagged_files path (model does not support PDF natively, or file exceeds the size limit) → runtime extracts text and injects it as XML context → LLM never sees a PDF attachment, so PDF skills are not invoked.

This decision is made entirely in the Copilot CLI server, not in the SDK itself. The SDK correctly passes FileAttachment and BlobAttachment (with MIME type) to the CLI; the CLI then decides the processing path.

Why This Is a Documentation Issue

The root cause of the confusion is a clear documentation gap:

1. Attachment docs only cover images. docs/features/image-input.md and all SDK READMEs describe BlobAttachment only in terms of images. There is no mention of PDF support, the application/pdf MIME type, or how PDFs should be attached.
2. No explanation of model-dependent PDF routing. There is nowhere in the SDK documentation that explains the supportedNativeDocumentMimeTypes mechanism, or that PDF processing varies based on which model is active.
3. No guidance on the fallback behavior. The nativeDocumentPathFallbackPaths fallback (triggered by file size) is entirely undocumented from the SDK consumer perspective.

Recommended Documentation Additions

The following gaps should be addressed:

• A new docs/features/pdf-input.md (or extension of image-input.md) explaining:
  • How to attach PDF files using FileAttachment (path) or BlobAttachment (base64 + mimeType: "application/pdf")
  • That the runtime chooses between native PDF delivery and skill-pdf text extraction based on the active model's capabilities
  • That UserMessageData.supportedNativeDocumentMimeTypes and nativeDocumentPathFallbackPaths events can be observed to understand which path was taken
  • Guidance on choosing a model known to support native PDF (when PDF skill invocation is required)
• Update the existing image-input.md / READMEs to note that BlobAttachment also applies to PDFs and other document types.

I've applied the documentation label accordingly.

Generated by Bug Handler for issue #1147 · ● 682.1K · ◷

[alok992-tech]

i have observed the CLI choose different options with the same Model. Some cases it sends as blob attachment and some cases as file attachment.

[SteveSandersonMS]

This sounds like a question about Copilot CLI, not Copilot SDK. Assuming that's the case, would it be OK to re-file this issue at https://github.com/github/copilot-cli? Thanks!