MCP support

[noorbhatia]

Hey @mattt
I want to connect to MCP servers for dynamic tool discovery
Are there any plans for MCP support in AnyLanguageModel? If so, what's the intended approach?

[airy10]

You can have a look at this project for dynamically creating AnyLanguageModel Tools from MCP server tools : https://github.com/airy10/AppleFoundationMCPTool

[mattt]

@noorbhatia Yeah, that's a good question. Most of the documentation around Foundation Models focuses on tools and generable types declared at build time and compiled with macros. What you want for MCP is to build these dynamically at runtime, which I think is possible, but I haven't seen yet.

Here's a sketch of what I think that looks like at a high level:

/// A Tool backed by an MCP server tool discovered at runtime.
struct MCPTool: Tool {
    typealias Arguments = GeneratedContent
    typealias Output = String

    let name: String
    let description: String
    let parameters: GenerationSchema

    /// The closure that actually calls the MCP server.
    private let handler: @Sendable (GeneratedContent) async throws -> String

    init(
        name: String,
        description: String,
        inputSchema: JSONSchema,   // from MCP tools/list response
        handler: @escaping @Sendable (GeneratedContent) async throws -> String
    ) throws {
        self.name = name
        self.description = description
        self.handler = handler

        // Convert the MCP JSON Schema → DynamicGenerationSchema → GenerationSchema
        let dynamicSchema = convertToDynamicSchema(inputSchema, name: name)
        self.parameters = try GenerationSchema(
            root: dynamicSchema, 
            dependencies: []  // add deps if the schema has $defs/references
        )
    }

    func call(arguments: GeneratedContent) async throws -> String {
        try await handler(arguments)
    }
}

There are some finer details to work out wrt/schema conversion and MCP client statefulness, but this seems to be the general shape of a solution. Just a matter of documenting it and then deciding what additional affordances we want to provide — and how (custom API in AnyLanguageModel? separate library?)

[Adi2K]

---

Hey @mattt — I'd like to take a crack at implementing MCP support directly in AnyLanguageModel, following the direction from your sketch above. Here's the approach I'm planning:

Trait-gated integration — MCP support behind a package trait ("MCP"), bringing in modelcontextprotocol/swift-sdk as the dependency.

Two layers, cleanly separated:

1. MCPTool — Follows your sketch closely. Conforms to Tool with GeneratedContent as Arguments, String as Output. Schema conversion goes through JSONSchema → DynamicGenerationSchema → GenerationSchema,
   leveraging the existing convertToDynamicSchema pattern.
2. MCPToolBridge — Convenience layer for connection management and tool discovery. Handles the MCP client lifecycle, wraps discovered tools as MCPTool instances, and manages argument/response conversion.
   Also exposes an init(client:) for developers who want to manage their own MCP client.

Both live under Sources/AnyLanguageModel/MCP/. Tests use the MCP SDK's InMemoryTransport for deterministic in-process testing.

[airy10]

1. MCPTool — Follows your sketch closely. Conforms to Tool with GeneratedContent as Arguments, String as Output. Schema conversion goes through JSONSchema → DynamicGenerationSchema → GenerationSchema,
   leveraging the existing convertToDynamicSchema pattern.
2. MCPToolBridge — Convenience layer for connection management and tool discovery. Handles the MCP client lifecycle, wraps discovered tools as MCPTool instances, and manages argument/response conversion.
   Also exposes an init(client:) for developers who want to manage their own MCP client.

In case that was missed, https://github.com/airy10/AppleFoundationMCPTool/ already have a MCPDynamicTool and MCPToolBridge implementation - certainly not perfect (and not fully tested) but probably worth a look as it basically does what you want...

[mattt]

@Adi2K Thanks for your offer. I'm not eager to layer in MCP support at this time, if only because I suspect that Apple may soon introduce new APIs for MCP — either in the March SDK release or at WWDC (I have no direct knowledge of this; this is pure speculation on my part). Until then, I'd prefer to adopt a more neutral stance and say that MCP support is left up to the API consumer; at most we can steer folks in the right direction, because there's a lot of variation in how an app might interact with MCP servers.

@airy10 Thank you for sharing this. Taking a quick look at the implementation, I found a few potential issues. For example, this code seems to truncate multimedia tool responses to the first few bytes.

[airy10]

@airy10 Thank you for sharing this. Taking a quick look at the implementation, I found a few potential issues. For example, this code seems to truncate multimedia tool responses to the first few bytes.

Right. It's not complete, not properly tested (and probably buggy) while it's working fine with AnyLanguageModel for my own use case.
But pointing to it as either some starting point or just for showing some implementations ideas (and that's actually where my first implementation of AnyLanguageModel->SystemLanguageModel schema conversion for SystemLanguageModel tools support came from..)

[christopherkarani]

For MCP-style dynamic tool discovery, Conduit is taking the runtime-schema-tool path: https://github.com/christopherkarani/Conduit

The current branch adds DynamicTool, which lets a caller register tools with runtime GenerationSchema values and GeneratedContent arguments/outputs. That is a useful bridge for MCP servers because discovered tools can be mapped into the same tool executor without generating static Swift types first: https://github.com/christopherkarani/Conduit/tree/codex/conduit-foundationmodels-provider-expansion

[mattt]

Closing as not planned. My position from February hasn't changed: MCP integration is best done by the app, and the MCPTool sketch above is still how I'd approach it. Let me know if you have any more thoughts on the matter!