Skip to content

[Enhancement] Add portable LLM tool/function definitions for AI agent frameworks #721

@deepgram-robot

Description

@deepgram-robot

Summary

Add pre-built tool/function definitions that allow AI agent frameworks (CrewAI, Anthropic tool use, OpenAI function calling, LlamaIndex) to call Deepgram's STT, TTS, and Audio Intelligence APIs as tools — without developers manually writing JSON schema definitions for each Deepgram capability.

Problem it solves

Developers building AI agents that need voice capabilities must manually write tool definitions (JSON schemas, function signatures, description strings) for every Deepgram API they want to expose. This is tedious, error-prone, and must be redone for each agent framework's tool format. Meanwhile, competitors ship pre-built "skills" packages (e.g., ElevenLabs' skills repo with 156 stars) that make their APIs instantly callable from any LLM agent. Pre-built tool definitions make Deepgram the default voice layer in the growing AI agent ecosystem.

Proposed API

from deepgram.tools import transcribe_audio, speak_text, analyze_audio

# Use with Anthropic tool use
tools = [transcribe_audio.to_anthropic(), speak_text.to_anthropic()]

# Use with OpenAI function calling
functions = [transcribe_audio.to_openai(), speak_text.to_openai()]

# Use with CrewAI
from crewai import Agent
agent = Agent(tools=[transcribe_audio.to_crewai(), analyze_audio.to_crewai()])

# Use with LangChain
from deepgram.tools import DeepgramTranscribeTool, DeepgramSpeakTool
tools = [DeepgramTranscribeTool(), DeepgramSpeakTool()]

# Each tool handles execution internally
result = await transcribe_audio.execute(url="https://example.com/audio.mp3")

Acceptance criteria

  • Provides tool definitions for core APIs: transcribe (file/URL), transcribe (streaming), speak, analyze
  • Each tool exports to at least 3 formats: Anthropic, OpenAI, and generic JSON Schema
  • Tools are self-executing — calling the tool invokes the Deepgram API with proper auth
  • Documented with usage examples for each supported framework
  • Compatible with existing SDK API (uses same client/auth)

Raised by the DX intelligence system.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions