Add Groq agent tool cassette regressions#2011
Merged
Merged
Conversation
ef29a14 to
13a5d08
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a broad cassette-backed Groq regression suite for Rig's OpenAI-compatible Groq chat-completions path.
New scenarios under
tests/providers/groq/agent_tool_sessions.rscover:parallel_tool_calls;ToolChoice::{Auto, Required, Specific, None}wire behavior;Models used
llama-3.3-70b-versatile: primary stable Groq-hosted chat/tool model for tool loops, streaming, JSON object mode, usage, and metadata.openai/gpt-oss-20b: JSON-schema structured-output scenario because Groq rejectsjson_schemaresponse format forllama-3.3-70b-versatile.Inspiration references
Reviewed these implementation/test patterns in
many_rigs/inspirations:vercel-ai-sdk/packages/groq/src/groq-chat-language-model.tsvercel-ai-sdk/packages/groq/src/groq-chat-language-model.test.tsvercel-ai-sdk/packages/groq/src/groq-prepare-tools.tsvercel-ai-sdk/packages/groq/src/groq-prepare-tools.test.tspydantic-ai/pydantic_ai_slim/pydantic_ai/models/groq.pypydantic-ai/tests/models/test_groq.pypydantic-ai/tests/providers/test_groq.pylangchain/libs/partners/groq/langchain_groq/chat_models.pylangchain/libs/partners/groq/tests/integration_tests/test_chat_models.pyThose references motivated coverage for specific tool choice, Groq
response_format, streamingx_groq.usage, reasoning-token usage details, and Groq timing metadata.Bugs found and fixed
max_tokenswas not forwarded in chat-completions requests.ToolChoice::Specificincorrectly reused the generic OpenAI Chat Completions conversion, which rejected specific tool choices even though Groq supports a single function-choice object.output_schemafor Groq; Groq now maps Rig output schemas toresponse_format: { type: "json_schema", ... }.x_groq.usage; Rig now reads that path in addition to terminalusagechunks.completion_tokens,completion_tokens_details.reasoning_tokens, and Groq timing fields (queue_time,prompt_time,completion_time,total_time) and maps output token usage fromcompletion_tokenswhen present.-D warningsafter the latest toolchain started flagginglarge_enum_variant.Validation
Record-mode commands run with
GROQ_API_KEYfrom the environment:Replay/safety/unit checks:
cargo testwas also attempted and reached the pre-existing unrelated OpenAI cassette mismatch inopenai::cassette::permission_control::permission_control_streaming_example.Manual cassette inspection
/openai/v1/chat/completionspath and expected request bodies.tests/cassettes/groq/.system_fingerprint,usage,x_groq, and timing fields where the live API surfaced them.GROQ_API_KEY.Non-goals / known gaps
openai/gpt-oss-20bfor JSON-schema output after observingllama-3.3-70b-versatilereject that response format.