Merge streaming tool call deltas by index in OpenAiChatModel#6381
Merge streaming tool call deltas by index in OpenAiChatModel#6381jewoodev wants to merge 2 commits into
OpenAiChatModel#6381Conversation
d1d69a0 to
6c04458
Compare
|
Rebased onto main to resolve a conflict with #6365. #6365 changed The streaming test added in #6365 ( |
The streaming chunk aggregation detects the start of a new tool call by the presence of the delta id field. OpenAI omits id on continuation deltas, but some OpenAI-compatible endpoints (e.g. DeepSeek) send an empty string instead, so every continuation delta surfaced as a separate tool call with fragmented arguments and tool execution failed. Key the merge on the delta's required index field instead, which the streaming API defines for correlating tool call deltas and which the aggregation used until 2.0.0-M8. This also lifts the one-tool-call-per- chunk limitation and merges parallel tool call deltas correctly. Fixes spring-projectsgh-6374 Signed-off-by: jewoodev <jewoos15@naver.com>
6c04458 to
b03dc62
Compare
|
Thanks for this PR — the While testing with DashScope/Qwen, I also hit two 1. When a continuation chunk carries // current
Function lastFromTc1F = lastFromTc1.function().get();Suggested: Function lastFromTc1F = lastFromTc1.function()
.orElseGet(() -> Function.builder().build());2. All four // current
toolCallBuilder.id(tc.id().get());
toolCallBuilder.function(ChatCompletionMessageFunctionToolCall.Function.builder()
.name(tc.function().get().name().get())
.arguments(tc.function().get().arguments().get())
.build());Suggested: toolCallBuilder.id(tc.id().orElse(""));
ChatCompletionMessageFunctionToolCall.Function.Builder funcBuilder = ChatCompletionMessageFunctionToolCall.Function.builder();
tc.function().ifPresent(f -> {
funcBuilder.name(f.name().orElse(""));
funcBuilder.arguments(f.arguments().orElse(""));
});
toolCallBuilder.function(funcBuilder.build());Reproduced on Spring AI 2.0.0 GA with DashScope Qwen3 streaming + tool_calls. Happy to open a separate issue if that would help track it. Thanks! (Originally submitted as #6522, closed in favor of this PR and #6373.) |
|
@jewoodev Thanks for the great work on this PR! Would you be open to me adding the If yes, could you add me as a collaborator (Write permission) to your fork at https://github.com/jewoodev/spring-ai/settings/access ? My GitHub username is If you prefer to keep the PR scope tight, I'm happy to wait and open a follow-up PR after this one merges. Just let me know which you prefer. Thanks! |
Avoid direct Optional.get() calls when converting streamed tool call deltas into chat completions. Missing tool call ids, functions, names, and arguments are now normalized to empty strings during conversion. Add regression coverage for arguments-only tool call deltas and deltas without a function object. Signed-off-by: jewoodev <jewoos15@naver.com>
|
Thanks for the feedback. Added a small follow-up commit that guards the remaining The I'll keep follow-up changes on this branch on my side to keep the PR history straightforward. Feel free to leave any additional feedback in this PR. |
|
@sdeleuze Could you take a look at this PR when you have a chance? It fixes streaming tool-call merging for empty |
OpenAI-compatible streaming endpoints can send continuation tool-call deltas with
id: ""instead of omittingid.OpenAiChatModel.ChunkMergertreats any present id as a new tool-call boundary, so argument fragments become separateToolCalls with empty id/name and tool execution receives empty or null arguments.This PR merges tool-call deltas by the required
indexfield instead. It fixes the empty-id case from gh-6374, keeps OpenAI-style omitted ids working, and removes the one-tool-call-per-chunk assumption so parallel or multi-delta chunks merge correctly. #6369 covers the empty-id continuation case; this PR uses the index-keyed path so those related cases stay covered together.Tests cover omitted ids, empty ids, parallel tool calls, and multiple deltas per chunk. Verified with
./mvnw -pl models/spring-ai-openai package.Fixes gh-6374