Skip to content

Commit 659a23b

Browse files
docs(api): correct adaptive chat completion cache doc (keyed by endpoint+model) (#1492)
The doc called the compatibility cache "model-independent", but cacheKey is `${normalizedEndpoint}\n${model.trim()}` — learned compatibility is keyed by endpoint AND model, not shared across models. The tests confirm it ("isolates learned compatibility by endpoint and model"; LRU evicts after N endpoint-model keys). Describe the actual behavior.
1 parent fffe5a2 commit 659a23b

1 file changed

Lines changed: 3 additions & 2 deletions

File tree

‎apps/api/src/adaptive-chat-completions.ts‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -150,8 +150,9 @@ async function negotiate<TResponse extends ChatCompletionHttpResponse>(
150150
}
151151

152152
/**
153-
* Sends a chat completion using a process-local, model-independent compatibility
154-
* cache. Only structured unsupported-parameter responses trigger negotiation.
153+
* Sends a chat completion using a process-local compatibility cache keyed by
154+
* endpoint and model. Only structured unsupported-parameter responses trigger
155+
* negotiation.
155156
*/
156157
export async function postAdaptiveChatCompletion<
157158
TResponse extends ChatCompletionHttpResponse,

0 commit comments

Comments
 (0)