Skip to content

Commit 4b8b336

Browse files
committed
feat(aicore): add patch_litellm_for_credential_rotation for LangGraph agents
Agents using ChatLiteLLM (LangGraph) call litellm.completion directly, bypassing the SDK completion() wrapper and its reactive 401 handler. patch_litellm_for_credential_rotation() wraps litellm.completion/acompletion at the module level so ALL callers get transparent credential reload on AuthenticationError — including ChatLiteLLM — without any code changes to the agent's LLM call patterns. Recommended startup pattern for LangGraph agents: set_aicore_config() patch_litellm_for_credential_rotation() # reactive reload for ChatLiteLLM watch_aicore_config() # proactive reload on rotation Adds test_langgraph_compat.py (15 tests) covering: env sharing across callers, reactive path scope documentation, watcher sufficiency, patch behaviour, idempotency, and full startup pattern end-to-end. Addresses review comment from thiagob on PR #256 re: LangGraph template compat.
1 parent 10bfdc8 commit 4b8b336

2 files changed

Lines changed: 525 additions & 0 deletions

File tree

src/sap_cloud_sdk/aicore/__init__.py

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -240,9 +240,68 @@ def _watch() -> None:
240240
return thread
241241

242242

243+
def patch_litellm_for_credential_rotation() -> None:
244+
"""Patch ``litellm.completion`` / ``litellm.acompletion`` globally so ALL callers
245+
get transparent credential reload on ``AuthenticationError``.
246+
247+
LangGraph agents typically call ``litellm.completion`` through ``ChatLiteLLM``
248+
(LangChain), bypassing the SDK's own ``completion()`` wrapper and its built-in
249+
401-reload handler. Call this function once at agent startup to extend the same
250+
reactive reload behaviour to **every** litellm caller in the process.
251+
252+
Idempotent — calling more than once has no additional effect.
253+
254+
Recommended startup pattern for LangGraph / ChatLiteLLM agents::
255+
256+
from sap_cloud_sdk.aicore import (
257+
set_aicore_config,
258+
patch_litellm_for_credential_rotation,
259+
watch_aicore_config,
260+
)
261+
262+
set_aicore_config() # load credentials
263+
patch_litellm_for_credential_rotation() # reactive reload for ChatLiteLLM
264+
watch_aicore_config() # proactive reload on secret rotation
265+
266+
Agents that already use the SDK's ``completion()`` / ``acompletion()`` wrappers
267+
do not need this — those wrappers already handle 401s transparently.
268+
"""
269+
import litellm as _litellm
270+
271+
if getattr(_litellm, "_sap_aicore_patched", False):
272+
return
273+
274+
_orig_completion = _litellm.completion
275+
_orig_acompletion = _litellm.acompletion
276+
277+
def _completion(*args, **kwargs):
278+
try:
279+
return _orig_completion(*args, **kwargs)
280+
except _litellm.AuthenticationError:
281+
logger.info("AI Core credentials reloading after authentication failure")
282+
set_aicore_config()
283+
return _orig_completion(*args, **kwargs)
284+
285+
async def _acompletion(*args, **kwargs):
286+
try:
287+
return await _orig_acompletion(*args, **kwargs)
288+
except _litellm.AuthenticationError:
289+
logger.info("AI Core credentials reloading after authentication failure")
290+
set_aicore_config()
291+
return await _orig_acompletion(*args, **kwargs)
292+
293+
_litellm.completion = _completion
294+
_litellm.acompletion = _acompletion
295+
_litellm._sap_aicore_patched = True
296+
logger.info(
297+
"litellm patched for AI Core credential rotation — applies to all callers"
298+
)
299+
300+
243301
__all__ = [
244302
"set_aicore_config",
245303
"watch_aicore_config",
304+
"patch_litellm_for_credential_rotation",
246305
"set_filtering",
247306
"disable_filtering",
248307
"completion",

0 commit comments

Comments
 (0)