Skip to content

Remove retries in LLM router#127

Open
aswink wants to merge 3 commits intomainfrom
aswin/02-26-remove_retries
Open

Remove retries in LLM router#127
aswink wants to merge 3 commits intomainfrom
aswin/02-26-remove_retries

Conversation

@aswink
Copy link
Contributor

@aswink aswink commented Feb 26, 2026

lingua is more intended to do provider translation than request/retry
semantics, so remove retries from this library. Callers can take care of
retries themselves.

Copy link
Contributor Author

aswink commented Feb 26, 2026

This stack of pull requests is managed by Graphite. Learn more about stacking.

@aswink aswink changed the title Remove retries Remove retries in LLM router Feb 26, 2026
@aswink aswink requested review from knjiang and remh February 26, 2026 22:41
@aswink aswink marked this pull request as ready for review February 26, 2026 22:41
@aswink aswink force-pushed the aswin/02-26-remove_retries branch from e18f58f to 4b5c643 Compare February 26, 2026 22:42
Self {
connect_timeout: Duration::from_secs(10),
request_timeout: Duration::from_secs(300),
// Similar to what anthropic and openai SDKs use
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really related to the rest of the change, happy to drop it if it seems unnecessary. Though I do wonder if we'll need the higher request timeout

Copy link
Contributor

@knjiang knjiang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just one comment to think through. i'm not sure if i totally like passing a retry policy throiugh but that was my first thought

@aswink aswink force-pushed the aswin/02-26-remove_retries branch from a73b975 to 0395cea Compare February 27, 2026 00:21
@aswink aswink force-pushed the aswin/02-26-remove_retries branch 3 times, most recently from f8ef27f to e02a55a Compare February 27, 2026 00:40
@aswink
Copy link
Contributor Author

aswink commented Feb 27, 2026

CI passes on the main repo: https://github.com/braintrustdata/braintrust/pull/11253

@aswink aswink force-pushed the aswin/02-26-remove_retries branch from f7a3b61 to 0483aa0 Compare March 2, 2026 18:54
aswink added 3 commits March 4, 2026 11:04
lingua is more intended to do provider translation than request/retry
semantics, so remove retries from this library. Callers can take care of
retries themselves.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants