Retry rate-limited searches and surface a keyless search provider - #199
Merged
Conversation
A single HTTP 429 used to throw straight out of the provider, and the
orchestrator advances its chain on any throw. One transient rate limit
therefore downgraded a whole session to the weakest configured provider.
A GAIA validation campaign logged 1341 keyless-Exa 429s, 44% of all tool
failures in the run.
searchHttp now retries a 429 against the same provider (2 retries, 500ms
doubling) before returning it. A server Retry-After wins over the local
schedule; both clamp to 10s so one hostile header cannot stall a turn.
The header rides the existing curl -w meta line via %header{retry-after},
read off the marker rather than a fixed field because a header value may
itself contain the separator. curl older than 7.83 emits the literal
format string, which is treated as absent.
Retries are spent, not skipped, when a limit is real: the fallback chain
remains the backstop once they are exhausted.
Refs #179
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
web.search.provider defaults to exa, whose MCP endpoint answers keyless when EXA_API_KEY is unset. The duckduckgo fallback then works exactly as designed, so nothing hard-fails and no error surfaces; the run just produces weaker groundings than the operator configured. Silent degradation is the worst of both worlds because it neither works well nor says why. Emit one stderr line at tool construction naming the provider, the unset variable, and the fallback that will actually serve traffic. Warning once at construction rather than per search keeps a long autonomous run from drowning in it. Also documents both mechanisms in AGENTS.md and corrects the module map, which still described duckduckgo as the default provider. Refs #179 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
sosidudku1
force-pushed
the
fix/search-429-backoff-and-key-warning
branch
from
August 20, 2026 14:44
9b59167 to
e5c4801
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the largest single source of tool failure in the GAIA validation
campaign: 1341
Exa returned HTTP 429errors, 44% of all 3039 toolfailures. Addresses suggestions 1 and 3 of #179.
Retry before falling through
A 429 threw straight out of the provider, and the orchestrator advances
its fallback chain on any throw. One transient rate limit therefore
downgraded the rest of the session to the weakest configured provider.
searchHttpnow retries a 429 against the same provider (2 retries,500 ms doubling) before returning it. A server
Retry-Afterwins over thelocal schedule; both clamp to 10 s so one hostile header cannot stall a
turn. The header rides the existing
curl -wmeta line via%header{retry-after}; curl older than 7.83 emits the literal formatstring, which is treated as absent. Retries are spent, not skipped, when a
limit is real — the fallback chain remains the backstop.
Name the degradation
web.search.providerdefaults toexa, which runs keyless whenEXA_API_KEYis unset. Theduckduckgofallback then works as designed,so nothing hard-fails and no error surfaces — the run just produces weaker
groundings than configured.
One stderr line at tool construction now names the provider, the unset
variable, and the fallback that will actually serve traffic. Once at
construction, not per search, so a long autonomous run does not drown in it.
On suggestion 2 (
cacheTtlMinutes)Not changed, deliberately. The cache is per-process, in-memory, capped at
256 entries, and keyed on the exact query string. A longer TTL neither
survives the per-task restarts a campaign does nor catches the near-miss
rephrasings that actually burn quota, while it would serve staler results
for time-sensitive lookups. A restart-surviving cache is the real fix and
is a separate piece of work.
Testing
24 new tests;
npm run lintclean. Full suite: 4861 passed, 4 failed —all 4 pre-existing on
origin/main(fs-glob-realdepends on files in$HOME;llm-health-polleris a timing flake that passes in isolation).🤖 Generated with Claude Code