gquota

Drop-in rate-limit layer for the Gemini API free tier. Exponential backoff, TTL response cache, RPM pacing, and tolerant JSON recovery — wrapped in one function call.

from gquota import gemini

r = gemini.generate("gemini-2.5-flash", "Give me a one-line market summary")
r.text    # plain text
r.data    # parsed JSON dict (if the model emitted JSON — recovered tolerantly)
r.raw     # full API response
r.cached  # True if served from cache, zero quota spent

That's it. 429s get retried with backoff + jitter, repeated prompts hit a TTL cache, calls are paced under your RPM budget, and slightly-broken JSON still parses. Zero dependencies (stdlib urllib only).

Why this exists: the Delphi war-story

Delphi is a prediction-market scanning agent that prices ~15 markets per scan using the Gemini free tier (~10 RPM). It runs as three concurrent scans against the same API key — a 4h cron, an hourly scan, and an ad-hoc CLI.

One afternoon all three fired inside the same minute. The result was a mutual 429 stampede, and it looked like this in the scan log:

15:59:47 open markets: 15
15:59:47 MARKET 0x2c178e08 ...  implied=? model=FAIL
15:59:47 MARKET 0xb8fcc2c6 ...  implied=? model=FAIL
15:59:47 MARKET 0x76225fe8 ...  implied=? model=FAIL
   ... (15/15 markets FAIL, all in the same second)

15/15 markets failed, instantly. Every process got a 429, every process crashed straight through its per-market handling, and — worse — when the crons retried on schedule they hit each other again, because three processes retrying at fixed intervals stay in lockstep forever.

The fix, shipped in delphi-agent/agent.mjs (geminiCall), was four layers:

1. Backoff 4s → 8s → 16s + jitter — the jitter is the part that actually kills the stampede: uniform 0–1.5s noise decorrelates the three processes so they stop colliding.
2. Response cache with TTL (2h) — the 4h cron was re-pricing unchanged questions. Identical (model, prompt) pairs skip the network entirely. Cache hits cost no quota and skip pacing.
3. RPM pacing (6.5s between calls) — ~10 RPM measured on the free tier; staying under it proactively beats reacting to 429s.
4. Tolerant JSON recovery — Gemini occasionally emits unescaped quotes inside a reasoning string (e.g. quoting a market title). Strict JSON.parse fails, the scan marks the market FAIL, quota already spent. The recovery ladder: strict parse → strip markdown fences → extract outermost {...} → per-field regex salvage.

Four minutes later, the hardened scan ran again:

16:03:37 0x2c178e ... implied=0.488 model=0.480@gemini-flash-latest edge=-0.8pp →pass 56h
16:03:51 0x76225f ... implied=0.459 model=0.459@gemini-3-flash-preview edge=-0.0pp →pass 52h
16:04:01 0xb4ded8 ... implied=0.166 model=0.170@gemini-3-flash-preview edge=+0.4pp →pass 67h

Prices recovered, markets passing again, no code changes to the market logic — only to the call layer. The next day's scan ran nearly clean (4 FAILs out of 15, all far-dated markets the models legitimately declined). That pattern has been running in production since August 2026.

gquota is that layer, extracted, generalized, and tested — so your project doesn't have to re-learn it during its own stampede.

Install

pip install git+https://github.com/jtabsbm/gquota.git

Requires Python ≥ 3.9. No dependencies.

Usage

The one-liner

import os
os.environ["GEMINI_API_KEY"] = "..."   # or GOOGLE_API_KEY

from gquota import gemini

r = gemini.generate("gemini-2.5-flash", 'Output ONLY JSON: {"p": <probability>, "reasoning": "<one sentence>"}')
print(r.data["p"], r.data["reasoning"])

Configure once

from gquota import gemini

gemini.configure(
    api_key="...",            # optional, falls back to GEMINI_API_KEY
    rpm=10,                   # pacing: min interval becomes 60/10 = 6s
    cache_ttl_s=7200,         # 2h response cache
    backoff_waits_s=[4, 8, 16],
    models=[                  # fallback ladder if your model 404s mid-run
        "gemini-2.5-flash",
        "gemini-flash-latest",
        "gemini-2.0-flash",
    ],
)

The full layer (any REST endpoint)

Provider is endpoint-agnostic — inject your own transport (which is also how the tests mock HTTP):

from gquota.core import Backoff, Pacer, Provider, ResponseCache

provider = Provider(
    api_key="...",
    base_url="https://generativelanguage.googleapis.com/v1beta",
    endpoint="models/{model}:generateContent",
    models=["gemini-2.5-flash", "gemini-2.0-flash"],
    backoff=Backoff(waits_s=[4, 8, 16], jitter_s=1.5),
    cache=ResponseCache(ttl_s=7200),
    pacer=Pacer(min_interval_s=6.5),
)

r = provider.generate("gemini-2.5-flash", "Hello", system="You are terse.", temperature=0.2)

Error handling

Exception	Meaning
gquota.core.RateLimitError	Every model returned 429/503 past the whole backoff schedule. Back off longer, or reduce RPM.
gquota.core.RetryExhausted	No model produced a usable response (HTTP errors, empty candidates, unparseable body). The message lists model:reason per failure.

Stats

Every provider counts what it did — useful for tuning:

provider.stats   # {"calls": 3, "retries": 2, "cache_hits": 1, "pace_sleeps": 4}

Demo run (mock transport, real code path)

$ python demo.py
call 1  text: {"p": 0.42, "confidence": 0.8, "reasoning": "base rate for August typh
        data: {'p': 0.42, 'confidence': 0.8, 'reasoning': 'base rate for August typhoons'}
        stats: {'calls': 3, 'retries': 2, 'cache_hits': 0, 'pace_sleeps': 0}
call 2  cached: True | HTTP requests still: 3 | stats: {'calls': 3, 'retries': 2, 'cache_hits': 1, 'pace_sleeps': 0}

The first call hit two 429s, backed off twice, and recovered; the second identical prompt was served from cache with zero additional HTTP requests.

How the four layers work

Layer	Default	What it does
Backoff	4s, 8s, 16s + 0–1.5s jitter	Retries 429/503; jitter decorrelates concurrent processes
ResponseCache	2h TTL, 4096 entries	Identical (model, prompt) never re-burns quota
Pacer	6.5s min interval (~9 RPM)	Proactively stays under the free-tier RPM cap
recover_json	—	Strict → fences → outermost {...} → per-field salvage

Plus: model fallback — if the requested model 404s (deprecated mid-run, as gemini-flash-latest once did during a live Delphi scan), the next model in the ladder is tried immediately with no penalty.

Testing

python -m unittest discover -s tests -v

34 tests, all with HTTP mocked via injected transport — no network, no API key, runs in ~0.1s. Verified with Python 3.11.

Sponsor

If gquota saved your scan (or your weekend), consider GitHub Sponsors — it funds the free-tier quota that produced every bug in here.

License

MIT