feat(gateways): AI-generated plugin for cohere - #5
Open
startupmini wants to merge 15 commits into
Open
Conversation
Adds a gateway generator that lets a free-tier LLM (Gemini flash-lite by default, OpenRouter :free fallback) produce custom gateway plugins for providers whose APIs do not match the OpenAI-compatible shape. How it works: - Register a gateway in packages/collectors/manifest.json (url, auth, secrets, optional static sample). - pnpm gen-gateway <id> probes the endpoint (or uses the sample), captures a redacted fixture, summarizes the response shape, and asks the LLM for a complete CustomGateway plugin. Output is structurally validated (importable, correct id, has collect(), no forbidden patterns) and retried up to 3 times. - heal mode (--heal) regenerates a plugin from the existing fixture when a sample changes or a mapping breaks. - .github/workflows/gateway-ai.yml runs the generator on demand and opens a review PR; it also validates the manifest on PRs that touch it. Includes the first generated example: cohere.ts (written by the LLM from a static sample), its fixture, a generic harness test that validates generated plugins against their fixtures, and the COHERE_API_KEY secret registration.
… terminated early)
…y loop The generator now treats typecheck failures and live collection errors as validation failures, so the LLM is asked to fix them (up to 3 attempts) instead of shipping a plugin that compiles structurally but breaks typecheck or maps live API data to invalid Model records. - validateGeneratedPlugin: forbidden patterns + structural + tsc --noEmit + optional live check - runTypecheck: spawns node with the local tsc.js (no shell, cross-platform) - bootstrap/heal pass liveSecrets into the loop; duplicate post-hoc live check removed - cohere.ts: fix pagination field (next_page -> next_page_token) from review of AI output
probeGateway now probes the live endpoint first when the gateway's key is available, falling back to the manifest sample when it is not (or when the live probe fails). This feeds the LLM the real response shape (field names, nulls, pagination) instead of a stale static sample, so generated plugins match the live API and the loop's live check converges. Also instruct the LLM to never use 'any' (use unknown + narrowing) so attempts are not wasted on forbidden patterns.
checkCollection now captures the first ModelSchema issue per invalid record (up to 5 unique examples) so the retry prompt tells the LLM exactly which field fails and why (e.g. 'context_window: expected number, received null') instead of a bare count, letting it converge within the attempt budget.
…fallback OpenRouter (deepseek-chat-v3-0324:free by default) is now the primary LLM provider when OPENROUTER_API_KEY is set, falling back to Gemini on failure. LLM_PROVIDER=openrouter|gemini forces a single provider. This gives the retry loop a stronger code-generation model than flash-lite so plugins converge within the attempt budget.
- llm.ts: add Requesty provider (router.requesty.ai) with free model mistral/leanstral-1-5 by default; provider order requesty -> gemini -> openrouter; LLM_PROVIDER env override; OpenRouter free candidates rotate (deepseek :free no longer available). - gateway-ai.yml: pass REQUESTY_API_KEY to the generator env. - prompts.ts: forbid unused variables (noUnusedLocals) explicitly. - manifest sample: add next_page_token and null context_length/tokenizer_url entries so the LLM learns pagination and nullable fields. - generate/heal: raise default maxAttempts to 4. - cohere.ts: regenerated plugin handles pagination (next_page_token, maxPages) and nullable fields; passes forbidden/typecheck/unit validation.
…ct Model type in prompt
…s, truncate heal feedback, handle non-positive context_length
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AI-generated gateway plugin for \cohere\ (mode: bootstrap).
Reviewed before merge, per docs/08_Gateway_Plugin_Security.md: