Skip to content

feat(ai-settings): auto-discover models for OpenAI-compatible providers - #2415

Open
Quim T. (espetro) wants to merge 1 commit into
browseros-ai:mainfrom
espetro:feat/auto-discover-models
Open

feat(ai-settings): auto-discover models for OpenAI-compatible providers#2415
Quim T. (espetro) wants to merge 1 commit into
browseros-ai:mainfrom
espetro:feat/auto-discover-models

Conversation

@espetro

Copy link
Copy Markdown

Closes #2411

Problem

For OpenAI-compatible endpoints (including Ollama and LM Studio), the Add Provider dialog showed a bundled models.dev catalog that is only ever a stale sample of what the user's endpoint actually serves. Users with local or proxied models had to paste exact model IDs by hand, and short catalogs (e.g. LM Studio's three entries) read as the complete list, leading users to conclude their loaded models were unsupported.

Implementation

Server (apps/server)

  • New listProviderModels in lib/clients/llm/list-models.ts: normalizes the base URL (strips trailing slash and trailing /v1), fetches GET {base}/v1/models with optional Bearer auth, 15s timeout, and maps { id, context_length } entries. Non-string ids are filtered out.
  • New route factory createListModelRoutes in api/routes/provider.ts, mounted at POST /list-models in api/routes/index.ts (separate from /test-provider to avoid the ugly /test-provider/models path).
  • Soft-fail: on any upstream error the route returns 200 with { models: [], message }, so the client always keeps its free-form model entry UX.

Client (apps/app)

  • New listModels in lib/llm-providers/listModels.ts: posts to the agent server and maps results to ModelInfo, defaulting contextLength to 128000 when absent. Any failure returns an empty array (no throw).
  • NewProviderDialog: a useDiscoveredModels hook auto-fetches (600ms debounce) whenever the provider type serves user-loaded models (openai-compatible, ollama, lmstudio) and a base URL is present. Discovered models replace the stale catalog in the picker; the free-form "paste exact ID" path is retained as fallback. A small refresh button next to the Model label re-triggers discovery. The "lists only common models" hint is suppressed once discovery returns results.
  • Ollama and LM Studio get this for free (same USER_LOADED_MODEL_PROVIDERS set).

Test plan

  • apps/server/src/lib/clients/llm/list-models.test.ts: happy path, missing data, non-string ids, 401/500 throws, URL normalization (/v1, /v1/, trailing slash, bare host), Bearer header forwarding. All 15 tests pass.
  • apps/app/lib/llm-providers/listModels.test.ts: mapping plus contextLength default, network failure returns [], non-200 returns []. Passes.
  • biome check clean on all touched files; tsc --noEmit clean for touched files (pre-existing generated-graphql errors unchanged); full server and app suites show only pre-existing failures (verified identical on clean main).

Out of scope

  • Capability detection from /models metadata beyond context_length (not consistently present across providers).
  • Persistent caching of discovered model lists.

@greptile-apps

greptile-apps Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

PR author is not in the allowed authors list.

@github-actions

github-actions Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA. Thank you!
Posted by the CLA Assistant Lite bot.

@espetro

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature Request] Auto-discover available models for OpenAI Compatible providers (fetch GET /v1/models instead of manual ID entry)

1 participant