Skip to content

fix(providers): keep model metadata lookups offline when requested - #909

Open
LivXue wants to merge 1 commit into
mainfrom
fix/model_catalog_cache_only
Open

LivXue wants to merge 1 commit into
mainfrom
fix/model_catalog_cache_only

Conversation

@LivXue

@LivXue LivXue commented Oct 11, 2026

Copy link
Copy Markdown
Member

Summary

Creating an ACP session or opening the model picker could query LiteLLM's provider metadata API even with allow_fetch=False. Unknown Ollama models could block session/new beyond the EOF cleanup grace period and drop its reply.

  • Forward the lookup policy to both context-window and output-ceiling metadata queries. Cache misses skip get_model_info when fetching is disabled.
  • Keep loaded LiteLLM rows, cached OpenRouter metadata, and SDK lookups where fetching is allowed.
  • Cover uncached model spellings, cached aliases, real model.options handling, and all three ACP handshake replies after EOF.

Windows stdio validation uses the stdin fix from #908 in an isolated source snapshot. This PR carries only the metadata fix and its regression tests.

Type

  • Fix
  • Feature
  • Docs
  • CI / tooling
  • Refactor
  • Other

Verification

  • Relevant tests pass locally
  • Relevant lint / type checks pass locally
  • User-facing docs or screenshots are updated when needed

Native Windows, Python 3.13.12: 602 relevant tests passed.

uv run --frozen pytest -n 0 tests/test_provider_rates.py tests/test_rpc_model.py tests/test_acp_server.py tests/test_acp_methods.py tests/test_acp_config_options.py tests/test_agent_loop_lazy_provider.py tests/test_lazy_provider.py tests/test_token_wise_pricing.py tests/test_rpc_session_init_bundle.py -q --tb=short

Before the fix, 12 new regression cases failed while both allowed-fetch controls passed. Removing only the output guard produced 5 failures; removing only the context-window guard produced 7 failures, including the RPC and ACP cases. Source was restored after each mutation.

Native stdio smoke: 6 passed, including the complete handshake, with PYTHONPATH pointing at an isolated copy of this change plus raven/cli/acp_commands.py from #908.

uv run --frozen pytest -n 0 tests/integration/test_acp_stdio_smoke.py -m integration -q -x --tb=short

A direct Windows smoke run on the branch alone remains blocked by the inherited WinError 6 stdin transport failure, reproduced before this fix. The combined run above verifies the two fixes together. No LLM calls or real Ollama service were used.

The following checks passed; all 10 import contracts were kept.

uv run --frozen pre-commit run --files raven/providers/rates.py tests/test_provider_rates.py tests/test_rpc_model.py tests/test_acp_server.py
uv run --frozen ty check raven/providers/rates.py
uv run --frozen lint-imports
uv run --frozen python scripts/check_large_files.py origin/main
uv run --frozen python scripts/check_source_language.py origin/main
git diff --check

No user-facing documentation update is needed: configuration and protocol contracts are unchanged. Resolver docstrings document the enforced lookup policy.

Risk

  • Security impact considered
  • Backward compatibility considered
  • Rollback path is clear for risky changes

When every local source misses, callers requesting local metadata retain the existing unknown-value or configured-default behavior. A self-hosted model can set agents.defaults.contextWindowTokens explicitly. Fetch-enabled lookup behavior is preserved. Roll back by reverting this change.

Related Issues

#908 (Windows stdin fix used for combined stdio validation).

Honor allow_fetch=False for context-window and output-ceiling metadata,
including a loaded LiteLLM whose SDK can query provider endpoints.
Keep cached rows and explicitly allowed SDK lookups available.

Cover cache misses, aliases, model options, and ACP session creation at EOF.

Co-authored-by: Codex <noreply@openai.com>
@LivXue
LivXue requested review from 0xKT and gloryfromca October 11, 2026 13:24

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers; this can merge as far as I am concerned.

Reviewed the full github/main...HEAD diff and the surrounding resolver, candidate-alias, OpenRouter-cache, model-picker, binding, and ACP call paths, plus the relevant history. The new policy gate prevents litellm.get_model_info from running when allow_fetch=False while preserving already-loaded static rows, cached OpenRouter rows, and the existing fetch-enabled default. The optional parameters preserve compatibility, and the tests add coverage without weakening existing assertions.

I also checked the repository rules in AGENTS.md/CLAUDE.md, the domain-term routing in CONTEXT-MAP.md, test naming, source language/comments, and the provider/RPC architecture constraints; I found no violations.

Verification: uv run pytest -n 0 tests/test_provider_rates.py tests/test_rpc_model.py tests/test_acp_server.py tests/test_acp_methods.py tests/test_acp_config_options.py tests/test_agent_loop_lazy_provider.py tests/test_lazy_provider.py tests/test_token_wise_pricing.py tests/test_rpc_session_init_bundle.py -q --tb=short passed (602 tests); uv run ty check raven/providers/rates.py and git diff --check github/main...HEAD also passed. An initial run in the newly created base environment failed during autouse setup because the optional workspace package raven_everos was absent; after the declared dev extra was synced, the three modified suites passed 358/358 and the broader run above passed.

@LivXue

LivXue commented Oct 11, 2026

Copy link
Copy Markdown
Member Author

No blockers in the reviewed scope.

Scope: I reviewed the full PR diff (4 changed files), including tests and documentation, and traced affected callers and contracts at 41734946f8ff. This review covers all changed areas, not only raven/agent.

Checked both context-window and output-ceiling entry points, loaded-table aliases, SDK lookup refusal when allow_fetch=False, cached OpenRouter fallback, fetch-enabled compatibility, model-picker and binding callers, lazy providers, and ACP session/config-option paths. The no-fetch flag reaches both LiteLLM import policy and SDK metadata policy.

Verification (native Windows, Python 3.12, isolated source snapshots; local JUnit-output options omitted below):

  • PR head: uv run --no-sync pytest -o addopts= tests/test_provider_rates.py tests/test_rpc_model.py tests/test_acp_server.py tests/test_acp_methods.py tests/test_acp_config_options.py tests/test_agent_loop_lazy_provider.py tests/test_lazy_provider.py tests/test_token_wise_pricing.py tests/test_rpc_session_init_bundle.py -q --tb=short -> 602 passed.

The local combined tree with #908 also passed uv run --no-sync --with truststore==0.10.4 pytest -o addopts= tests/integration/test_acp_stdio_smoke.py -q --tb=short (11 tests). No new finding was identified. The full repository suite was not run.

git diff --check over the complete merge-base-to-head range passed. No new inline finding is being filed.

@LivXue

LivXue commented Oct 11, 2026

Copy link
Copy Markdown
Member Author

Windows compatibility review: no new issue found in the inspected scope.

Reviewed head 41734946f8ffa579e6fd1a0117a3774d34ccd500 against base 612696546702411e63d22f4b87d742d5e78b146b: all 4 changed files, provider metadata/cached-alias lookup callers, model-picker paths and ACP session handling. Checked cache-only lookup, permitted-fetch controls and frame-loop responses, including the EOF regression.

Native Windows / Python 3.13.12, detached temporary checkout with PYTHONUTF8=1, UV_NO_SYNC=1 and isolated user/Raven homes:

uv run pytest tests/test_provider_rates.py tests/test_rpc_model.py tests/test_acp_server.py -n 0 -x --tb=short --basetemp "$reviewRunDir/pr909/tests"

Result: 358 passed. The ACP frame-loop test uses in-memory streams; it does not validate native inherited stdin or the separate Windows stdin repair. No live provider/Ollama service or full suite was run. This scoped result does not establish full Windows compatibility.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants