Skip to content

fix: build session prompt for the provider/model that actually runs - #1210

Merged
SamSaffron merged 1 commit into
SamSaffron:mainfrom
sam-saffron-jarvis:feat/session-prompt-actual-provider
Oct 6, 2026
Merged

SamSaffron merged 1 commit into
SamSaffron:mainfrom
sam-saffron-jarvis:feat/session-prompt-actual-provider

Conversation

@sam-saffron-jarvis

Copy link
Copy Markdown
Contributor

Problem

The session system prompt is resolved from the agent's preferred provider/model, not the provider/model the session actually runs on. This affects {{provider}}, {{model}} and {{provider_model}}, and also the model gates ([[[claude-bin]]] …) in the user-level AGENTS.md.

Observed on Jarvis, an agent whose preference is chatgpt:gpt-6.1-sol:

Run Actually ran on Prompt was built for
Web session, model picker set to Claude CLI / opus claude-bin:opus chatgpt:opus
ask --agent jarvis --provider claude-bin:opus claude-bin:opus chatgpt:gpt-6.1-sol

The web case is a half-applied override. prepareRunnerAgent copies the agent and replaces Model with the requested model, but keeps the agent's Provider. ResolveSettingsInDir / resolveSessionPromptTools then template from agent.Provider + agent.Model. The provider the session really uses is resolved separately into cfg by applyProviderOverridesWithAgent, and prompt resolution never reads it. ask, serve startup and loop don't consult the provider flag at all.

Practical impact: a [[[claude-bin]]] block in ~/.config/term-llm/AGENTS.md never reaches Claude CLI sessions started from the web UI, and the model is told the wrong identity.

Fix

  • CLIFlags gains ActiveProvider / ActiveModel: the pair the caller has already resolved into cfg after the CLI, request, agent and config overrides. When ActiveProvider is set, both ResolveSettingsInDir and resolveSessionPromptTools use the pair in place of the agent's preference. The model always comes with its own provider, so a preferred model is never paired with another provider.
  • activeLLMFlags(cfg) returns cfg.DefaultProvider + activeModel(cfg).
  • These callers set the pair. Each one has already applied its overrides at that point:
    • the runner (web, Telegram, serve runtimes)
    • ask and both ask --resume prompt paths
    • serve startup
    • loop
  • loop resolved its prompt before applying provider overrides. Applying the overrides is now moved ahead of prompt resolution.
  • Chat TUI is unchanged: resolveChatRuntimeSystemContextWithConfig already sets the agent copy's provider and model to the ones the session runs on, and this PR applies that same idea to the other paths.

When no active pair is supplied, behaviour is unchanged (agent preference > config).

Tests

  • TestCmdRunnerPreparePromptUsesSelectedProviderAndModel: an agent that prefers preferred:preferred-model, prepared through the runner with these requests:
    • provider + model → selected:big. Before the fix this rendered preferred:big, the same shape as chatgpt:opus.
    • provider only → selected:selected-default. Before the fix: preferred:preferred-model.
    • provider:model flag → selected:big. Before the fix: preferred:preferred-model.
    • no selection → agent preference, unchanged.
  • TestResolveSettingsActiveLLMWinsOverAgentPreference: the active pair wins for templating, and a [[[claude-bin]]] user AGENTS.md block is included only when the active provider is claude-bin. Ungated content is always kept.
  • go test ./... passes under an isolated HOME/XDG per AGENTS.md. go build ./... passes.
  • End to end, ask --agent jarvis --provider debug "hi" with a throwaway XDG_DATA_HOME, reading the persisted system message:
    • current release: Current LLM: **chatgpt:gpt-6.1-sol**
    • this branch: Current LLM: **debug** (the debug provider has no configured model)

Not changed

From reading the code (not tested): when a web session already has selected prompt inputs, the runner reuses them (cli.inputs), so a mid-conversation model switch most likely keeps the prompt resolved when the session started. That is the existing refreshable-inputs design and is out of scope here. Sessions already created with a wrong prompt keep it.

The system prompt ({{provider}}, {{model}}, {{provider_model}} and
model-gated user AGENTS.md blocks) was resolved from the agent's
preferred provider/model, not the one the session runs on. The runner
overrode only the model, so a web session on claude-bin:opus for an agent
preferring chatgpt rendered "chatgpt:opus" and dropped [[[claude-bin]]]
gates. ask/serve/loop ignored --provider entirely.

Callers now report the provider/model they resolved into cfg via
CLIFlags.ActiveProvider/ActiveModel, which win over agent preferences.
loop applies provider overrides before resolving its prompt.
@SamSaffron
SamSaffron merged commit ee07cb8 into SamSaffron:main Oct 6, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants