Skip to content

feat: warn the agent at 80% of a run budget before hard stop - #4179

Open
salignatmoandal wants to merge 3 commits into
docker:mainfrom
salignatmoandal:feat/soft-budget-warn
Open

salignatmoandal wants to merge 3 commits into
docker:mainfrom
salignatmoandal:feat/soft-budget-warn

Conversation

@salignatmoandal

@salignatmoandal salignatmoandal commented Sep 5, 2026

Copy link
Copy Markdown

Summary

The budget already existed. The agent never saw it until the run was killed.

  • Warn once at 80% of max_cost / max_tokens / max_time, then keep going.
  • On the next turn the model gets a system message so it can cheapen tools, summarize, or finish instead of slamming into the hard stop.
  • The budget_exceeded JSON contract and kill-switch at 100% are unchanged.

Why this is interesting

enforceBudget was a fuse, not a policy. The runtime already counted cost, tokens, and time (record / snapshot / per-agent spend), but the model only learned the ceiling after iterationStop. From the user's point of view that looks like a crash: Execution stopped after reaching budget.max_cost.

That is the EcoAgent-Bench gap: completing under a budget is not the same as choosing economical actions. A kill-switch with no feedback trains the agent to spend until the wall.

This PR turns the counters into a signal inside the loop — a change in agency policy, not an extra log line.

Before After
Budget = kill-switch Budget = signal + kill-switch
Agent is blind until death Agent can still adapt
Counters unused by the model Counters feed the next prompt

What that buys docker-agent:

  1. Fewer runs cut mid-task (code, MCP, delegation).
  2. Less waste: the agent can stop repeating tools or finish sooner.
  3. A reusable primitive: the runtime can now talk to the model from a metric. Soft budget today; later, model routing or denying expensive tools.

What it does not claim:

  • It does not pick cheaper tools by itself — it gives the LLM the information.
  • It does not add YAML (80% is an internal constant, no schema bump).
  • It does not relax the ceiling. Ignore the warning and the 100% stop still fires.

Behavior

At the existing turn-boundary check in RunStream:

  1. If any ceiling is hit → same as today (budget_exceeded + assistant stop message + iterationStop).
  2. Else if a limit is in [80%, 100%) and that tracker has not warned for that limit yet:
    • emit the existing Warning event (TUI / JSON stream)
    • append a system message the model will see on the next GetMessages call
  3. Each tracker warns once per limit (cost, then tokens, then time — same priority as exceeded()).
  4. Unpriced spend still does not invent a cost warning (the existing unpriced Warning remains).

Test plan

  • go test ./pkg/runtime -count=1 -run 'TestBudget|TestNilBudget|TestEnforceBudget|TestRecordBudget'
  • go test ./pkg/runtime -count=1 -run 'TestBudgetExceededEventJSONContract|TestEnforceBudgetWarnsOnceThenStillHardStops'
  • task test
  • task lint

@salignatmoandal
salignatmoandal requested a review from a team as a code owner September 5, 2026 10:28
@aheritier aheritier added area/runtime Runtime engine, agent loop execution, tool dispatch, loop detection kind/feat PR adds a new feature (maps to feat:). Use on PRs only. status/needs-signed-commits Some commits in the PR are signed with a valid SSH/GPG key labels Sep 5, 2026
@aheritier

Copy link
Copy Markdown
Collaborator

👋 Some commits in this PR are not signed and verified by GitHub. Please sign your commits with a GPG or SSH key registered in your GitHub account, then force-push.

Commits that are not verified: 8858c9a

See GitHub's guide on signing commits for setup instructions. I've added status/needs-signed-commits; it will be removed automatically once every commit in this PR carries a valid GitHub-verified signature.

Give the model one chance to cheapen or finish before the existing
kill-switch fires, without changing the budget_exceeded contract.
@aheritier aheritier removed the status/needs-signed-commits Some commits in the PR are signed with a valid SSH/GPG key label Sep 5, 2026

@aheritier aheritier left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Prompt-cache concern: this persists the 80%-budget warning as a system message in session history (pkg/runtime/budget.go:506). That rewrites the cached system/instructions prefix for Anthropic, Bedrock, and ChatGPT/Codex; with num_history_items, trimming can also relocate it ahead of the conversation. The first call after warning may therefore require a full-context cache write at exactly the point this feature is meant to limit cost.

Please route this through the existing transient extraSystemMessages mechanism instead of persisting it, using stable wording to preserve cache reuse. This also avoids stale/accumulating warnings after session resume and loss of the warning after compaction.

Separate blocker: CI lint is currently red (budget_wiring_test.go:245, use assert.Len).

@dgageot @rumpl — requesting your eyes on the prompt-caching implications, as this is a nuanced provider-level behaviour.

@aheritier aheritier added the status/needs-rebase PR has merge conflicts or is out of date with main label Sep 14, 2026
@aheritier
aheritier marked this pull request as draft September 14, 2026 17:09
@aheritier aheritier removed the status/needs-rebase PR has merge conflicts or is out of date with main label Sep 14, 2026
@salignatmoandal
salignatmoandal marked this pull request as ready for review September 16, 2026 09:29
@salignatmoandal
salignatmoandal marked this pull request as draft September 16, 2026 09:30
Keep the model-visible approaching signal out of session history so it
does not bust the prompt-cache prefix, survive resume as a stale system
message, or vanish after compaction.
@salignatmoandal
salignatmoandal marked this pull request as ready for review September 16, 2026 13:14
@aheritier
aheritier requested review from dgageot and rumpl September 16, 2026 19:03
@aheritier aheritier added the status/needs-rebase PR has merge conflicts or is out of date with main label Sep 18, 2026
@aheritier

Copy link
Copy Markdown
Collaborator

👋 This PR has merge conflicts with the base branch. Please rebase or merge the latest base branch and resolve them. I've moved it to draft and added status/needs-rebase; it'll be picked back up automatically once the conflicts are cleared.

Resolve the budget.go conflict between the 80% approaching warning and
main's shared-budget / unpriced-spend-warning changes.
@aheritier aheritier removed the status/needs-rebase PR has merge conflicts or is out of date with main label Sep 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/runtime Runtime engine, agent loop execution, tool dispatch, loop detection kind/feat PR adds a new feature (maps to feat:). Use on PRs only.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants