Skip to content

feat(models): agent x model cross-breakdown (--by-agent)#710

Merged
iamtoruk merged 1 commit into
mainfrom
feat/708-models-by-agent
Jul 16, 2026
Merged

feat(models): agent x model cross-breakdown (--by-agent)#710
iamtoruk merged 1 commit into
mainfrom
feat/708-models-by-agent

Conversation

@iamtoruk

Copy link
Copy Markdown
Member

Problem

The dashboard shows Skills and Agents and By Model as separate panels, so orchestrator setups with agents pinned to different models cannot see which agent drives which model's spend. Closes #708.

Fix

codeburn models --by-agent: one row per (provider, model, agent), mirroring --by-task exactly across the flag, aggregation, and all four output formats (table, markdown, csv, json). The agent dimension comes from each subagent session's own agentType joined with its calls, which is the authoritative pairing; spawn markers on main sessions are deliberately not used. Ordinary main sessions and non-Claude providers bucket under main. --by-task and --by-agent are mutually exclusive with a clear error. The renderers gained a generic third-column concept rather than a copy of the task path.

Real output from a machine with a heavy orchestrator day (the exact question from the issue, answered):

Claude  Opus 4.8   main             691.7M  446.51
                   general-purpose   61.3M   50.20
Claude  Fable 5    main             203.2M  248.13
                   general-purpose   24.7M   55.21
Claude  Haiku 4.5  general-purpose   68.8K    0.03

The issue's secondary ask, exposing subagentType on per-record exports, is left for a follow-up to keep this diff on one surface.

Verification

  • 8 new tests: the model x agent cross (one agent on two models, two agents on one model, main, non-Claude), group ordering, all four renderers, and the mutual-exclusion error. Mutation-checked twice: forcing every bucket to main fails exactly the two aggregation tests; mislabeling the third column fails exactly the two renderer pins.
  • tsc clean; full suite green apart from the pre-existing app/electron/cli.test.ts fails on a fresh clone: dev-resolution tests require a built dist/cli.js #681 electron failures and the playwright-dependent sync e2e on this checkout.
  • aggregateModels callers without options (usage-aggregator) are behaviorally unchanged.

Orchestrator setups pin different agents to different models (planner on
Opus, implementers on Sonnet, scouts on Haiku). The dashboard shows
"Skills & Agents" and "By Model" as separate panels, so there was no way
to see which agent drove which model's spend, or catch an agent that
silently switched models. `codeburn models --by-agent` closes that gap:
one row per (provider, model, agent), mirroring the existing --by-task
pattern across the table, markdown, CSV, and JSON renderers.

The agent label comes from each Claude subagent transcript session's own
agentType crossed with that session's model calls, aggregated through the
same assistantCalls path --by-task uses (not modelBreakdown), so cache-tier
dedup, savings, and Codex-credit logic all carry over unchanged. This is
the authoritative pairing; main-session subagentTypes spawn markers are
deliberately not joined to models, since a spawn marker names an agent but
not the model that agent actually ran on.

Ordinary main sessions (no agentType) and every non-Claude provider have
no agent, so their spend buckets under the label "main" rather than being
dropped or fabricated into a false agent. --by-agent is mutually exclusive
with --by-task; passing both errors politely.

Exposing subagentType + per-record model on `codeburn export` (so users
can build custom crosses over the export) is a separate follow-up and is
intentionally out of scope here.
@iamtoruk
iamtoruk merged commit 6e66d34 into main Jul 16, 2026
3 checks passed

@ozymandiashh ozymandiashh left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed against a parallel implementation of the same issue that went through a three-model adversarial panel today, so these findings are battle-tested rather than speculative. The core design matches what we converged on independently: session-level attribution from the subagent transcript's own agentType, spawn markers deliberately unused (crossing per-call subagentTypes would bill the parent's Task-invocation tokens to the child agent). The generic third-column renderer is a nicer shape than mirroring the task path. Three findings:

  1. The bare main sentinel collides with a real agent named "main". agentType is free-form text from the meta file, not an enum: a user-defined subagent literally named main merges silently into the main-session bucket and the attribution the feature exists for is corrupted for exactly that agent. A non-slug sentinel (we used (main) with parentheses) cannot collide; worth a test pinning that a subagent named main stays distinct.

  2. The inherited --min-cost 0.01 default quietly hides sub-cent agent buckets. At task granularity that default is invisible; at agent granularity sub-cent buckets are the norm (scouts, one-shot helpers), so agents disappear from the audit without a trace. One README sentence recommending --min-cost 0 for a complete agent inventory saves the support question.

  3. For the deferred export follow-up: when subagentType/model land on sessions.csv, append them AFTER all legacy columns — inserting mid-row shifts positions for by-index CSV consumers. We have the export half already built and panel-reviewed (records.csv with one row per call, sessions.csv with appended fields, formula-injection-safe); happy to open it as the follow-up PR so it composes with this one.

iamtoruk pushed a commit that referenced this pull request Jul 16, 2026
…t follow-ups (#712)

Follow-ups to #710, completing issue #708's second ask plus the two review
notes:

- export: new records.csv with one row per call incl. subagentType and
  model; sessions.csv gains both fields APPENDED after all legacy columns
  so by-index consumers are unaffected; JSON fields additive; formula-
  injection escaping preserved
- models --by-agent: the main-session bucket sentinel becomes '(main)' -
  agentType is free-form text, so a real subagent literally named 'main'
  no longer merges into the main bucket (test-pinned across all formats)
- README: note that the --min-cost 0.01 default can hide sub-cent agent
  buckets; --min-cost 0 shows the complete agent inventory
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature request: agent × model cross-breakdown (e.g. codeburn agents --by-model)

2 participants