Skip to content

feat(optimize): mcp-deferral-gaps finding family (part 1 of #614)#630

Closed
AVSRPA1KR wants to merge 1 commit into
getagentseal:mainfrom
AVSRPA1KR:submit/mcp-deferral-gaps
Closed

feat(optimize): mcp-deferral-gaps finding family (part 1 of #614)#630
AVSRPA1KR wants to merge 1 commit into
getagentseal:mainfrom
AVSRPA1KR:submit/mcp-deferral-gaps

Conversation

@AVSRPA1KR

Copy link
Copy Markdown
Contributor

Implements part 1 (detection) of the deferral-coverage design, #614.

Adds a read-only mcp-deferral-gaps finding family to optimize — three detectors, no file mutation anywhere in this PR:

  • mcp-deferral-off — sessions carrying MCP tool-def overhead with zero ToolSearch invocations and no deferred_tools_delta inventory across the window. Cause attribution, each with its own message and fix: stale ENABLE_TOOL_SEARCH=false (settings env, any scope, or shell profile), non-first-party ANTHROPIC_BASE_URL (reported as unknown proxy — never assumes capability), Vertex AI, all observed Claude Code versions predating default-on tool search (v2.1.7), or a generic fallback.
  • mcp-alwaysload-hygienealwaysLoad-pinned servers whose observed call rate is below 1 call / 5 sessions. Notes the 5s startup-blocking cost.
  • mcp-defer-thresholdENABLE_TOOL_SEARCH=auto:N overrides whose threshold the estimated def volume never reaches; recommends the tightest auto:N that would defer.

Detection only — no FindingApply payloads, no act changes, no file mutation. Zero new output for users without deferral gaps (deferral-active evidence suppresses the findings). 32 fixture-based unit tests; config readers take an injectable homeDir for hermetic tests.

Live-docs discrepancies noted in the commit body (changelog records default-on at v2.1.7 not first-ship; per-tool anthropic/alwaysLoad lives in server-served _meta, so server-level only).

Refs #614

Add three read-only detectors for MCP tool-def deferral coverage gaps:

- mcp-deferral-off: sessions carrying MCP tool-def overhead with zero
  ToolSearch invocations and no deferred_tools_delta inventory across
  the window. Cause attribution, each with its own message and fix:
  stale ENABLE_TOOL_SEARCH=false (settings env, any scope, or shell
  profile), non-first-party ANTHROPIC_BASE_URL (reported as unknown
  proxy — never assumes capability), Vertex AI config, all observed
  Claude Code versions predating default-on tool search (v2.1.7), or
  a generic deferral-appears-inactive fallback.
- mcp-alwaysload-hygiene: alwaysLoad-pinned servers whose observed
  call rate is below 1 call per 5 sessions (named constant). Notes the
  5s startup-blocking cost of alwaysLoad in the explanation.
- mcp-defer-threshold: ENABLE_TOOL_SEARCH=auto/auto:N overrides whose
  threshold the estimated def volume never reaches, so tools load
  upfront; recommends the tightest auto:N that would defer.

Detection only: no FindingApply payloads, no act-layer changes, no
file mutation. Users without deferral gaps see zero new output —
deferral-active evidence (ToolSearch calls or inventory) suppresses
the findings, and each detector gates on conservative named
thresholds. Config readers take an injectable homeDir (PlanContext
style) for hermetic tests.

Discrepancies vs the design notes, codebase/live-docs win:
- The Claude Code changelog records default-on (2.1.7), not first
  ship, so the version cause uses the default-on boundary.
- Per-tool "anthropic/alwaysLoad" lives in server-served tool _meta,
  not static config; server-level only, limitation documented.
- Live docs confirm auto default threshold 10% of context window and
  the alwaysLoad v2.1.121+ requirement.

Refs getagentseal#614
@iamtoruk

Copy link
Copy Markdown
Member

Merged as part of #631, whose branch carried this PR's commits as its base per the stacking you described. The detection family, all 51 dedicated tests, and the finding text shipped exactly as designed here; the squash credits you as author. Closing this one so the pair has a single merge history. The empirical note from verification, since this PR asked the right question: across 120 real deferral-active sessions the deferred-tool inventory is emitted at session start regardless of usage, so the detector's no-signal heuristic holds with no minimum-usage floor needed. Thank you for the cleanest big first contribution this repo has seen, and looking forward to the part 3 design issue.

@iamtoruk iamtoruk closed this Jul 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants