Classify Kernel API failures on tool errors

[masnwilliams]

Summary

Every failed MCP tool call currently records $mcp_error_type = "Error", so a stale session id, an org hitting its concurrency limit, and a fault on our side are indistinguishable in analytics. This names them.

toolErrorResponse becomes throwToolError, which throws instead of returning an isError result. The thrown error's name carries the Kernel API status — KernelApiError404, KernelApiError429, KernelApiError502 — or a fixed name for transport failures that never got a response (KernelApiTimeout, KernelApiConnectionError, KernelApiAborted).

Why this works

The analytics SDK reads the error category from a thrown error's name. A returned isError result has no error object, so it coerces from the result text and always lands as a generic Error. That's why the existing data has exactly one error type.

Because throwToolError is the single funnel for caught API errors, the split falls out with no per-call-site logic:

• KernelApiError<status> — the Kernel API rejected the request.
• Error — the tool itself rejected the call (missing session_id, conflicting arguments, invalid config), which never reaches the API.

Agents see no change

The MCP SDK converts a thrown error back into the same isError text result, and the message string is byte-identical to what the helper produced before. Verified against the real API:

• manage_browsers get with an unknown session id → Error in manage_browsers (get): 404 browser session '...' not found, isError: true
• manage_browsers get with no session id → Error: session_id is required for get action., isError: true

Privacy

Status codes only. No message, parameters, or response are captured — confirmed on the captured events — and no $exception events are emitted. $mcp_error_type is already on the send allow-list, so nothing new is added to what leaves the server.

Verification

Ran the route locally against the production API and checked the captured events:

call	$mcp_error_type	duration	message / params / response captured
API 404 on unknown session	KernelApiError404	1983 ms	none
input guard, no session_id	Error	2 ms	none

tsc --noEmit and prettier --check pass on every touched file.

Tests

bun test (no new runner, bun is already the package manager) covering the classification table, message preservation, and what the client actually receives over an in-memory MCP transport. Wired into CI as a step after the type check -- this is the repo's first test, so the harness comes with it.

Writing the transport case found a bug in the first version of this PR. errorName fell back to error.constructor.name, and the SDK's error classes are bundled and minified in the production build (APIConnectionTimeoutError is emitted as fN), so a timeout would have been recorded as $mcp_error_type = "fN". Transport failures are now classified by instanceof and the last-resort fallback reads error.name, which can never be a mangled identifier. The status path was always safe -- it reads error.status, not a class name.

Follow-up, deliberately not here

The generic Error bucket still mixes several kinds of tool-side rejection (missing argument vs conflicting arguments vs invalid config). Naming those too is mechanical but touches ~95 call sites, so it belongs in its own change if the split turns out to be worth it.

---

Note

Medium Risk
Wide change to error handling on every MCP tool path; behavior for agents should be unchanged but analytics and throw semantics differ from returned errors.

Overview
Replaces toolErrorResponse with throwToolError, which throws a ToolCallError whose name encodes the failure kind (KernelApiError404, KernelApiError429, transport types via instanceof, etc.) so PostHog $mcp_error_type is no longer always generic Error. MCP clients still get the same isError text—the SDK maps the throw back to the prior shape.

All Kernel API tool catch blocks now call throwToolError instead of returning an error result. Input validation keeps errorResponse.

Adds bun test ( responses.test.ts ), bun-types, a test script, and a CI step after typecheck.

^{Reviewed by Cursor Bugbot for commit 4d70afc. Bugbot is set up for automated code reviews on this repo. Configure here.}

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
mcp	Ready	Preview	Aug 4, 2026 2:27pm

[dcruzeneil2]

Looks good to me. The new error classification makes the analytics much more useful while preserving the existing error messages agents receive and keeping sensitive data out of PostHog. It would be helpful to add regression tests for HTTP status classification, timeout errors, and the unchanged MCP response behavior in a follow-up, but this does not need to block the PR. Approved.

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	bun-types@​1.3.14

View full report

[masnwilliams]

Took the regression tests here rather than as a follow-up, and they earned it immediately.

bun test covers the status classification, timeout/transport errors, message preservation, and the unchanged MCP response behaviour (a real client over an in-memory transport, asserting isError and the exact text for both a thrown API failure and an input guard). Added as a CI step after the type check. No new runner — bun is already the package manager; the only new dependency is @types/bun.

The timeout case failed against what was on the branch. errorName fell back to error.constructor.name, and the SDK's error classes are bundled and minified in the production build — APIConnectionTimeoutError is emitted as class fN — so a timeout would have landed in PostHog as $mcp_error_type = "fN". Transport failures are now classified by instanceof into KernelApiTimeout / KernelApiConnectionError / KernelApiAborted, and the last-resort fallback reads error.name, which can never be a mangled identifier. The status path was never affected — it reads error.status.

Client-visible behaviour and messages are unchanged, which is now asserted rather than asserted-by-me.

[masnwilliams]

On Socket's flag of @types/bun — swapped it for bun-types, which is what it was pulling in anyway.

@types/bun is a DefinitelyTyped stub whose only content is a dependency on bun-types@1.3.14; bun-types is published by the Bun team out of oven-sh/bun. Depending on it directly drops a package from the tree and takes the types from the first-party source. Referenced with a /// <reference types="bun-types" /> in the test file rather than tsconfig's types array, since setting that array would disable the automatic @types/* includes the rest of the project relies on.

Runtime never needed it — bun test passes with no types installed at all; this is purely so tsc --noEmit can resolve bun:test. Only devDependencies changed, nothing ships.