Add the pi extension that will replace the CLI - #82
Merged
Conversation
pi owns the agent loop, session, and UI; the extension contributes the tools, the browser they run against, and the provider wiring native surfaces need. Adapted from an earlier spike onto current APIs. Notably it uses neither attach() nor the harness: pi owns the model collection, so the extension takes the two pieces that are not pi-shaped -- the catalog compiler and CuaExecutionResources -- and applies headers and payload transforms through pi's own hooks. That is evidence for the framework-neutral split the rename is built on. Selectors now cover every provider-native surface rather than only Anthropic's computer tool, and /cua-tools with no argument lists what the current model can take, with the catalog compiler's own reason for anything it cannot.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.
Bugbot Autofix prepared fixes for both issues found in the latest run.
- ✅ Fixed: Native CUA transports never attach
- The extension now registers CUA-wrapped Anthropic/OpenAI/Google providers and feeds each stream call the active compiled catalog model plus
cuaIncomingToolPlan, enabling native OpenAI, Google, and Anthropic fallback dispatch.
- The extension now registers CUA-wrapped Anthropic/OpenAI/Google providers and feeds each stream call the active compiled catalog model plus
- ✅ Fixed: Header hook skips reconcile
before_provider_headersnow reconciles first, avoids unsafe recompilation, and removes stale Anthropic beta requirements when compatibility drops so model switches no longer throw or leak stale header tokens.
Or push these changes by commenting:
@cursor push e7e610e4e9
Preview (e7e610e4e9)
diff --git a/packages/pi-extension/src/index.ts b/packages/pi-extension/src/index.ts
--- a/packages/pi-extension/src/index.ts
+++ b/packages/pi-extension/src/index.ts
@@ -1,4 +1,5 @@
import { fileURLToPath } from "node:url";
+import type { Api, Model, Provider } from "@earendil-works/pi-ai";
import type { ExtensionAPI, ExtensionContext } from "@earendil-works/pi-coding-agent";
import { createCuaModels, type CuaToolSpec } from "@onkernel/cua-ai";
import {
@@ -35,6 +36,7 @@
let sessionActive = false;
let runtime: CuaBrowserRuntime | undefined;
let allSpecs = new Map<string, CuaToolSpec>();
+ let compiledCatalog: ReturnType<typeof compileSpecs> | undefined;
function configureDeclarations(): void {
allSpecs = new Map(allSelectableSpecs(selection.coordinates).map((spec) => [spec.name, spec]));
@@ -81,6 +83,27 @@
};
pi.appendEntry(CONFIG_ENTRY, state);
}
+ function catalogForModel(model: Model<Api>): ReturnType<typeof compileSpecs> | undefined {
+ if (!compiledCatalog) return undefined;
+ return compiledCatalog.model.provider === model.provider && compiledCatalog.model.id === model.id ? compiledCatalog : undefined;
+ }
+ function withIncomingPlan(options: unknown, catalog: ReturnType<typeof compileSpecs> | undefined): unknown {
+ if (!catalog) return options;
+ return { ...(isRecord(options) ? options : {}), cuaIncomingToolPlan: catalog.incoming };
+ }
+ function withCatalogProvider(provider: Provider): Provider {
+ return {
+ ...provider,
+ stream(model, context, options) {
+ const catalog = catalogForModel(model);
+ return provider.stream((catalog?.model ?? model) as never, context, withIncomingPlan(options, catalog) as never);
+ },
+ streamSimple(model, context, options) {
+ const catalog = catalogForModel(model);
+ return provider.streamSimple((catalog?.model ?? model) as never, context, withIncomingPlan(options, catalog) as never);
+ },
+ };
+ }
function reconcile(ctx: ExtensionContext, activateInitial = false): void {
const specs = currentSpecs();
const current = pi.getActiveTools();
@@ -92,17 +115,18 @@
!initialized || activateInitial || forcedInactive ? selectedNames : priorCua.filter((name) => selectedNames.includes(name));
try {
if (desired.length && !ctx.model) throw new Error("no pi model is selected");
- if (desired.length && ctx.model) {
- compileSpecs(
+ compiledCatalog = desired.length && ctx.model
+ ? compileSpecs(
ctx.model,
specs.filter((spec) => desired.includes(spec.name)),
- );
- }
+ )
+ : undefined;
compatibilityError = undefined;
forcedInactive = false;
activeNames = new Set(desired);
pi.setActiveTools([...current.filter((name) => !allSpecs.has(name)), ...desired]);
} catch (error) {
+ compiledCatalog = undefined;
compatibilityError = error instanceof Error ? error.message : String(error);
forcedInactive = true;
activeNames = new Set();
@@ -121,9 +145,15 @@
// CUA's Anthropic wrapper retries an inaccessible native browser beta through
// an equivalent function-tool transport, which pi's builtin provider cannot do.
- const anthropic = createCuaModels().getProvider("anthropic");
- if (!anthropic) throw new Error("CUA Anthropic provider is unavailable");
- pi.registerProvider(anthropic);
+ const cuaModels = createCuaModels();
+ for (const id of ["anthropic", "openai", "google"] as const) {
+ const provider = cuaModels.getProvider(id);
+ if (!provider) {
+ if (id === "anthropic") throw new Error("CUA Anthropic provider is unavailable");
+ continue;
+ }
+ pi.registerProvider(withCatalogProvider(provider));
+ }
pi.registerCommand("cua", {
description: "Show CUA tool and browser status",
@@ -170,18 +200,22 @@
pi.on("model_select", (_event, ctx) => reconcile(ctx));
pi.on("before_agent_start", (_event, ctx) => reconcile(ctx));
pi.on("before_provider_headers", (event, ctx) => {
- if (!activeNames.size || compatibilityError || !ctx.model) return;
- Object.assign(event.headers, compileSpecs(ctx.model, activeSpecs()).headers.merge(event.headers));
+ reconcile(ctx);
+ if (!activeNames.size || compatibilityError || !compiledCatalog) {
+ withoutCuaHeaderRequirements(event.headers, currentSpecs());
+ return;
+ }
+ Object.assign(event.headers, compiledCatalog.headers.merge(event.headers));
});
pi.on("before_provider_request", async (event, ctx) => {
reconcile(ctx);
- if (!activeNames.size || compatibilityError || !ctx.model) {
+ if (!activeNames.size || compatibilityError || !compiledCatalog) {
// setActiveTools() normally removes CUA declarations before serialization.
// This hook is the final pre-wire guard for a model switch that invalidates
// a catalog after pi has already built a payload for the turn.
return currentSpecs().length ? withoutCuaToolSchemas(event.payload, allSpecs) : undefined;
}
- return compileSpecs(ctx.model, activeSpecs()).payload.apply(event.payload, ctx.model);
+ return compiledCatalog.payload.apply(event.payload, compiledCatalog.model);
});
pi.on("tool_call", (event) => {
if (!allSpecs.has(event.toolName)) return;
@@ -238,6 +272,26 @@
return seconds;
}
+function withoutCuaHeaderRequirements(headers: Record<string, string | null | undefined>, specs: readonly CuaToolSpec[]): void {
+ const requiredBetas = new Set(
+ specs.flatMap((spec) => spec.providerBinding?.kind === "anthropic-native" ? [spec.providerBinding.beta] : []),
+ );
+ if (!requiredBetas.size) return;
+ const headerName = Object.keys(headers).find((name) => name.toLowerCase() === "anthropic-beta");
+ if (!headerName) return;
+ const current = headers[headerName];
+ if (typeof current !== "string") {
+ delete headers[headerName];
+ return;
+ }
+ const kept = current
+ .split(",")
+ .map((token) => token.trim())
+ .filter((token) => token.length > 0 && !requiredBetas.has(token));
+ if (kept.length > 0) headers[headerName] = [...new Set(kept)].join(",");
+ else delete headers[headerName];
+}
+
function withoutCuaToolSchemas(payload: unknown, cuaSpecs: ReadonlyMap<string, CuaToolSpec>): unknown {
if (!isRecord(payload) || !Array.isArray(payload.tools)) return payload;
const tools: unknown[] = [];
diff --git a/packages/pi-extension/test/extension.test.ts b/packages/pi-extension/test/extension.test.ts
--- a/packages/pi-extension/test/extension.test.ts
+++ b/packages/pi-extension/test/extension.test.ts
@@ -126,7 +126,7 @@
"cua-profile-save-changes": false,
});
extension(pi.api);
- expect(pi.providers.map((provider) => provider.id)).toContain("anthropic");
+ expect(pi.providers.map((provider) => provider.id)).toEqual(expect.arrayContaining(["anthropic", "openai", "google"]));
await getHandler(pi, "session_start")({}, anthropicCtx);
expect(pi.active).toContain("computer");
@@ -139,6 +139,26 @@
expect(transformed).toEqual({ tools: [expect.objectContaining({ name: "computer", type: "computer_20260701" })] });
});
+ it("reconciles headers on model switches and removes stale CUA betas", async () => {
+ const pi = makePi({
+ "cua-tools": "anthropic-computer",
+ "cua-coordinates": "pixels",
+ "cua-browser-timeout": "300",
+ "cua-profile-save-changes": false,
+ });
+ extension(pi.api);
+ await getHandler(pi, "session_start")({}, anthropicCtx);
+ expect(pi.active).toContain("computer");
+
+ const headers: Record<string, string> = {};
+ await getHandler(pi, "before_provider_headers")({ headers }, anthropicCtx);
+ expect(headers["anthropic-beta"]).toContain("computer-use-2026-07-01");
+
+ expect(() => getHandler(pi, "before_provider_headers")({ headers }, ctx)).not.toThrow();
+ expect(headers["anthropic-beta"]).toBeUndefined();
+ expect(pi.active).toEqual(["bash"]);
+ });
+
it("keeps the browser out of the request path, and blocks execution after shutdown", async () => {
const get = vi.spyOn(CuaBrowserRuntime.prototype, "get");
try {You can send follow-ups to the cloud agent here.
Reviewed by Cursor Bugbot for commit ee643c4. Configure here.
openai-computer and google-browser declare requiresApi, which only takes effect on the compiled model -- pi resolves and streams its own registry model, so those transports never engaged. Anthropic's native browser fallback reads cuaIncomingToolPlan, which pi builds. All three were present and inert. Also reconcile and tolerate a failed compile in before_provider_headers, so a model switch that invalidates the catalog omits CUA's headers instead of throwing and leaving a stale provider beta.
pi resolves and streams its own registry model, but the transport a native surface needs is derived onto the compiled model. The registered provider now swaps in catalog.model -- the resolved model with only api replaced -- and passes the incoming native-call plan. pi's resolved credential rides along in options.apiKey. That restores anthropic-browser, openai-computer, and google-browser rather than leaving them out. A test drives the registered provider and asserts the wire gets openai-cua-computer with the plan; without the wrapper it gets openai-responses and no plan.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Phase 5a: the pi extension that phase 5b will delete the CLI in favor of. Adapted from #73 onto post-4b APIs. +1,715 lines, and it lands beside the CLI — the replacement works before the thing it replaces goes, same discipline as 4a before 4b.
What it is
pi owns the agent loop, session, TUI, print/RPC modes, and model selection. The extension contributes only what is Kernel's: the tools, the browser they execute against, and the provider wiring that native surfaces need.
pi install ./packages/pi-extension pi -p --provider anthropic --model claude-opus-5 --cua-tools anthropic-computer "open example.com"The finding worth reading
The extension uses neither
attach()nor the harness. pi owns the model collection, socompiled.models, the retry wrapper, and the harness behaviors have nothing to attach to. What it needs is the catalog compiler andCuaExecutionResources— and it applies headers and payload transforms through pi's ownbefore_provider_headers/before_provider_requesthooks instead.That is not a gap. It is the first real evidence for the framework-neutral split the rename is built on: a consumer that does not use pi's harness reaches for exactly the pieces measured as neutral, and for nothing that is pi-shaped. It also means 5d has a second consumer to enforce the boundary against, not just a lint rule.
What changed versus the spike
The spike predates phases 0–4b. Its design survived — it never touched the agent classes — but four things moved underneath it:
compileCuaToolCatalog'sviewportoption, removed in phase 1. Since main's Anthropic native computer takes no display dimensions and only20260701, the entire viewport plumbing disappears: noDEFAULT_VIEWPORT, no viewport threading, and no "provision the browser early just to learn its size" special case in the request path.anthropic-computer). Now all four:anthropic-computer,anthropic-browser,openai-computer,google-browser./cua-toolswith no argument now lists every selector for the current model and shows the catalog compiler's own reason for any it cannot take — phase 3's rule delivered here rather than restated.An empty
/cua-toolsused to be ambiguous between "list" and "clear". It lists;noneclears.Two tests that changed meaning
removes stale incompatible CUA schemas from the provider payloadused an unknown provider (not-a-cua-model) as its incompatible model. That no longer compiles as incompatible — phase 2 removed the allowlist, so an unlisted model passes through and a plain function tool has no provider binding to violate. The test was encoding pre-phase-2 behavior. It now uses a genuinely incompatible pairing — Anthropic's native computer against an OpenAI model — and I added its complement asserting that an unlisted model keeps an ordinary function tool active, which is the phase-2 behavior nothing was pinning.Also added: compiling declarations, generating headers, and transforming a payload must not provision a browser. Only executing a tool does. That was implicit in the spike and is now asserted.
Verification
By exit status, not grep: typecheck 0, cua-ai 94, cua-agent 281, cua-cli 143, extension 23, all three builds 0. CI gains a
pi-extension-unitjob.The extension's suite includes an end-to-end run that spawns real
piin print and RPC modes against a fake provider and a fake Kernel server, asserting the tool executes, the payload carries the declaration, and the owned browser is deleted on shutdown. That is the "does this actually work" check, and it passes locally in 2.8s.Deliberately not here
No release workflow and no publish path for
@onkernel/cua-pi-extension. This package merges into@onkernel/loopin 5c, and the first publish under the new name has to be manual anyway (npm binds trusted publishers to a repository + workflow filename pair, and a brand-new package name has none). Adding release automation under a name we are about to retire would be throwaway.The spike also carried an unrelated cua-ai change — Anthropic's documented native computer versions
20250124and20251124alongside early-access20260701. That is worth having and is independent of the extension, so it is not in this PR.Note
Medium Risk
New surface area touches provider streaming, request payloads, and Kernel browser lifecycle (API keys, provision/delete), though behavior is covered by unit and spawned-
pie2e tests.Overview
Introduces
@onkernel/cua-pi-extension, a new workspace package that plugs Kernel computer-use tools into pi’s existing agent loop (no second model loop). Users select tools with--cua-tools, manage them via/cuaand/cua-tools, and optionally attach or lazily provision a Kernel browser.The extension deliberately does not use
attach()or the harness—it wirescompileCuaToolCatalogandCuaExecutionResourcesthrough pi hooks (before_provider_headers,before_provider_request) and wraps Anthropic/OpenAI/Google streaming so native surfaces get the compiled model transport and incoming native-call plan. Tool/model compatibility is enforced by compiling the selection; incompatible tools deactivate with the compiler’s reason. Owned browsers are created on first tool execution and deleted on shutdown.Monorepo/CI:
packages/pi-extensionis added to workspaces andtsconfigreferences;.github/workflows/ci.ymlgains api-extension-unitjob. README anddocs/architecture.mddocument the new package and boundaries.Reviewed by Cursor Bugbot for commit 083e37e. Bugbot is set up for automated code reviews on this repo. Configure here.