Skip to content

Demonstration: the Antigravity CLI (agy) as a harness base - #695

Draft
richard-epsilla wants to merge 17 commits into
mainfrom
feat/antigravity-cli
Draft

richard-epsilla wants to merge 17 commits into
mainfrom
feat/antigravity-cli

Conversation

@richard-epsilla

@richard-epsilla richard-epsilla commented Nov 14, 2026 •

Copy link
Copy Markdown
Collaborator

Draft, for demonstration only. Not for merge, and not for redistribution.
The Antigravity CLI (agy) is Google's software under Google's terms. It is not ours to redistribute, so it is in no HarnessRouter release or published image. If you'd like to try it, check out this branch and build an image locally. The CLI is then downloaded from Google at first start, pinned by version and checksum, only when HR_BACKENDS names it.

This branch shows Google's Antigravity CLI running as an opt-in harness base inside HarnessRouter's own sandbox, the way Codex and Claude Code do. Each session runs under its own user, in its own workspace, and HarnessRouter drives it through the usual task, session, streaming and file APIs.

How it works

  • Driven over agy acp, the CLI's Agent Client Protocol server, one session per task. A follow-up resumes the same conversation and can change model or level.
  • The key never reaches the agent. The CLI talks to the Gemini API through the runner's local relay, and the agent only ever holds a placeholder. The instance's Google AI Studio key is added outside the session. Verified live: no key in the session's environment, its command lines or its files.
  • Quiet by default: auto-update, telemetry and crash reporting are switched off on every start.
  • Models: gemini-4-hydrogen, gemini-4-carbon, gemini-4-argon, gemini-3.8-flash (default), gemini-3.6-flash and gemini-3.1-pro-preview. Each model's thinking levels are taken from the CLI.
  • Tools:
    • The harness's MCP servers reach the CLI through the runner's relay, and skills and AGENTS.md instructions are read.
    • Shell can be switched off for real.
    • The CLI's other tool switches are asked of the agent rather than enforced, and are labelled that way.
  • Operator gate: run this base only on an instance where everyone who runs tasks is trusted. Set HR_AGY_TRUSTED_USERS=1 to confirm; until then the base is listed as unavailable, with that sentence.

Tests:

  • New runner tests, with a fake ACP agent and the CLI's recorded event shapes.
  • Catalog and gate tests.
  • The install pin.
  • Key-absence checks.
  • The whole gateway and console suites pass, and the runner suite fails only where main fails too.

Retest after fixes (2026-11-16): fully conformant

Head b5eb7df3, rebased on current main. Built the way a member would build it: check out the branch and run docker build ., with no build arguments. Run on a self-hosted instance with a Google AI Studio key.

UHP conformance (--class full, suite 2026.10.19.post9): 110 of 110, CONFORMANT at class full. Nothing failed and nothing was skipped. The step-budget check (T-12) was run 10 more times on its own and passed all 10.

Every model: 33 of 33 (6 models × every thinking level).

What changed to get here:

End-to-end results (2026-11-14)

Run on a self-hosted instance built locally from this branch (0315aa72 for the matrix and conformance, 58247793 for the logo), CLI 1.5.8, with a Google AI Studio key. Every turn below was served by that key, read back from each session.

Every model, through the public API: 33 of 33 passed, in 6 minutes

Model Levels offered File written, read back via the files API Follow-up recalls it Shell runs (6*7 = 42) Other levels answer
gemini-4-argon low, medium, high pass pass pass low, high pass
gemini-4-carbon low, medium, high pass pass pass low, high pass
gemini-4-hydrogen low, medium, high, xhigh pass pass pass low, medium, xhigh pass
gemini-3.8-flash (default) low, medium, high pass pass pass low, high pass
gemini-3.6-flash low, medium, high pass pass pass low, high pass
gemini-3.1-pro-preview low, high pass pass pass low pass

The first three turns per model ran at its middle level. Also on the default model:

  • A harness's instructions are followed.
  • Asked to show its key-shaped environment variables, the agent has no key to show.
  • With shell switched off, the agent can't run a command.

In the console, as a user (1440, 1024, 768 and 390 wide):

  • Antigravity's own mark (the one antigravity.google serves) shows beside "Antigravity CLI" in the harness list, on the harness card, in the page header and in the agent stack.
  • New harness: "Antigravity CLI" is in the harness list, and its model list is the six above.
  • First task: saved with one sentence; it wrote status.md, showed it as a file and summarised it (17 s).
  • Follow-up: answered from the file (5 s). Its steps show the memory recall, the file it created and the file it read.

UHP conformance (--class full): 92 of 110 passed, 14 failed, 4 skipped. The highest class fully passed is extended. None of the failures is this base's:

  • 13 failures are the Memories chapter (ME-01 to ME-13): on this build a memory made with an API key can't be read back by that key, so the suite can't make one under it. The same happens on a Claude Code harness and on the released 0.41.16. It's being looked at separately.
  • EN-07 fails and F-09 skips because of the suite's own setup: they run on a harness the suite makes on the first base the server lists (here Codex), with the model under test, and Codex doesn't serve a Gemini model. EN-07 passes on the same instance when run with Claude Code.
  • T-13 and T-14 skip: the question-for-the-person tool (ask_user) is offered on Claude Code only.
  • T-12 skips: the step-budget tool in the fixture was never called.

🤖 Generated with Claude Code

@vercel

vercel Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
unified-harness-protocol Ready Ready Preview Nov 16, 2026 10:01pm UTC

Request Review

Comment thread runner/server.py Dismissed
Comment thread runner/server.py Dismissed
Comment thread runner/server.py Dismissed
Comment thread runner/server.py Dismissed
Comment thread runner/server.py Dismissed
Comment thread runner/server.py Dismissed
Comment thread runner/server.py Dismissed
Comment thread runner/server.py Dismissed
Comment thread runner/server.py Dismissed
Comment thread runner/server.py Dismissed
richard-epsilla and others added 15 commits November 16, 2026 11:16
acp_turn.Acp takes an optional permit(params) callback. When it returns False, the client answers
session/request_permission with the agent's own reject option, or "cancelled" when the agent offers
none. Without the callback every request is allowed, as before, so the dcode and fast-agent drivers
are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…DS names it

agy is opt-in: not in the default HR_BACKENDS. install_agy fetches the 1.5.8 release archive from
Google's release bucket (the archive Google's own installer fetches) instead of piping its install
script, checks it against a sha256 committed here for each architecture (computed 2026-11-13; each
agreed with the sha512 in the release server's manifest), unpacks only the `antigravity` binary as
the base's install uid, installs it as $TOOLS/bin/agy, and proves it with `agy --version`.

HR_AGY_VERSION overrides the version, with HR_AGY_BUILD (the build number in the release path) and
HR_AGY_SHA256; an override without a digest installs as "unverified" and the start says so, as for
the other pinned bases. The base's install uid is appended at the end of INSTALL_UID_ORDER, so no
other base's uid moves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
agy runs on a Google AI Studio key through the runner's loopback relay, the same as gemini-cli.
settings.json names the Gemini API (modelProvider "gemini"). The CLI gets GEMINI_API_KEY set to the
relay's placeholder and GOOGLE_GEMINI_BASE_URL set to the relay. The relay adds the real key in
x-goog-api-key on the way to generativelanguage.googleapis.com. The key never enters the agent's
environment, argv or files.

Measured on agy 1.5.8 against Google's API, with a relay that held the key:
- `agy -p` posts usage data to play.googleapis.com and nothing switches that off. `agy acp
  --disable-telemetry` posts none. So the turn is `agy acp --disable-telemetry`, driven by
  runner/agy_driver.py on acp_turn.
- AGY_CLI_DISABLE_AUTO_UPDATE=true stops the self-update ("1" does not). The runner sets it on every
  turn, and AGY_/ANTIGRAVITY_ names are reserved from a harness's own variables.
- The session's model id carries the thinking level (gemini-3.8-flash-high). The catalog's model
  and the turn's level map to it, and the relay does not write a level of its own.
- The prompt's usage counts thinking inside outputTokens. It maps to output_tokens with
  reasoning_tokens as its subset, the cache read split out, for that prompt only.
- session/load in a second process continues the conversation. The CLI's own store
  (conversations/<id>.db) says whether it is there.
- MCP servers go in ~/.gemini/config/mcp_config.json, written every turn (command/args/env,
  serverUrl/headers). An SSE server goes through the stdio bridge. A switched-off MCP tool goes in
  disabledTools.
- The ACP server reads no settings file. It asks the client before a shell command and nothing
  else, so a switched-off shell is answered no (hard). The other tools are asked off in the
  instructions.

The MCP config, the CLI's credential store, logs, crash dumps and updater state stay out of
checkpoints. The conversation travels. Unpacked caches are rebuilt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…led it

- Wiring: ("google", "agy") only, because the CLI calls a helper model of its own by Google's id.
  In _NATIVE_ONLY_BACKENDS, so a brokered sandbox is refused. In owner trust the runner's relay
  holds the key.
- Catalog: gemini-4-argon, gemini-3.8-flash (default), gemini-3.6-flash and gemini-3.1-pro-preview,
  the CLI's models that Google's table serves as themselves. Still qualifying. The bases document
  carries each model's levels as this CLI takes them (model_levels).
- Base entry: the CLI's tools, with the shell labelled hard and the rest labelled instruction.
- notes: what a person should know (key stays on the instance, no self-update, no usage data, the
  helper model on the same key).
- Console: an Antigravity CLI entry (generic glyph, AGENTS.md), offered only where the instance
  installed it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The self-hosting guide says how to enable it and what to know about it. The support-matrix notes
record what was measured on 1.5.8: auth, telemetry, models, the helper model, usage, resume, MCP
and tool switches. NOTICE names its terms (Google's own; downloaded from Google at first start,
never redistributed). The README counts it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Antigravity CLI base now offers gemini-4-carbon (low to high) and
gemini-4-hydrogen (low to xhigh) beside its four other models, under the
ids and levels the CLI's own model list gives them. A Google AI Studio
connection serves the two for this base only: the shared Google model
table, and every other base on a Google key, is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every start of the Antigravity CLI, each turn and the install's version
check, now has its crash reporting switched off
(ANTIGRAVITY_SENTRY_SAMPLE_RATE=0), beside its usage reporting and its
self-update. The base's notes and the self-hosting guide say so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Antigravity CLI base now waits on one switch on the instance. Until
HR_AGY_TRUSTED_USERS=1 is set, the bases document reads it "unavailable"
with the reason, the console and New setup do not offer it, and a task on
a harness built on it is refused before anything runs, in the same
sentence: "Run this base only on an instance where everyone who runs
tasks is trusted; set HR_AGY_TRUSTED_USERS=1 to confirm." The base's
notes and the self-hosting guide carry the same words. A base that is not
installed still says so first.

The release checks that skip a base they cannot run skip an unavailable
one too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The mark antigravity.google serves (assets/image/antigravity-logo.png), scaled to 192 px like the
other marks: a gradient on transparency that reads on light and dark.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ading its definition is not a step

agy routes every MCP tool through one tool, call_mcp_tool, whose input wraps the tool's arguments
beside the server and tool names. The call was already named mcp__<server>__<tool>; its input is now
that tool's own arguments (the inner `Arguments`, parsed when it is a JSON string, the whole input
when it is neither), as every other base hands them on.

Before its first call to an MCP tool, agy reads the tool's definition with its read_file, from
<HOME>/.gemini/antigravity-cli/mcp/<server>/<tool>.json. The runner's step budget counted that read
as a step, so a one-step budget stopped the turn as the tool's call went out and the tool was never
answered. The driver now marks the first read of each such file `not_a_step` and the step counter
passes over a call carrying the mark; the read stays in the record, and a second read of the same
file counts as any other call.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…equests)

The runner serves its ask_user tool (Tasks §3.2, #476) to agy as it does to Claude Code. A turn that
takes input requests gets the tool as one more stdio MCP server in the CLI's mcp_config.json, named
harnessrouter, so the driver names its call mcp__harnessrouter__ask_user and hands on the tool's own
arguments. The turn's record arms the watcher for agy too: at the tool's result the process is
stopped and the turn ends incomplete with reason input_required and the question beside it. The
answer is the next turn's prompt on the same conversation, which the driver loads over ACP; a secret
answer reaches the CLI's environment for that turn.

The gateway lists agy among the bases served the tool, so the runner's job carries input_requests
and GET /v1/bases says takesInputRequests for it. Codex and the rest still say they do not.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The self-hosting guide's list of what to know about the base says it can end a task on a question
and continue from the answer, and that a secret answer reaches the agent for that turn only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… is not a step either

Some turns agy lists the folder its MCP tool definitions are kept in (`ls -la` of it, then of a
server's folder) before it calls a tool, and that spent a one-step budget before the tool was called
(UHP T-12 skipped one run in five). A plain ls or cat whose every path is inside that folder, with
nothing a shell would act on, is the same lookup: the first of each is marked not_a_step.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the definitions folder

A path with `..`, or one that passes through a symbolic link below the CLI's MCP definitions folder,
is never inside it, and only plain listing flags (-l -a -A -1 -h) count as a listing; cat takes none.
So a link planted in the folder, a recursive listing or a numbered cat spends a step like any call.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Measured on 1.5.8: some turns agy looks its MCP tools up with `find <folder>/ -maxdepth 3
2>/dev/null` (UHP T-12 skipped one run in five). find on paths inside the folder, with only
-maxdepth/-mindepth N and -type f|d and at most an exact trailing 2>/dev/null, counts as the lookup;
-exec, -name, any other redirect or a path outside still spend a step.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e uses

agy names its arguments its own way (CommandLine, toolSummary, Path, Content, Target,
Replacement), so the step list showed "Ran a command" with an empty "$", reads and writes
with no file, and edits with no change. The record now gives them the names it gives
every other base's (command, description, file_path, content, old_string, new_string);
a name the input already uses is never overwritten, and an MCP call's input stays the
tool's own arguments.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Google refuses a call whose deadline is under ten seconds (400 "Manually set deadline 8s
is too short. Minimum allowed deadline is 10s."). The Antigravity CLI asks 8 s in
x-server-timeout for its helper model's call, so that call was refused once a step. On
Google's own endpoint the relay now raises a shorter deadline to the minimum; a value that
is not a number, and any other provider's header, is left as sent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch was successfully deployed

1 active deployment
Preview — 2242b529 Deployed Nov 16, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants