Skip to content

feat(harness): make aforge the default provider across Python, Go, and TypeScript - #905

Merged
AbirAbbas merged 11 commits into
mainfrom
agent/aforge-do-harness
Aug 17, 2026
Merged

feat(harness): make aforge the default provider across Python, Go, and TypeScript#905
AbirAbbas merged 11 commits into
mainfrom
agent/aforge-do-harness

Conversation

@AbirAbbas

@AbirAbbas AbirAbbas commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

What

Makes AForge AgentField's default coding harness across all three SDKs, and adds the
AForge provider itself (Python, Go, TypeScript) with the aforge exec --json contract.

A harness call with nothing configured now works:

result = await app.harness("Fix the failing test in tests/test_auth.py", schema=Report)
const result = await app.harness(task, { schema });
result, err := agent.Harness(ctx, task, schema, &dest, harness.Options{})

Previously each of those raised "No harness provider specified". Picking Claude Code,
Codex, Gemini CLI, or OpenCode is now an override rather than a prerequisite — same
loop code, different worker.

Why

The harness was the one AgentField primitive that could not be used out of the box: every
call needed a provider name plus a third-party CLI installed and authenticated first.
AForge ships alongside af, so the default path costs the user one env var
(OPENROUTER_API_KEY) and nothing else. Orchestrating someone else's harness stays a
first-class, documented story — it just stops being the price of admission.

The model default moved for the same reason. It was "sonnet", a Claude-specific id
that was silently wrong for every other provider. It is now empty, meaning "the provider's
own default"; claude-code carries sonnet internally so explicit claude-code users see
no change.

Provider precedence

Order Source
1 Explicit value — .harness(provider=…), HarnessConfig(provider=…), Options.Provider, { provider }
2 AGENTFIELD_HARNESS_PROVIDER env var
3 Default — aforge

Implemented once per SDK and applied at both the config-materialisation and the
runner-resolution points, so a runner built with no config at all follows the same chain.

Validation contract

Behaviours this PR must exhibit, in caller-observable terms:

  1. A harness call with no provider configured anywhere runs aforge — it does not raise.
  2. With AGENTFIELD_HARNESS_PROVIDER=<name> set and no explicit provider, that provider runs.
  3. An explicit provider always wins over the env var and over an agent-level default.
  4. Blank / whitespace-only provider values (explicit or env) are treated as unset and fall
    through to the next source.
  5. A genuinely unknown provider name still fails loudly, with the supported list in the message.
  6. HarnessConfig built with no arguments is a complete, valid config in all three SDKs.
  7. An unset model means the provider's own default. claude-code with no model still sends
    sonnet, so no existing claude-code caller changes behaviour.
  8. The AForge adapter invokes aforge exec --json -w <root> --timeout <outer-5s> --context-fill 60 --completion-reserve 65536, delivers the task on stdin, and never puts
    the prompt on argv. AGENTFIELD_AFORGE_COMMAND=do selects the routed workflow instead.
  9. All three adapters locate the binary via AFORGE_BIN or PATH and exec it directly —
    no download, no unpacking, no build step.

Every item above has a test. Existing tests were changed only where the documented contract
changed (the three "provider is required" assertions and the two model == "sonnet"
assertions); each was rewritten to assert the new behaviour rather than deleted.

Verification

Gates run locally, matching .github/workflows/sdk-python.yml, sdk-go.yml,
sdk-typescript.yml and the sdk surfaces of coverage.yml:

Gate Command (from the workflow file) Result
sdk-python.yml lint ruff check . with the pinned ruff==0.15.22 clean
sdk-python.yml tests ./scripts/run_pytest.sh (full suite) 1973 passed, 4 skipped, 38 deselected
sdk-go.yml tidy go mod tidy + git diff --exit-code go.mod go.sum no drift
sdk-go.yml build go build ./... clean
coverage.yml (sdk-go) go test ./... -count=1 all 8 packages ok (agent, ai, client, did, harness, inputs, triggers, types)
sdk-typescript.yml install npm ci on Node 20 clean, lockfile in sync
sdk-typescript.yml lint npm run lint (tsc --noEmit) clean
build npm run build (tsup + dts) clean
sdk-typescript.yml tests npm test 849 passed across 82 files
Go formatting gofmt -l ./harness ./agent no touched file listed (the 7 files it does list are unformatted on main too)
readme-links.yml python3 scripts/check-utm-links.py passed — 45 tracked links, manifest in sync

Manual confirmation of items 8 and 9 of the contract: the --timeout / --context-fill /
--completion-reserve flags are present in all three adapters
(sdk/python/agentfield/harness/providers/aforge.py, sdk/go/harness/aforge.go,
sdk/typescript/src/harness/providers/aforge.ts), and none of the three contains any
archive/decompression path — each resolves AFORGE_BIN or PATH and execs.

CI

Green on this head: Python SDK CI (3.10 / 3.11 / 3.12 + both websockets matrices), Go SDK CI,
TypeScript SDK CI, Coverage Summary (all five surfaces + the gate), Functional Tests
(local + postgres), Performance Check, README Links, CodeQL for actions/go/python.

CodeQL / Analyze (javascript-typescript) is red for an unrelated reason: it died in
Set up job during a GitHub Actions incident that also took out five other jobs on this PR
(codeload returning 429/503 while downloading actions/setup-go, plus GraphQL 503s). Every
other affected job passed on re-run; that one is a Default-Setup code-scanning run, which the
API refuses to retry, so it will clear on the next push to this branch.

Still draft — un-draft conditions

  1. The aforge installer PR must merge first — "feat(install): provision the aforge
    harness binary alongside af". Until it does, aforge is not on a user's PATH and the
    new default would fail for anyone who has not built it themselves. That PR also adds
    af aforge ensure, which this PR's docs and the Python install hint now point at.
  2. Sibling PR adds the aforge row to af harness doctor — this PR deliberately does
    not touch control-plane/internal/cli/harness_doctor.go to keep that change conflict-free.

Relationship to #913 (Pi and OMP harnesses)

#913 currently lands "OMP as the default". That conflicts with the product decision here:
aforge is the default, and pi/omp are optional providers alongside claude-code, codex,
gemini, and opencode. #913 should drop its default-provider change and keep the two new
providers; the precedence chain in this PR (explicit > AGENTFIELD_HARNESS_PROVIDER >
aforge) is where a user selects them.

Whichever of the two rebases second owns these 24 overlapping files:

README.md
docs/design/harness-v2-design.md
docs/harness-providers.md
sdk/go/agent/harness.go
sdk/go/agent/harness_test.go
sdk/go/harness/factory.go
sdk/go/harness/provider.go
sdk/go/harness/runner.go
sdk/go/harness/runner_invariant_test.go
sdk/go/harness/runner_test.go
sdk/python/agentfield/agent.py
sdk/python/agentfield/harness/_availability.py
sdk/python/agentfield/harness/_runner.py
sdk/python/agentfield/types.py
sdk/python/tests/test_harness_provider_availability.py
sdk/python/tests/test_harness_runner.py
sdk/python/tests/test_harness_types.py
sdk/python/tests/test_types.py
sdk/typescript/src/harness/cli.ts
sdk/typescript/src/harness/providers/factory.ts
sdk/typescript/src/harness/providers/index.ts
sdk/typescript/src/harness/runner.ts
sdk/typescript/src/harness/types.ts
sdk/typescript/tests/harness_runner.test.ts

The three hot spots are the provider-name registry (SUPPORTED_PROVIDERS /
ResolveProviderName / the factory switch), the runner's provider-resolution line, and the
provider table in docs/harness-providers.md. Everything else is additive on both sides.

Follow-ups

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Performance

SDK Memory Δ Latency Δ Tests Status
Python 9.0 KB - 0.31 µs -11%
Go 181 B -35% 0.74 µs -26%
TS 408 B +17% 1.73 µs -14%

Regression detected:

  • TypeScript memory: 350 B → 408 B (+17%)

@github-actions

github-actions Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

📊 Coverage gate

Thresholds from .coverage-gate.toml: per-surface ≥ 84%, aggregate ≥ 85%, max per-surface regression ≤ 1.0 pp, max aggregate regression ≤ 0.50 pp.

Surface Current Baseline Δ
control-plane 87.20% 87.40% ↓ -0.20 pp 🟡
sdk-go 92.80% 92.00% ↑ +0.80 pp 🟢
sdk-python 94.20% 93.73% ↑ +0.47 pp 🟢
sdk-typescript 91.28% 90.42% ↑ +0.86 pp 🟢
web-ui 84.76% 84.79% ↓ -0.03 pp 🟡
aggregate 85.66% 85.75% ↓ -0.09 pp 🟡

✅ Gate passed

No surface regressed past the allowed threshold and the aggregate stayed above the floor.

@github-actions

github-actions Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

📐 Patch coverage gate

Threshold: 80% on lines this PR touches vs origin/main (from .coverage-gate.toml:thresholds.min_patch).

Surface Touched lines Patch coverage Status
control-plane 0 ➖ no changes
sdk-go 292 90.00%
sdk-python 0 ➖ no changes
sdk-typescript 140 94.00%
web-ui 0 ➖ no changes

✅ Patch gate passed

Every surface whose lines were touched by this PR has patch coverage at or above the threshold.

@AbirAbbas AbirAbbas changed the title feat(harness): add Aforge parity across Python, Go, and TypeScript feat(harness): add AForge parity across Python, Go, and TypeScript Aug 15, 2026
AbirAbbas added a commit to Agent-Field/contract-af that referenced this pull request Aug 17, 2026
Drop the git-SHA pin. agentfield 0.1.129 is on PyPI and already ships the
Python aforge provider (harness/providers/aforge.py, which runs
`aforge exec --json -w <root>`) and HarnessConfig.aforge_bin, so the
default path this repo now takes is served by a real release. It also
means the image build no longer needs a git checkout to install the SDK.

Two things stay inert until Agent-Field/agentfield#905 is released:
AGENTFIELD_AFORGE_COMMAND is forwarded to the harness subprocess but read
by nobody, and the 0.1.129 provider passes no --timeout, so aforge's own
15-minute wall applies. Bump both pins to the #905 release when it exists.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@AbirAbbas
AbirAbbas force-pushed the agent/aforge-do-harness branch from bfd3442 to d4be7bf Compare August 17, 2026 15:29
@AbirAbbas AbirAbbas changed the title feat(harness): add AForge parity across Python, Go, and TypeScript feat(harness): make aforge the default provider across Python, Go, and TypeScript Aug 17, 2026
@AbirAbbas

Copy link
Copy Markdown
Contributor Author

recheck

@AbirAbbas AbirAbbas closed this Aug 17, 2026
@AbirAbbas AbirAbbas reopened this Aug 17, 2026
@AbirAbbas
AbirAbbas marked this pull request as ready for review August 17, 2026 17:57
@AbirAbbas
AbirAbbas requested a review from a team as a code owner August 17, 2026 17:57
AbirAbbas and others added 9 commits August 17, 2026 13:57
`app.harness("...")` with nothing configured now runs AForge, AgentField's
native coding harness, instead of raising "No harness provider specified".

Provider precedence is explicit value > AGENTFIELD_HARNESS_PROVIDER >
"aforge", implemented once in harness/_defaults.py and applied both where
HarnessConfig materialises its default and where the runner resolves options
(so a runner built without any config follows the same chain).

HarnessConfig.model stops defaulting to "sonnet" — that was Claude-specific
and wrong for every other provider. The default is now empty, meaning "use
the provider's own default", and the claude-code provider carries "sonnet"
internally so explicit claude-code users see no change.

Also drops the private-repo build instruction from the aforge install hint;
the binary ships alongside `af` and `af aforge ensure` (re)installs it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Runner.Run no longer errors on an empty Options.Provider. BuildProvider and
Run both route the name through ResolveProviderName, which applies the same
precedence as the Python SDK: explicit value > AGENTFIELD_HARNESS_PROVIDER >
DefaultProvider ("aforge"). The resolved name is written back onto the
options so error messages and provider construction see the real provider.

Model stays empty by default and means "the provider's own default"; the
Harness doc example no longer suggests a Claude-specific "sonnet".

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Rewrites the harness-providers lead so the zero-setup path is the headline:
AForge is the default, `af aforge ensure` installs it alongside `af`, and
picking Claude Code / Codex / Gemini CLI / OpenCode is an override of one
option rather than a prerequisite. Documents the provider precedence chain
(explicit > AGENTFIELD_HARNESS_PROVIDER > aforge) and that an unset model
means the provider's own default.

Drops the "go build -o aforge ./cmd/aforge" instruction — that repo is not
public and is no longer how anyone gets the binary. Adds the grok row
(Python SDK only) so the install table matches the supported provider set.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The design doc still asserted that HarnessConfig.provider has no implicit
default and that a call without one raises. Both are false now: provider
resolves through explicit > AGENTFIELD_HARNESS_PROVIDER > "aforge", and
model defaults to the provider's own rather than "sonnet".

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
AbirAbbas and others added 2 commits August 17, 2026 13:58
HarnessRunner.run no longer throws when nothing sets a provider. Both the
runner and buildProvider route the name through resolveProviderName, which
applies the same precedence as the Python and Go SDKs: explicit value >
AGENTFIELD_HARNESS_PROVIDER > DEFAULT_HARNESS_PROVIDER ("aforge"). The
resolved name is written back onto the options so providers and error
messages see the real provider.

HarnessConfig.provider becomes optional to match, so `{}` is a complete
config.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Go and TypeScript adapters told anyone hitting a missing binary to
"Build it from https://github.com/Agent-Field/aforge-v2" — a repo they
cannot open. AForge ships with `af`, so the message now names
`af aforge ensure` and the AFORGE_BIN escape hatch, matching the Python
install hint.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@AbirAbbas
AbirAbbas force-pushed the agent/aforge-do-harness branch from d4be7bf to dbb19b2 Compare August 17, 2026 18:04
@AbirAbbas
AbirAbbas merged commit 6b8c9be into main Aug 17, 2026
31 of 32 checks passed
@AbirAbbas
AbirAbbas deleted the agent/aforge-do-harness branch August 17, 2026 18:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant