Skip to content

feat: make AForge exec the default harness - #129

Merged
AbirAbbas merged 9 commits into
mainfrom
agent/aforge-exec-default
Aug 18, 2026
Merged

feat: make AForge exec the default harness#129
AbirAbbas merged 9 commits into
mainfrom
agent/aforge-exec-default

Conversation

@AbirAbbas

@AbirAbbas AbirAbbas commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

Status update — agentfield v0.1.130 is released; this PR is ready

Everything below the line was written against the 0.1.129 pin and is kept for history. What is true on the current head:

  • Pins bumped: agentfield>=0.1.130 in pyproject.toml and the Dockerfile.
  • "Known no-ops" resolved: 0.1.130 reads AGENTFIELD_AFORGE_COMMAND (exec default, do opt-in), passes --timeout/--turns; AFORGE_BIN already worked. Manifest / README / .env.example wording updated accordingly.
  • Live verification (real OpenRouter key, default provider): image built with default args from agentfield.ai + PyPI (aforge v0.1.0, agentfield 0.1.130); node registered on a fresh agentfield/control-plane:latest; sec-af.run_config_scanner on a 3-file throwaway repo returned 4 secrets + 6 misconfigs with correct file:line, 21 s, sub-cent; .aforge/trace/1.trace.log written; live argv aforge exec --json -w … --timeout 1795 --turns 12 --context-fill 60 --completion-reserve 65536 --model … --plan-model …. Code default with all provider env unset = aforge; SEC_AF_PROVIDER=claude-code / HARNESS_PROVIDER=opencode flip it. Tests: 105 passed; the 2 failures (tests/test_scoring.py) are identical on main.
  • Un-draft preconditions met (site live, 0.1.130 on PyPI).

Summary

Makes AForge the default SEC-AF harness and runs it through aforge exec, and makes that default actually installable:

  • Default flipHARNESS_PROVIDER / AIIntegrationConfig.provider / AuditConfig.provider all default to aforge. OpenCode stays installed in the image and remains a configuration-only rollback (HARNESS_PROVIDER=opencode).
  • AForge is fetched, not pulled — the previous revision copied the binary out of ghcr.io/agent-field/aforge-v2:chat-v2-exec, a private tag that does not exist. The Dockerfile now has a fetch stage that downloads the published gzipped release for the build's TARGETARCH, gunzips it, and verifies the decompressed SHA-256 against the release checksums.txt before installing it to /usr/local/bin/aforge. A bad download fails the build.
  • SDK re-pinned to PyPIagentfield @ git+…@bfd34426…#subdirectory=sdk/pythonagentfield>=0.1.129 in both pyproject.toml and the Dockerfile. 0.1.129 ships the aforge harness provider and the aforge_bin field on HarnessConfig, which is everything this repo uses. [tool.hatch.metadata] allow-direct-references is dropped along with the direct reference.
  • Honest docsAGENTFIELD_AFORGE_COMMAND was documented as a subcommand selector with do as an override. It is not one today (see below); it is now labelled reserved wherever it appears.

Build arguments

Build arg Default Purpose
AFORGE_BASE_URL https://agentfield.ai/downloads/aforge Download host root
AFORGE_VERSION v0.1.0 Released AForge version to install

Both are plumbed through docker-compose.yml so a mirror can be pointed at without editing files.

Known no-ops against agentfield 0.1.129

Verified by reading the installed 0.1.129 wheel and by running the resolution inside the built image:

  • AGENTFIELD_AFORGE_COMMAND is a no-op. agentfield/harness/providers/aforge.py in 0.1.129 hardcodes [bin, "exec", "--json", "-w", root]. The variable is forwarded into the harness subprocess environment and ignored. It becomes meaningful only if/when the SDK exposes a command selector (feat(harness): make aforge the default provider across Python, Go, and TypeScript agentfield#905).
  • SEC_AF_AFORGE_BIN / AFORGE_BIN do work today — contrary to the earlier assumption. HarnessConfig.aforge_bin exists in 0.1.129 and build_provider passes it to AforgeProvider(bin_path=…); SEC-AF reads both env names and forwards the value. Confirmed in-container: SEC_AF_AFORGE_BIN=/usr/local/bin/aforgeaforge_bin: /usr/local/bin/aforge.
  • No --timeout is passed to aforge. 0.1.129's provider wraps the subprocess in AGENTFIELD_HARNESS_TIMEOUT_SECONDS (default 1800s) but passes no --timeout flag, so aforge's own 15-minute wall is the effective per-call limit.

When agentfield#905 is released, bump the floor in exactly two places:

  • pyproject.toml:14 "agentfield>=0.1.129",
  • Dockerfile:54 "agentfield>=0.1.129" \

(A floor bump is required, not merely permitted: Docker layer caching keys off the literal constraint string, so leaving >=0.1.129 will keep restoring the cached layer and shipping 0.1.129.)

There is no Go node in this repo (go.mod count: 0), so there is no sdk/go pseudo-version to carry.

Validation contract

Observable behaviours this change must exhibit:

  1. docker build . succeeds with no registry credentials — only the public download host is required.
  2. The aforge binary installed in the image is byte-identical to the published release: its SHA-256 equals the aforge-linux-<arch> line in checksums.txt.
  3. A corrupted or tampered download fails the build rather than shipping.
  4. Both linux/amd64 and linux/arm64 resolve to the correct asset and verify.
  5. aforge is on PATH inside the image and aforge --help exits 0.
  6. A real aforge exec --json run inside the image returns a single JSON envelope and exits 0.
  7. With no provider env set, the container resolves to the AForge provider and finds the bundled binary.
  8. HARNESS_PROVIDER=opencode still resolves to the OpenCode provider (rollback intact).
  9. SEC_AF_AFORGE_BIN overrides the binary path.
  10. python -m sec_af.app imports cleanly against the PyPI-pinned SDK.
  11. The repo's test suite gains no new failures.

How verified

This repo has no CI, so local verification is the gate. Everything below was run against the committed tree; the build used a staging mirror via --build-arg AFORGE_BASE_URL=… (production agentfield.ai/downloads/aforge/… currently returns 404 — see "What un-drafts this").

# Evidence
1 docker build --build-arg AFORGE_BASE_URL=<mirror> -t sec-af-aforge-test .writing image sha256:0a3a629043aa… DONE
2 Release line: 61217a18135f392d50c337d971fa5b228f436f8df144800b37fbb340020c79f5 aforge-linux-amd64. In-image: sha256sum /usr/local/bin/aforge61217a18135f392d50c337d971fa5b228f436f8df144800b37fbb340020c79f5 — exact match
3 Negative test: same pipeline with printf "tamper" >> aforgesha256sum: WARNING: 1 computed checksum did NOT match / aforge: FAILED / rc=1
4 docker build --target aforge --no-cache-filter aforge --build-arg TARGETARCH=arm64+ arch=arm64aforge: OKDONE 1.5s
5 docker run --rm --entrypoint aforge sec-af-aforge-test --help → usage text, exit 0; command -v aforge/usr/local/bin/aforge
6 echo "Reply with exactly OK" | aforge exec --json -w /tmp/w --timeout 120 --budget 20000 --turns 3 in-container → {"text":"OK","stop":"done","usage":{"calls":1,"prompt_tokens":3548,"completion_tokens":2,"cost":0.000279…},"artifacts":[],"turns":1,"elapsed_ms":3352}, exit 0
7 In-container resolution dump: {"agentfield_version":"0.1.129","HARNESS_PROVIDER_env":"aforge","resolved_provider":"aforge","AuditConfig_default_provider":"aforge","which_aforge":"/usr/local/bin/aforge","provider_class":"AforgeProvider"}
8 -e HARNESS_PROVIDER=opencodeprovider: opencode -> OpenCodeProvider
9 -e SEC_AF_AFORGE_BIN=/usr/local/bin/aforgeaforge_bin: /usr/local/bin/aforge
10 python -c "import sec_af.app" in-container → app import OK; harness provider = aforge | aforge_bin = aforge
11 pytest -q2 failed, 105 passed; same 2 failures on origin/main (2 failed, 103 passed)

Also: docker compose config resolves with the AForge defaults and honours AFORGE_BASE_URL / AFORGE_VERSION overrides.

Pre-existing red, not caused by this PR

origin/main (9a72858) already fails these two, both in tests/test_scoring.py and both from the same root cause (reachability defaults to the externally-reachable multiplier when the tag set is empty):

  • test_compute_exploitability_score_defaults_reachability_when_missing
  • test_reachability_multipliers_and_default_behavior[tags5-2.5]

They are unrelated to the harness change and are deliberately not fixed here#128 fixes them and should keep the credit. Everything else is green: 105 passed on this branch vs 103 on main (the +2 are the new tests added here).

ruff check . reports 113 findings on this branch and 113 on origin/main — identical, all pre-existing. mypy src reports 48 errors on both. Neither is enforced by CI today and neither is touched here.

Follow-ups

  • Bump the two pinned lines above once agentfield#905 lands in a PyPI release, so AGENTFIELD_AFORGE_COMMAND stops being decorative and an explicit --timeout can be passed.
  • Consider surfacing AGENTFIELD_HARNESS_TIMEOUT_SECONDS as a documented SEC-AF knob; today the effective per-call ceiling is aforge's internal 15-minute wall, which is shorter than the SDK's 1800s subprocess timeout.

What un-drafts this

Two external preconditions, both outside this repo:

  1. https://agentfield.ai/downloads/aforge/v0.1.0/{aforge-linux-amd64.gz,aforge-linux-arm64.gz,checksums.txt} must serve 200 (they return 404 today — the host is live and serves the older build-<sha> coordinate, but the v0.1.0 assets have not been published to it yet). Once they do, a plain docker build . with no build-arg overrides is the real test — re-run it, then mark ready.
  2. Optional but preferred: agentfield#905 released to PyPI, so the two pinned lines can be bumped in the same PR and the reserved-variable caveats can be deleted.

Precondition 1 alone is enough to un-draft; precondition 2 can follow as its own bump.

🤖 Generated with Claude Code

AbirAbbas and others added 7 commits August 17, 2026 10:57
The image previously pulled AForge out of a `ghcr.io/agent-field/aforge-v2`
container image, which made the build depend on a private registry tag that
does not exist yet.

Replace that with a fetch stage that downloads the published, gzipped release
binary for the build's TARGETARCH, decompresses it, and verifies the
*decompressed* SHA-256 against the release `checksums.txt` before it is copied
into the runtime image. A tampered or truncated download fails the build
instead of shipping.

`AFORGE_BASE_URL` and `AFORGE_VERSION` stay overridable as build args (and are
plumbed through docker-compose) so CI or a local mirror can point elsewhere
without editing the Dockerfile.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The branch depended on an unmerged agentfield commit via a direct git
reference, so `pip install sec-af` only worked with VCS access and
`allow-direct-references` turned on.

agentfield 0.1.129 is on PyPI and already ships the `aforge` harness provider
plus the `aforge_bin` field on `HarnessConfig`, which is everything this repo
uses today. Pin the floor there and drop the now-unnecessary
`[tool.hatch.metadata] allow-direct-references`.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two behaviours had no coverage:

- `AuditConfig` (the orchestrator's runtime config) defaults to the `aforge`
  provider, not just `AIIntegrationConfig`.
- The pinned agentfield floor actually accepts the harness settings `app.py`
  sends it. `aforge_bin` only exists on `HarnessConfig` from 0.1.129 onward, so
  lowering the floor now fails a test instead of failing at agent startup.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ND reserved

The docs claimed `AGENTFIELD_AFORGE_COMMAND` selects the AForge subcommand and
that `do` is an available override. It does not: agentfield 0.1.129's aforge
provider always runs `aforge exec`, so the variable is forwarded to the harness
environment and ignored. Label it reserved everywhere it is documented rather
than promise behaviour that does not exist yet.

Also document `AFORGE_BASE_URL` / `AFORGE_VERSION` and state that the image
installs a checksum-verified AForge at /usr/local/bin/aforge.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
aforge-v2 cut its first semver release (v0.1.0), so the AFORGE_VERSION
default moves off the build-<sha> coordinate onto the tag. Bumping the
string is what busts the fetch layer's cache, so this is what actually
pulls the released binary instead of restoring the stale one.

Moves all four places the coordinate is written: the Dockerfile ARG, the
docker-compose build arg fallback, the .env.example hint, and the README
build-arg table. The README's override example loses its build-<sha>
placeholder shape at the same time, since versions are tags now. The
AgentField SDK pin is deliberately left alone — it bumps on its own
release.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
agentfield 0.1.130 is the first release that ships the aforge harness
provider as the default, so the runtime pin moves up with it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…eld>=0.1.130

The "reserved / no-op" wording was written against 0.1.129, whose aforge
provider hardcoded `exec`. 0.1.130 reads AGENTFIELD_AFORGE_COMMAND (default
`exec`, `do` opts into the routed workflow), so describe the real behaviour.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@AbirAbbas
AbirAbbas marked this pull request as ready for review August 18, 2026 02:28
@AbirAbbas
AbirAbbas merged commit bb5d482 into main Aug 18, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant