Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
c0d70b3
Add Pi and OMP harness providers
santoshkumarradha Aug 12, 2026
3b412a3
Make OMP the default harness provider
santoshkumarradha Aug 12, 2026
2ec6a8a
Improve Pi and OMP harness coverage
santoshkumarradha Aug 12, 2026
f89a419
Merge branch 'main' into agent/add-pi-omp-harnesses
santoshkumarradha Aug 13, 2026
973913e
address review comments on #913
santoshkumarradha Aug 24, 2026
be9b0c5
Merge origin/main into agent/add-pi-omp-harnesses: keep aforge as the…
AbirAbbas Aug 27, 2026
8bb9f08
harness: keep aforge as the default; document Pi and OMP as additiona…
AbirAbbas Aug 27, 2026
188feac
harness: resolve the provider at dispatch time so aforge stays the de…
AbirAbbas Aug 27, 2026
3ae5157
fix(ts-sdk): classify pi/omp failures with failureType and returnCode
AbirAbbas Aug 27, 2026
ba76a4c
fix(ts-sdk): honour projectDir as the working directory in every prov…
AbirAbbas Aug 27, 2026
944b388
fix(cli): probe pi/omp over stdin like the SDK adapters do
AbirAbbas Aug 27, 2026
bdbaf47
fix(harness): stop sending Pi an approval flag it rejects
AbirAbbas Aug 27, 2026
4cfaf13
test(python): clear AGENTFIELD_HARNESS_PROVIDER in the aforge-default…
AbirAbbas Aug 27, 2026
d944abb
fix(cli): stop the --provider help implying the flag defaults to aforge
AbirAbbas Aug 27, 2026
8c3332d
test(cli): drop the "OMP default" premise from a harness doctor test …
AbirAbbas Aug 27, 2026
58d1a77
docs(readme): list pi and omp in the harness provider-swap row
AbirAbbas Aug 27, 2026
a4d5cdc
fix(harness): stop reporting a recovered Pi/OMP turn as a failed run
AbirAbbas Aug 27, 2026
54dcfaa
fix(ts-sdk): derive the schema output root from the same ladder the p…
AbirAbbas Aug 27, 2026
ee542c9
test(ts-sdk): cover ClaudeCodeProvider's projectDir handling
AbirAbbas Aug 27, 2026
dfc6d74
fix(cli): make `af doctor` survey aforge, the default harness provider
AbirAbbas Aug 27, 2026
6247a63
fix(cli): stop `--probe` reporting a silently broken pi/omp install a…
AbirAbbas Aug 27, 2026
413d069
test(cli): guard `af doctor`'s aforge detection against a silent revert
AbirAbbas Aug 27, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -251,7 +251,7 @@ Two examples already run at this load. The [deep-research engine](https://agentf

- **[Reasoners & Skills](https://agentfield.ai/docs/build/building-blocks/reasoners?utm_source=github-readme&utm_campaign=github-readme&utm_id=github-readme-reasoners)** - `@app.reasoner()` for AI judgment, `@app.skill()` for deterministic code
- **[Structured AI](https://agentfield.ai/docs/reference/sdks/python?utm_source=github-readme&utm_campaign=github-readme&utm_id=github-readme-structured-ai)** - `app.ai(schema=MyModel)` → typed Pydantic/Zod output from any LLM
- **[Harness](https://agentfield.ai/docs/build/intelligence/harness?utm_source=github-readme&utm_campaign=github-readme&utm_id=github-readme-harness)** - `app.harness("Fix the bug")` dispatches multi-turn tasks to AForge, AgentField's own coding harness — no setup. Add `provider="claude-code"` (or `codex`, `gemini`, `opencode`) to orchestrate someone else's.
- **[Harness](https://agentfield.ai/docs/build/intelligence/harness?utm_source=github-readme&utm_campaign=github-readme&utm_id=github-readme-harness)** - `app.harness("Fix the bug")` dispatches multi-turn tasks to AForge, AgentField's own coding harness — no setup. Add `provider="claude-code"` (or `codex`, `gemini`, `opencode`, `pi`, `omp`) to orchestrate someone else's.
- **[Cross-Agent Calls](https://agentfield.ai/docs/build/coordination/cross-agent-calls?utm_source=github-readme&utm_campaign=github-readme&utm_id=github-readme-cross-agent-calls)** - `app.call("other-agent.func")` routes through the control plane with full tracing
- **[Discovery](https://agentfield.ai/docs/reference/sdks/python?utm_source=github-readme&utm_campaign=github-readme&utm_id=github-readme-discovery)** - `app.discover(tags=["ml*"])` finds agents and capabilities across the mesh. `tools="discover"` lets LLMs auto-invoke them.
- **[Memory](https://agentfield.ai/docs/build/coordination/shared-memory?utm_source=github-readme&utm_campaign=github-readme&utm_id=github-readme-memory)** - `app.memory.set()` / `.get()` / `.similarity_search()` - KV + vector search, four scopes, no Redis needed
Expand Down Expand Up @@ -284,7 +284,7 @@ Two examples already run at this load. The [deep-research engine](https://agentf
|---|---|
| Structured output (Pydantic/Zod) | `app.ai(schema=MyModel)` |
| Multi-turn coding agents | `app.harness("task")` — AForge by default |
| Orchestrate another harness | `app.harness("task", provider="claude-code")` |
| Orchestrate another harness | `app.harness("task", provider="claude-code")` (also `codex`, `gemini`, `opencode`, `pi`, `omp`) |
| LLM auto-discovers agents and tools | `app.ai(tools="discover")` |
| Multimodal (text, image, audio) | `app.ai("Describe", image_url="...")` |
| Streaming responses | `app.ai("...", stream=True)` |
Expand Down Expand Up @@ -381,7 +381,7 @@ Two examples already run at this load. The [deep-research engine](https://agentf
| Feature | How |
|---|---|
| Zero-setup default harness | AForge (`aforge`), installed alongside `af` |
| Swap the worker, keep the loop | `provider="claude-code"` \| `"codex"` \| `"gemini"` \| `"opencode"` |
| Swap the worker, keep the loop | `provider="claude-code"` \| `"codex"` \| `"gemini"` \| `"opencode"` \| `"pi"` \| `"omp"` |
| Fleet-wide default override | `AGENTFIELD_HARNESS_PROVIDER=codex` |
| Schema-constrained output | `schema=ResultModel` (Pydantic/Zod) |
| Cost capping | `max_budget_usd=3.0` |
Expand Down
146 changes: 126 additions & 20 deletions control-plane/internal/cli/doctor.go
Original file line number Diff line number Diff line change
Expand Up @@ -79,16 +79,31 @@ var providerEnvVars = []struct {
{Name: "google", EnvVar: "GOOGLE_API_KEY", Model: "gemini-1.5-pro"},
}

// harnessProviders is the canonical list of CLIs `app.harness()` knows how to drive.
var harnessProviders = []struct {
Name string // value passed to provider= in app.harness()
Binary string // executable name to look up on PATH
ProbeArgs []string // minimal one-shot invocation used by `--probe`
}{
// doctorHarnessProvider describes one CLI `app.harness()` knows how to drive,
// as `af doctor` surveys it.
type doctorHarnessProvider struct {
Name string // value passed to provider= in app.harness()
Binary string // executable name to look up on PATH
ProbeArgs []string // minimal one-shot invocation used by `--probe`; empty means "never probe"
ProbeStdin string // prompt fed over stdin, mirroring how the SDK adapters invoke this CLI
// JSONLStream marks providers whose probe stdout is a JSON event stream
// rather than plain text, so a non-empty stdout does not by itself mean
// the provider completed anything.
JSONLStream bool
}

// harnessProviders is the canonical list of CLIs `app.harness()` knows how to
// drive. aforge leads it: it is the SDK's default provider and ships with `af`.
var harnessProviders = []doctorHarnessProvider{
// aforge declares no ProbeArgs on purpose — see runHarnessProbes.
{Name: "aforge", Binary: "aforge"},
{Name: "claude-code", Binary: "claude", ProbeArgs: []string{"-p", "Say OK"}},
{Name: "codex", Binary: "codex", ProbeArgs: []string{"exec", "Say OK"}},
{Name: "gemini", Binary: "gemini", ProbeArgs: []string{"-p", "Say OK"}},
{Name: "opencode", Binary: "opencode", ProbeArgs: []string{"run", "Say OK"}},
// Keep pi/omp stdin prompt delivery coupled to the SDK adapters.
{Name: "pi", Binary: "pi", ProbeArgs: []string{"--print", "--mode", "json"}, ProbeStdin: "Say OK", JSONLStream: true},
{Name: "omp", Binary: "omp", ProbeArgs: []string{"--print", "--mode", "json"}, ProbeStdin: "Say OK", JSONLStream: true},
}

// harnessProbeTimeout bounds a single provider smoke test. Coding-agent CLIs
Expand Down Expand Up @@ -119,7 +134,7 @@ func NewDoctorCommand() *cobra.Command {
Long: `Doctor inspects the local environment and reports what's available for
building AgentField multi-reasoner systems:

• Available harness provider CLIs (claude-code, codex, gemini, opencode)
• Available harness provider CLIs (aforge, claude-code, codex, gemini, opencode, pi, omp)
• Provider API keys set in the environment (without leaking values)
• Docker availability and whether the control-plane image is locally cached
• Whether a local control plane is reachable
Expand Down Expand Up @@ -175,17 +190,23 @@ func runHarnessProbes(report DoctorReport) map[string]HarnessProbeResult {
if !report.HarnessProviders[h.Name].Available {
continue
}
results[h.Name] = probeHarnessProvider(h.Name, h.Binary, h.ProbeArgs, harnessProbeTimeout)
// Every other probe is one trivial completion. Aforge's only one-shot is
// a full coding-agent run with working-directory write access, so `af
// doctor` must not start it; `af harness doctor` reports its health.
if len(h.ProbeArgs) == 0 {
continue
}
results[h.Name] = probeHarnessProvider(h.Name, h.Binary, h.ProbeArgs, h.ProbeStdin, harnessProbeTimeout, h.JSONLStream)
}
return results
}

// probeHarnessProvider runs one provider CLI's minimal one-shot invocation and
// classifies the outcome.
func probeHarnessProvider(name, binary string, args []string, timeout time.Duration) HarnessProbeResult {
func probeHarnessProvider(name, binary string, args []string, stdin string, timeout time.Duration, jsonlStream bool) HarnessProbeResult {
start := time.Now()
stdout, stderr, exitCode, timedOut := runProbeCommand(binary, args, timeout)
status := classifyProbe(exitCode, stdout, timedOut)
stdout, stderr, exitCode, timedOut := runProbeCommand(binary, args, stdin, timeout)
status := classifyProbe(exitCode, stdout, timedOut, jsonlStream)

result := HarnessProbeResult{
Provider: name,
Expand All @@ -196,6 +217,9 @@ func probeHarnessProvider(name, binary string, args []string, timeout time.Durat
switch status {
case "error":
result.Detail = firstLine(stderr)
if result.Detail == "" && jsonlStream {
_, result.Detail = piProbeOutcome(stdout)
}
case "timeout":
result.Detail = fmt.Sprintf("no response within %s", timeout)
case "empty":
Expand All @@ -207,11 +231,14 @@ func probeHarnessProvider(name, binary string, args []string, timeout time.Durat
// runProbeCommand executes bin with args under a timeout, returning stdout,
// stderr, the process exit code, and whether the timeout fired. A timeout is
// reported distinctly so it is never misclassified as a plain error.
func runProbeCommand(bin string, args []string, timeout time.Duration) (stdout, stderr string, exitCode int, timedOut bool) {
func runProbeCommand(bin string, args []string, stdin string, timeout time.Duration) (stdout, stderr string, exitCode int, timedOut bool) {
ctx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()

cmd := exec.CommandContext(ctx, bin, args...)
if stdin != "" {
cmd.Stdin = strings.NewReader(stdin)
}
var outBuf, errBuf strings.Builder
cmd.Stdout = &outBuf
cmd.Stderr = &errBuf
Expand All @@ -232,20 +259,86 @@ func runProbeCommand(bin string, args []string, timeout time.Duration) (stdout,
}

// classifyProbe maps a probe outcome to a status. Order matters: a timeout is
// checked before the exit code (a killed process also exits non-zero), and an
// empty completion on a clean exit is the real-world "silently broken provider"
// case that a mere PATH check misses.
func classifyProbe(exitCode int, stdout string, timedOut bool) string {
// checked before the exit code (a killed process also exits non-zero). Plain
// text probes require non-empty stdout; JSONL probes require successful
// assistant completion text rather than merely any event.
func classifyProbe(exitCode int, stdout string, timedOut bool, jsonlStream bool) string {
switch {
case timedOut:
return "timeout"
case exitCode != 0:
return "error"
case strings.TrimSpace(stdout) == "":
return "empty"
default:
}
if jsonlStream {
hasAssistantText, providerError := piProbeOutcome(stdout)
if providerError != "" {
return "error"
}
if !hasAssistantText {
return "empty"
}
return "ok"
}
if strings.TrimSpace(stdout) == "" {
return "empty"
}
return "ok"
}

// piProbeOutcome inspects a Pi-family JSON event stream the way the SDK
// adapters do. A completion requires assistant text, and only the last
// assistant message_end determines whether the provider stopped with an error.
func piProbeOutcome(stdout string) (hasAssistantText bool, providerError string) {
type message struct {
Role string `json:"role"`
Content json.RawMessage `json:"content"`
StopReason string `json:"stopReason"`
ErrorMessage string `json:"errorMessage"`
}
type event struct {
Type string `json:"type"`
Message message `json:"message"`
}
type contentPart struct {
Type string `json:"type"`
Text string `json:"text"`
}

for _, line := range strings.Split(stdout, "\n") {
if strings.TrimSpace(line) == "" {
continue
}
var e event
if err := json.Unmarshal([]byte(line), &e); err != nil || e.Type != "message_end" || e.Message.Role != "assistant" {
continue
}

var text string
if err := json.Unmarshal(e.Message.Content, &text); err != nil {
var parts []contentPart
if json.Unmarshal(e.Message.Content, &parts) == nil {
for _, part := range parts {
if part.Type == "text" {
text += part.Text
}
}
}
}
if strings.TrimSpace(text) != "" {
hasAssistantText = true
}

switch e.Message.StopReason {
case "error", "aborted":
providerError = e.Message.ErrorMessage
if providerError == "" {
providerError = fmt.Sprintf("stopped with reason %q", e.Message.StopReason)
}
default:
providerError = ""
}
}
return hasAssistantText, providerError
}

// firstLine returns the first non-empty line of s, trimmed, for compact error
Expand Down Expand Up @@ -279,7 +372,20 @@ func buildDoctorReport(controlPlaneURL string) DoctorReport {
// Harness CLIs
availableHarness := []string{}
for _, h := range harnessProviders {
status := checkTool(h.Binary, "--version")
// Share the harness doctor's provider-specific version arguments and its
// $AGENTFIELD_HOME/bin fallback wherever a binary-backed spec exists, so
// both doctors agree on what "installed" means. aforge in particular
// answers `version`, not `--version`, and `af aforge ensure` puts it in
// AgentField's own bin directory rather than on PATH. claude-code has no
// binary in that table (it is the pip-package wrapper), so it keeps the
// plain PATH check.
spec := findHarnessProviderSpec(h.Name)
var status ToolStatus
if spec != nil && spec.Binary != "" {
status = probeHarnessBinary(*spec)
} else {
status = checkTool(h.Binary, "--version")
}
report.HarnessProviders[h.Name] = status
if status.Available {
availableHarness = append(availableHarness, h.Name)
Expand Down
2 changes: 2 additions & 0 deletions control-plane/internal/cli/doctor_additional_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,8 @@ func TestDoctorHelpersAndCommand(t *testing.T) {
"codex": {},
"gemini": {},
"opencode": {},
"pi": {},
"omp": {},
},
ProviderKeys: map[string]ProviderKey{
"openrouter": {EnvVar: "OPENROUTER_API_KEY", Set: true},
Expand Down
Loading
Loading