Skip to content

Consume Verifiers eval process protocol#760

Draft
xeophon wants to merge 12 commits into
mainfrom
codex/prime-verifiers-process-adapter
Draft

Consume Verifiers eval process protocol#760
xeophon wants to merge 12 commits into
mainfrom
codex/prime-verifiers-process-adapter

Conversation

@xeophon

@xeophon xeophon commented Jun 24, 2026

Copy link
Copy Markdown
Member

Overview

Keeps Typer as the Prime command router while handing native V1 eval parsing and execution to Verifiers through its versioned process contract.

Native V1 flow

  • Prime validates the advertised process and trace schema versions.
  • Prime resolves the untouched V1 argv, reuses the returned run ID in memory, and replaces itself with the Verifiers run process.
  • Option-first argv, dynamic dotted options, config files, resume, help, TTY behavior, signals, and exit status stay owned by Verifiers.
  • The V1 -s shuffle flag is forwarded unchanged; only explicit --save-results selects the V0 compatibility path.
  • Process execution stays in the existing verifiers_bridge.py command bridge rather than adding a separate process module.

Artifact consumption

Prime push and Lab read the existing resolved config.toml and Trace results.jsonl artifacts through the shared evaluation utilities. Local run identity comes from the output directory name; no additional sidecar is required. Trace projection tolerates additive MessageNode fields and uses provider token usage when token IDs are unavailable.

Prime-owned options

V0 upload options remain scoped to the explicit V0 path. Native V1 runs can be uploaded separately with prime eval push, and hosted evaluation options remain Prime-owned.

Dependency

Verifiers protocol implementation: PrimeIntellect-ai/verifiers#1857

@xeophon xeophon changed the base branch from codex/verifiers-v1-cli to main June 24, 2026 15:43
@xeophon xeophon changed the title [codex] consume Verifiers eval process protocol Consume Verifiers eval process protocol Jun 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant