Skip to content

feat: harden consensus loop, codex error classification and orientation default - #241

Merged
antonbabenko merged 1 commit into
masterfrom
feat/audit-quality-cost-fixes
Oct 10, 2026
Merged

antonbabenko merged 1 commit into
masterfrom
feat/audit-quality-cost-fixes

Conversation

@antonbabenko

Copy link
Copy Markdown
Owner

Summary

Ten fixes from a quality, cost and performance audit of the consensus loop, the Codex provider, the analyzer and the orientation feature.

Results quality

  • Codex error classification now runs on codex's own stderr lines (prompt echo removed, ANSI stripped). A review prompt that mentions "auth" or "login" no longer turns a usage-limit error into a non-retryable auth error that tells the user to log in again.
  • Arbiter peer-opinion cap goes from 2000 to 3000 chars and keeps head (1800) and tail (1200). The review format puts the verdict last, so a head-only cut dropped the conclusion of every long review.
  • consensus-step excerpt: a non-APPROVE voice whose reply had no [category] bullets now carries the first and last 600 chars of its text, so the host arbiter can turn the objection into a decision instead of seeing "REQUEST_CHANGES (0 critical)" every round.
  • Round digests in the provider-arbiter loop list dissenting issues as addressed and report real line-count diffs, so peers stop repeating the same issue.
  • Orientation auto-attach is ON by default (schema, default config, runtime resolver, docs). With it off, file-blind reviewers judged plans against a repo they could not see. Explicit enabled: false opts out.

Cost

  • The per-round blind arbiter pass in runToConvergence was one extra provider call per round whose result nothing read. It now runs only when consensus.blindVote is set, the same key the single-shot path honors.
  • A failed arbiter revision leg stops the loop with stopReason: "arbiter-revision-failed" instead of resubmitting the same plan and buying the same dissent again.

Performance and cleanup

  • Analyzer: a 360 s absolute slow gate for models seen under consensus / consensus-step (120 s elsewhere), so a long review round does not flag the whole panel as slow.
  • provider.js: removed the unused opinion-envelope schema, parser and validator (about 200 lines) and their tests.
  • Comment fixes in result-cache.js (cwd is part of the cache key) and answer-floor.js.

Cross-model review

Pre-PR consensus code review (expert code-reviewer, panel: codex, gemini, grok, one OpenRouter delegate) converged in 3 rounds.

Accepted and fixed:

  • R1 codex: the orientation default flip was only in the config template; the runtime resolver still turned an omitted block into off and the missing-config path had no orientation key. Fixed with a shared resolveOrientation helper plus tests ORC1-ORC3.
  • R2 codex: two stale doc lines still said orientation was off by default. Fixed, host artifacts regenerated.

Dismissed, with reasons:

  • grok, glm (R1): "revised starts null and the stop check sits outside the revise branch, so a unanimous APPROVE stops as arbiter-revision-failed." The loop breaks on converged before any revision code; every non-converged path assigns revised, and okText returns null for blank or errored text, so an empty plan is never submitted.
  • grok, glm (R1): "accept decisions are recorded before the revision, so a failed revision records issues as addressed." Round history is written only by submitRevision, which never runs after the arbiter-revision-failed break; when a revision lands, the revision prompt is built from exactly those issues.

Verification

  • npm run sync:check clean.
  • npm run check: 1181 pass. Five failures (bridge M1e, openrouter O14, dashboard-server S13, dashboard-cli C-stop and C-restart) are timing tests that fail identically on untouched master in a temp worktree on the same machine; none touches files in this change.

…on default

Ten fixes from a quality, cost and performance audit:

- codex: classify errors on codex's own stderr lines (prompt echo removed,
  ANSI stripped) so a prompt that mentions "auth" or "login" no longer turns
  a usage-limit error into a non-retryable auth error
- runToConvergence: skip the per-round blind arbiter pass unless
  consensus.blindVote is set (it was never read by convergence or prompts)
- runToConvergence: stop with stopReason "arbiter-revision-failed" instead
  of silently resubmitting the same plan when the revision leg fails
- runToConvergence: record dissenting peers' issues as accepted so the next
  round's digest lists them as addressed; let summarizePlanDiff report real
  line counts instead of a fixed label
- arbiter: cap each peer opinion at 3000 chars keeping head and tail, so the
  verdict at the end of a long review survives the cut
- consensus-step: attach a bounded excerpt to a non-APPROVE voice with no
  parsed [category] bullets so the host arbiter can decide on it
- provider.js: delete the unused opinion-envelope schema and parser
- analyze: use a 360 s absolute slow gate for models seen under consensus
  tools so a long round does not flag the whole panel
- orientation: default ON in schema, default config, runtime resolver and
  docs; explicit enabled:false opts out
- comment fixes in result-cache and answer-floor

Tests added or updated for every behavior change; docs and generated host
artifacts regenerated.
@antonbabenko
antonbabenko merged commit 050a6a7 into master Oct 10, 2026
1 check passed
@antonbabenko
antonbabenko deleted the feat/audit-quality-cost-fixes branch October 10, 2026 15:35
@github-actions

Copy link
Copy Markdown
Contributor

This PR is included in v3.29.0 🎉

Claude Code

/plugin marketplace update antonbabenko
/reload-plugins

If the update still shows the old version, the marketplace re-pin is a few minutes behind; run it again shortly.

Codex, Kiro, Cursor, OpenCode plugins: update deliberation from your host's plugin manager.

MCP server (any host): @antonbabenko/deliberation-mcp@3.29.0 is published. Restart your host to pick it up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant