Skip to content

fix(reviewer): surface author rationale to reviewer agents - #62

Merged
AbirAbbas merged 6 commits into
mainfrom
fix/reviewer-sees-author-intent-v2
Aug 8, 2026
Merged

fix(reviewer): surface author rationale to reviewer agents#62
AbirAbbas merged 6 commits into
mainfrom
fix/reviewer-sees-author-intent-v2

Conversation

@AbirAbbas

Copy link
Copy Markdown
Contributor

Summary

Supersedes #25 — same fix, re-applied onto current main. The old branch predated a history rewrite, so GitHub showed a phantom ±40k-line diff; the actual change is this single commit (+33/−3), cherry-picked from 935e4c1 with one trivial conflict resolved.

Reviewer agents were producing findings that contradicted explicit design rationale stated in PR descriptions, because the rationale never reached them:

  1. Intake/anatomy truncated the description aggressively (500/1000/500 chars) — rationale past the cutoff was chopped before any model saw it.
  2. The reviewer agent never saw the raw description — only digested summaries (pr_narrative, risk_surfaces, intake_summary).
  3. The anatomy prompt explicitly discounts the description ("what the CODE says, not what the PR description says") — correct for divergence detection, but it left reviewers with no channel for author intent.

Changes

  • intake_phase (gate + fallback) and anatomy_phase: bump description truncation 500/1000/500 → 4000 chars uniformly.
  • review_dimension: new pr_description parameter with a dedicated "Author's Stated Intent" section near the top of the prompt. Framing tells the reviewer not to defer to the description — but a finding that contradicts an explicitly-justified design choice must rebut the stated rationale on its merits. Findings on points the description is silent about are unaffected.
  • ReviewOrchestrator._run_parallel_review: wire self.pr_data.description through to review_dimension.

Conflict resolution vs. old #25: main has since added feedback_section (human HITL guidance) at the same prompt-assembly insertion point — both sections are kept, human guidance first.

Test plan

  • Prompt-capture harness against a mocked router.app: with a 5000-char description carrying a rationale marker at char ~2400, the marker now reaches the intake gate, intake fallback, anatomy, and reviewer prompts; the reviewer prompt contains the "Author's Stated Intent" section capped at 4000 chars; empty description produces no section; human-guidance section still renders and precedes it. Same harness on main fails exactly these checks (10/10 pass on this branch, 4 relevant failures on main).
  • ruff check src/ scripts/ clean
  • pytest tests/ — 73 passed
  • go build ./... && go vet ./... && go test ./... + gofmt clean
  • docker build succeeds

🤖 Generated with Claude Code

Reviewers produced findings that contradicted explicit design rationale
from the PR description because the rationale never reached them.

- Intake gate / intake fallback / anatomy were truncating the description
  to 500 / 1000 / 500 chars. Thoughtful PR bodies with rationale past those
  cutoffs were silently chopped before any model saw them.
- review_dimension only received digested summaries (pr_narrative,
  risk_surfaces, intake_summary) — never the raw description, so the
  author's voice was laundered through two summarization layers before
  the reviewer looked at code.
- The anatomy prompt also tells the model to discount the description
  ("what the CODE says, not what the PR description says"), which is
  correct for divergence detection but leaves the reviewer with no
  channel for author intent.

Bump description truncation in intake/anatomy to 4000 chars uniformly,
add a pr_description param to review_dimension with a dedicated
"Author's Stated Intent" section near the top of the prompt, and wire
self.pr_data.description through _run_parallel_review.

The new section is explicitly NOT "trust the author" — it tells the
reviewer to verify the code regardless, but to rebut the author's
stated reasoning on its merits when a finding contradicts an
explicitly-justified design choice, rather than flagging as if the
rationale wasn't given. Findings on points the description is silent
about are unaffected.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@CLAassistant

CLAassistant commented Aug 3, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@AbirAbbas

Copy link
Copy Markdown
Contributor Author

Follow-up pass to make this merge-ready (merged current main + four commits):

  • Collision-safe description embedding: the author-controlled description was interpolated inside a fixed ``` fence — a description containing a fence could close the block and leak text into the instruction channel. All four embedding sites (intake gate, intake fallback, anatomy, reviewer section) now wrap it in collision-proof tags computed against the content, with explicit treat-as-data framing.
  • Go node parity (the default install since Installing this repo gives you the Go node, as pr-af #64): description caps 500/1000 → 4000 in the Go intake/anatomy prompts, and the "Author's Stated Intent" section — same wording, same placement after the human-guidance section — threaded through ReviewDimensionInputrunParallelReview, so spawned and coverage-gap dimensions get it too.
  • Committed tests on both sides: the PR-body capture harness is now tests/test_description_prompt_contracts.py plus Go prompt-contract tests and regenerated goldens: marker-at-char-2400 reachability, 4000-char cap, empty description omits the section, ordering after the feedback section, and fence-collision safety.

Gates: ruff clean, pytest 78 passed, go build/vet/test/gofmt clean.

@AbirAbbas
AbirAbbas merged commit 5a082d1 into main Aug 8, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants