Skip to content

feat(agents): streamline code review across local changes, branch diffs, and pull requests - #2784

Open
Katrien De Graeve (katriendg) wants to merge 7 commits into
mainfrom
feat/code-review-pr-changes
Open

feat(agents): streamline code review across local changes, branch diffs, and pull requests#2784
Katrien De Graeve (katriendg) wants to merge 7 commits into
mainfrom
feat/code-review-pr-changes

Conversation

@katriendg

Copy link
Copy Markdown
Contributor

Description

This change unified pre-PR and pull-request review under one target-aware Code Review workflow. It separated review targets from Standard, Full, and Custom profiles, added a required factual orientation stage, and presented review areas, preliminary signals, perspective ownership, omissions, and depth rationale in one decision-ready confirmation.

Review workflow

  • Added explicit target resolution for local changes, branch diffs, and pull requests, with immutable head-SHA checks before diff generation and native emission.
  • Replaced the PR-only findings perspective with Code Review Orientation, which prepares the factual walkthrough and dispatch board without assigning findings or a verdict.
  • Expanded Readiness to cover target packaging, scope hygiene, validation evidence, changed documentation, and conditional pull-request metadata.
  • Serialized target identity, profile selection, orientation tasks, perspective output paths, retry context, and dispatch state for fresh-context workers.

Evaluation and documentation

  • Added behavior scenarios for target/profile independence, decision-ready dispatch, checkout mismatch handling, bounded retries, unsupported-finding rejection, and emission identity checks.
  • Updated the Code Review guide and generated agent references for the consolidated workflow.
  • Replaced the distributable Code Review PR agent membership with Code Review Orientation in plugin.json.

Related Issue(s)

Fixes #2783

Type of Change

Select all that apply:

Code & Documentation:

  • Bug fix (non-breaking change fixing an issue)
  • New feature (non-breaking change adding functionality)
  • Breaking change (fix or feature causing existing functionality to change)
  • Documentation update

Infrastructure & Configuration:

  • GitHub Actions workflow
  • Linting configuration (markdown, PowerShell, etc.)
  • Security configuration
  • DevContainer configuration
  • Dependency update

AI Artifacts:

  • Reviewed contribution with hve-builder and addressed all actionable findings
  • Copilot instructions (.github/instructions/*.instructions.md)
  • Copilot prompt (.github/prompts/*.prompt.md)
  • Copilot agent (.github/agents/*.agent.md)
  • Copilot skill (.github/skills/*/SKILL.md)
  • Copilot hook (.github/hooks/*/*.json)
  • Eval spec added/updated for changed AI artifacts (evals/)

Note for AI Artifact Contributors:

  • Agents: Research, indexing/referencing other project (using standard VS Code GitHub Copilot/MCP tools), planning, and general implementation agents likely already exist. Review .github/agents/ before creating new ones.
  • Skills: Must include both bash and PowerShell scripts. See Skills.
  • Model Versions: Contributions MUST target models listed in the model catalog (scripts/linting/model-catalog.json) whose provider appears in providerAllowlist and whose status is ga or preview. Run npm run lint:models to validate references.
  • See Agents Not Accepted and Model Version Requirements.

Other:

  • Script/automation (.ps1, .sh, .py)
  • Other (please describe):

Sample Prompts (for AI Artifact Contributions)

User Request:

Review my current branch against origin/main using the standard profile.

Execution Flow:

  1. The Code Review agent resolved local changes, a branch diff, or a pull request independently from the requested profile.
  2. It verified the checked-out head SHA, generated one structured diff, and serialized target and worker state.
  3. Code Review Orientation grouped changes into coherent areas and attached evidence-backed preliminary signals.
  4. The agent presented one confirmation for review areas, perspective ownership, omitted perspectives, and depth.
  5. Selected perspective workers produced evidence-backed findings, which the orchestrator validated and merged before any human-gated emission.

Output Artifacts:

Interactive runs persisted artifacts under .copilot-tracking/reviews/code-reviews/<branch>/:

orientation-walkthrough.md
diff-state.json
dispatch-manifest.json
<perspective>-findings.json
review.md
metadata.json
emission-record.json

The final report contained the changed-file summary, validated findings, verdict, execution limitations, next actions, and required human-review disclaimer.

Success Indicators:

  • The resolved target and profile remained independent in serialized state.
  • Orientation completed before findings dispatch and contained no severity or verdict.
  • The confirmation named review areas, preliminary evidence, perspective ownership, omissions, and depth rationale.
  • Missing worker output received one complete-context retry.
  • Unsupported findings remained visible as rejected evidence and did not affect counts or verdict.
  • Native emission remained blocked until human confirmation and a matching provider head SHA.

Testing

  • npm run plugin:sync passed and confirmed that plugin.json was current.
  • npm run plugin:validate passed with 55 agents, 48 commands, 57 rules, and 73 skills.
  • npm run docs:generate:check passed with all 239 generated asset documents unchanged.
  • npm run lint:ai-artifacts passed with zero issues.
  • npm run validate:docs passed ESLint, label-registry consistency, TypeScript checks, and 78 Docusaurus tests.
  • npm run spell-check passed all 823 files after correcting a truncated ambiguity regex alternative in the changed Code Review stimulus.
  • npm run lint:md-links failed on four unchanged RAI and security reference files.
  • npm run validate:local reached the same unrelated Markdown-link failures and did not pass as an aggregate.

Diff-based review also confirmed synchronized workflow contracts, focused behavior scenarios, documentation coverage, plugin membership replacement, and no dependency or security-script changes. Manual testing was not performed.

Checklist

Required Checks

  • Documentation is updated (if applicable)
  • Files follow existing naming conventions
  • Changes are backwards compatible (if applicable)
  • Tests added for new functionality (if applicable)

AI Artifact Contributions

  • Used hve-builder review mode to review contribution
  • Addressed all actionable findings from the hve-builder review
  • Verified contribution follows common standards and type-specific requirements

Required Local Checks

The following local-safe validation commands must pass before merging:

  • Local validation aggregate: npm run validate:local (blocked by unchanged Markdown-link failures)
  • Documentation validation (if docs changed): npm run validate:docs
  • Spell checking: npm run spell-check
  • Link validation: npm run lint:md-links

Security Considerations

  • This PR does not contain any sensitive or NDA information
  • Any new dependencies have been reviewed for security issues (N/A - no dependency changes)
  • Security-related scripts follow the principle of least privilege (N/A - no security scripts changed)

GHCP Membership Changes

File Type Membership change
.github/agents/coding-standards/subagents/code-review-orientation.agent.md Agent Added
.github/agents/coding-standards/subagents/code-review-pr.agent.md Agent Removed

Additional Notes

The broader getting-started documentation, end-to-end evaluation, interaction-choice coordination, reviewer coaching, and quantitative risk-scoring work remained tracked by #2092, #2093, #2094, #2284, and #2501 respectively.

- separate orientation from findings
- bind PR targets to immutable head SHAs
- add readiness checks and regression scenarios
- update review documentation and generated assets
…h board

- update orientation narrative to include preliminary signals
- restructure dispatch board to present coherent review areas
- add decision-ready confirmation surface for perspective and depth selection
- improve error handling for subagent invocations
@katriendg
Katrien De Graeve (katriendg) requested a review from a team as a code owner August 26, 2026 11:43
@codecov-commenter

Codecov Comments Bot (codecov-commenter) commented Aug 26, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.09%. Comparing base (3c3dc02) to head (9270b0f).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #2784      +/-   ##
==========================================
+ Coverage   82.98%   83.09%   +0.11%     
==========================================
  Files         183      183              
  Lines       33787    34086     +299     
  Branches       25       25              
==========================================
+ Hits        28038    28325     +287     
- Misses       5746     5758      +12     
  Partials        3        3              
Flag Coverage Δ
docusaurus 89.92% <ø> (ø)
pester 83.83% <ø> (+0.37%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 3 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Eval Execution

Status: Passed — no merge-blocking failures (1 advisory assertion failure(s) present)

  • Artifacts evaluated: 9
  • Specs run: 9
  • Assertions passed: 80
  • Assertions failed (blocking): 0
  • Assertions failed (advisory): 1
  • Failed specs (merge-blocking): 0
Artifact Kind Status Specs Passed Failed (blocking) Failed (advisory)
code-review agent ✅ pass 1 45 0 0
code-review-accessibility agent ✅ pass 1 5 0 0
code-review-functional agent ✅ pass 1 5 0 0
code-review-orientation agent ✅ pass 1 5 0 0
code-review-readiness agent ✅ pass 1 5 0 0
code-review-security agent ✅ pass 1 5 0 0
code-review-standards agent ✅ pass 1 5 0 0
pr-review prompt ⚠️ advisory-fail 1 2 0 1
code-review skill ✅ pass 1 3 0 0

Legend — ✅ clean · ⚠️ advisory failures only (non-blocking) · ⏭️ skipped · ❌ merge-blocking failure

Only Failed specs (merge-blocking) gates this PR. Advisory assertion failures are signal-quality checks captured during iteration; review them, but they do not block merge and may be acceptable.

…ontracts

- clarify that scenarios are behavior-contracts, not live reviews
- update patterns to include additional PR context
- ensure prompts specify assumptions about resolved pull requests and branches
…d behavior contracts

- add environment files for coding standards
- update regex patterns for improved matching
- refine prompts for better user interaction

@jkim323 Jamie Kim (jkim323) left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is neat work!!! Thanks for working on this change

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(agents): streamline code review across local changes, branch diffs, and pull requests

3 participants