fix(agents): align RAI assessments with supplied materials - #2764
Conversation
- route every entry mode through template-first preflight - persist template state for resume and recovery - validate preflight state fixtures 🔍 - Generated by Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- create one Mural widget for every supported content row - preserve rai-plan sources when no template is supplied - allow impact assessment structures to grow with evidence 🧩 - Generated by Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- use neutral assessment-focused report prose - exclude conversation and drafting narration from reports 📝 - Generated by Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
@microsoft-github-policy-service agree company="Microsoft" |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2764 +/- ##
==========================================
- Coverage 83.00% 82.98% -0.02%
==========================================
Files 185 173 -12
Lines 34924 34456 -468
Branches 228 0 -228
==========================================
- Hits 28989 28594 -395
- Misses 5861 5862 +1
+ Partials 74 0 -74
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
Bill Berry (WilliamBerryiii)
left a comment
There was a problem hiding this comment.
Thank you for clarifying the template-first RAI workflow. Changes are requested before merge. The Phase 1 protocol is duplicated across the agent and identity instruction and should be consolidated into one concise canonical sequence. The review also found blocking gaps in persisted template metadata, Mural area binding, stable-ID recovery, template-reference validation, and sensitive-data handling before Mural writes. Please address those contracts, add boundary and recovery tests, and align the entry-mode descriptions.
- address review comments r3891047473 and r3891047475 - align capture, PRD, and security-plan entry modes 🧭 - Generated by Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- address review comment r3891047482 - separate document paths from opaque Mural identifiers 🔒 - Generated by Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- address review comments r3891047478 and r3891047488 - preserve source keys and stable IDs across recovery 🧭 - Generated by Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- address review comment r3891047484 - bind areas, preview payloads, and require count approval 🔒 - Generated by Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- address review comment r3891047489 - retain logical order and semantic row and column headers ♿ - Generated by Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- address review comment r3891047491 - cover template, recovery-map, and security-plan failures 🧪 - Generated by Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…fix/rai-template-population
f8764b1
into
microsoft:main
Pull Request
Description
Problems and fixes
project materials and optional templates, causing rework or missing evidence.
Fix: Phase 1 checks optional templates before project materials and
permitted evidence sources.
content to only part of the assessment. Fix: Supplied templates use a
stable-ID
assessment-content.mdprojection, and rows and widgets expandbeyond existing placeholders.
drafting, or tool narration that required cleanup. Fix: Generated
documents use assessment-focused prose and exclude process narration.
Related Issue(s)
None.
Type of Change
Select all that apply:
Code & Documentation:
Infrastructure & Configuration:
AI Artifacts:
hve-builderand addressed all actionable findings.github/instructions/*.instructions.md).github/prompts/*.prompt.md).github/agents/*.agent.md).github/skills/*/SKILL.md).github/hooks/*/*.json)evals/)Other:
.ps1,.sh,.py)Sample Prompts (for AI Artifact Contributions)
User Request:
Execution Flow:
document and Mural templates before project materials.
assessment-content.mdskeleton.rows or Mural widgets to existing placeholders.
rai-plan.mdand prepares only theoutputs produced by the assessment.
Output Artifacts:
.copilot-tracking/rai-plans/{project-slug}/rai-plan.mdcontaining thecanonical assessment findings, evidence, controls, decisions, and tradeoffs.
.copilot-tracking/rai-plans/{project-slug}/assessment-content.mdcontaining stable rendering IDs and the combined structure of every supplied
template.
Success Indicators:
rai-plan.md.assessment-content.mdis required before they are populated.Existing Mural widgets and anchors define layout only; the planner creates
one widget for every stable-ID content row.
## System Definition, A2 from### AI Component Inventory, and A3 from## Stakeholder Impact.narration.
For detailed contribution requirements, see:
Testing
All validation ran under Node.js 24:
npm run test:ps -- -TestPath scripts/tests/linting/Test-RaiStateSchema.Tests.ps1passed 7 of 7 tests.
npm run docs:generate:checknpm run plugin:validatenpm run validate:docsnpm run spell-checknpm run lint:md-linksnpm run lint:tablesnpm run lint:frontmatternpm run lint:ai-artifactsnpm run validate:skillsnpm run validate:localDiff review confirmed that Entry Modes and
suggestedDepthTierbehavior remainunchanged from the base branch and that placeholder/cardinality behavior is
contained in one commit. The HVE Builder behavior simulation passed with no
in-scope findings. Native Mural, WorkIQ, backlog, signing, and file-write
behavior was not exercised.
Checklist
Required Checks
The six-phase workflow and Entry Modes remain unchanged. The state schema now
requires persisted template-preflight records, so state created before this
change requires migration.
AI Artifact Contributions
hve-builderreview mode to review contributionhve-builderreviewRequired Local Checks
The following local-safe validation commands must pass before merging:
npm run validate:localnpm run validate:docsnpm run spell-checknpm run lint:md-linksSecurity Considerations
GHCP Artifact Maturity
Warning
This PR includes an experimental GHCP artifact that may have breaking changes.
.github/agents/rai-planning/rai-planner.agent.md.github/instructions/rai-planning/rai-identity.instructions.md.github/agents/rai-planning/rai-planner.agent.md.github/instructions/rai-planning/rai-identity.instructions.mdGHCP Maturity Acknowledgment
Additional Notes
The repository pull request template was preserved. Required AI artifact and
maturity acknowledgments are complete. Nothing has been pushed and no pull
request has been created.