auto-api-docs-writer: orchestration + branch-safety hardening, configurable base/dest#155
auto-api-docs-writer: orchestration + branch-safety hardening, configurable base/dest#155mattleibow wants to merge 10 commits into
Conversation
Rewrite the daily writer to match the upgraded api-docs skill, which edits the mdoc XML directly instead of the extract→write→merge JSON round-trip. Host steps: - Drop the "Extract placeholders and manifest" and "Upload extracted JSON" steps from regenerate-stubs, the "Download pre-extracted JSON" / "Save original JSON" pre-agent steps, and the "Save final JSON" post-step. None are needed without the JSON layer. - Add a non-fatal "Bootstrap SkiaSharp binding for snippet checks" pre-agent step (externals-download + dotnet build binding/SkiaSharp) so the example reviewer can compile-check snippets against a real SkiaSharp.dll. - Keep the SkiaSharpAPI symlink (for the host docs-format-docs post-step) and the formatting post-step. Agent prompt (runtime-imported body): - Replace the JSON phases with the skill's direct-XML add pipeline: resolve-scope new -> writer edits XML -> lint + 3 reviewers + synthesizer -> fix CRITICAL -> structural validate -> commit + PR. - Route each sub-agent through the task tool's per-role model (writer/factual/ examples = opus, quality/synthesizer = sonnet), with an engine.model fallback if the sandbox does not honor per-sub-agent models. - Export DOCS_GIT_ROOT/DOCS_DIR on docs-tool.ps1 calls so scope/validate use the docs repo (primary checkout) for git baselines while source lookups still use the SkiaSharp clone. Recompiled the .lock.yml via `gh aw compile`. Note: depends on the matching api-docs skill update in mono/SkiaSharp (the cloned skiasharp_branch must carry the new skill + docs-tool.ps1 DOCS_GIT_ROOT support). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
PoliCheck Scan ReportThe following report lists PoliCheck issues in PR files. Before you merge the PR, you must fix all severity-1 and severity-2 issues. The AI Review Details column lists suggestions for either removing or replacing the terms. If you find a false positive result, mention it in a PR comment and include this text: #policheck-false-positive. This feedback helps reduce false positives in future scans. ✅ No issues foundMore information about PoliCheckInformation: PoliCheck | Severity Guidance | Term |
|
Learn Build status updates of commit b9ed9e6: ✅ Validation status: passed
For more details, please refer to the build report. |
The api-docs eval per-role bake-off picked gpt-5.5 for reviewer-factual (only candidate that caught the seeded byte-order contradiction; best mean recall + precision). Align the workflow's per-role routing note with the skill's updated review.md table and reviewer-factual Model: header. Body-only change (runtime-imported), so the compiled .lock.yml is unaffected. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
PoliCheck Scan ReportThe following report lists PoliCheck issues in PR files. Before you merge the PR, you must fix all severity-1 and severity-2 issues. The AI Review Details column lists suggestions for either removing or replacing the terms. If you find a false positive result, mention it in a PR comment and include this text: #policheck-false-positive. This feedback helps reduce false positives in future scans. ✅ No issues foundMore information about PoliCheckInformation: PoliCheck | Severity Guidance | Term |
|
Learn Build status updates of commit 617b8a3: ✅ Validation status: passed
For more details, please refer to the build report. |
…put) Sibling of auto-api-docs-writer that runs the api-docs skill's REVIEW pipeline on a scope (default group:text) instead of filling placeholders. Doubles as the first CI exercise of per-sub-agent model routing: the orchestrator launches each reviewer/writer sub-agent via the task tool with an explicit model (factual -> gpt-5.5 per the eval bake-off) and prints a Routing report so the run is auditable. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
PoliCheck Scan ReportThe following report lists PoliCheck issues in PR files. Before you merge the PR, you must fix all severity-1 and severity-2 issues. The AI Review Details column lists suggestions for either removing or replacing the terms. If you find a false positive result, mention it in a PR comment and include this text: #policheck-false-positive. This feedback helps reduce false positives in future scans. ✅ No issues foundMore information about PoliCheckInformation: PoliCheck | Severity Guidance | Term |
|
Learn Build status updates of commit d28641a: ✅ Validation status: passed
For more details, please refer to the build report. |
…g report) Make the daily writer the common path: after the add pass (fill placeholders) it now also runs a REVIEW pass over a baked scope of existing docs (review-scope.txt, default group:text) — lint + 3 reviewers + synthesizer + fix CRITICAL/obsolete via per-role task models — then one validate + PR covers both passes. Because the writer is already registered on the docs default branch, this is dispatchable on a feature branch (skiasharp_branch input) WITHOUT landing a new workflow on main; the review scope is baked (not a dispatch input) so input validation against the default branch still passes. Adds a mandatory Routing report so the run doubles as the first CI exercise of per-sub-agent model routing. Removes the now-redundant standalone auto-api-docs-reviewer workflow. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
PoliCheck Scan ReportThe following report lists PoliCheck issues in PR files. Before you merge the PR, you must fix all severity-1 and severity-2 issues. The AI Review Details column lists suggestions for either removing or replacing the terms. If you find a false positive result, mention it in a PR comment and include this text: #policheck-false-positive. This feedback helps reduce false positives in future scans. ✅ No issues foundMore information about PoliCheckInformation: PoliCheck | Severity Guidance | Term |
|
Learn Build status updates of commit e535c65: ✅ Validation status: passed
For more details, please refer to the build report. |
…tself The first CI pilot run completed green but produced NO PR. Root cause: the orchestrator delegated the terminal fix step to a background "fixer" sub-agent, then ended its turn before that agent (and the PR) completed -- exactly the session-killing pattern the older workflow had guardrails against. Separately, per-sub-agent model routing turned out to be cosmetic in the gh-aw sandbox: the api-proxy token-usage log showed every call was claude-sonnet-4.6 regardless of the requested per-role model. Fixes: - Pin the run model: explicit engine.model claude-sonnet-4.6 (drops the GH_AW_MODEL_AGENT_COPILOT override; orchestrator + every sub-agent run on it). - Drop cosmetic per-role routing and the routing-report requirement; document per-role routing as a local-only skill feature. - Terminal fixes are now the orchestrator's own FOREGROUND work (A4/R4); the only sub-agents launched are the pass-A writer and the reviewers. No terminal background agent. Synthesis is the orchestrator's job (no synthesizer agent). - Restore the tight escape hatch: once reviewers report, timebox fixing to ~10 min, then validate + open the PR. - Add an explicit "No terminal background agent" Critical rule and simplify the completion gate (create_pull_request/noop is the orchestrator's own job). Recompiled auto-api-docs-writer.lock.yml. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
PoliCheck Scan ReportThe following report lists PoliCheck issues in PR files. Before you merge the PR, you must fix all severity-1 and severity-2 issues. The AI Review Details column lists suggestions for either removing or replacing the terms. If you find a false positive result, mention it in a PR comment and include this text: #policheck-false-positive. This feedback helps reduce false positives in future scans. ✅ No issues foundMore information about PoliCheckInformation: PoliCheck | Severity Guidance | Term |
|
Learn Build status updates of commit 3be9831: ✅ Validation status: passed
For more details, please refer to the build report. |
Validation run 28203165066 produced a real PR (the orchestration fix worked), but the agent rationalized out of step C: it saw it was already on the dispatch ref (a feature branch ahead of main), decided to "commit here and create the PR from this branch," and skipped `git checkout -b automation/write-api-docs`. Because safe-outputs preserves the committed branch and force-resets it (recreate_ref:true), this force-overwrote the workflow's own source branch. Fix: make step C unconditional — `git checkout -B automation/write-api-docs` before committing, even when already on a feature branch ahead of main, staging only SkiaSharpAPI/. Add a matching Critical rule explaining that committing on the dispatch ref destroys it under recreate_ref. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… runs Root cause of the run-156 branch clobber: gh-aw's checkout step makes the DISPATCH REF the agent's working branch (git checkout -B <dispatch-ref>). When dispatched from a feature branch, the agent starts on that branch; it then committed its doc work there instead of switching to automation/write-api-docs, and safe-outputs (preserve_branch_name + recreate_ref) adopted that branch name as the PR head and force-recreated it — erasing the workflow source commits. Relying on the agent to switch branches is fragile (it rationalized staying put because the branch was "already ahead by 1 commit"). Add a host pre-agent step that renames the working branch to automation/write-api-docs before the agent starts, so every commit and the PR head land on a throwaway branch regardless of which ref triggered the run. The prompt rule remains as defense-in-depth. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Parameterize the docs PR base (docs_base_branch) and PR head (docs_head_branch)
as workflow_dispatch inputs so the writer can be tested against an older docs
state where many 'To be added.' placeholders still exist.
- base-branch -> ${{ inputs.docs_base_branch || 'main' }}; gh-aw propagates it
to the safeoutputs config.json, the safe_outputs checkout ref, and the handler
env (all interpolation-safe).
- regenerate-stubs aligns the docs submodule to docs_base_branch (was hardcoded main).
- host PR-branch step checks out docs_head_branch (was hardcoded automation/write-api-docs).
- prompt no longer hardcodes the PR branch name: commit on the host-prepared current
branch, never the dispatch ref.
Backward compatible: schedule/push runs default base=main, head=automation/write-api-docs.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
mattleibow
changed the title
|
Learn Build status updates of commit 3be9831: ✅ Validation status: passed
For more details, please refer to the build report. |
|
Learn Build status updates of commit e30f07b: ✅ Validation status: passed
For more details, please refer to the build report. |
A shallow submodule clone has no origin/<branch> remote-tracking ref for an arbitrary docs_base_branch, so 'git checkout -B stub-base origin/<branch>' fails (it only worked for main, which is tracked). Use FETCH_HEAD, set by the preceding fetch, which resolves for any branch. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Learn Build status updates of commit cdae111: ✅ Validation status: passed
For more details, please refer to the build report. |
PoliCheck Scan ReportThe following report lists PoliCheck issues in PR files. Before you merge the PR, you must fix all severity-1 and severity-2 issues. The AI Review Details column lists suggestions for either removing or replacing the terms. If you find a false positive result, mention it in a PR comment and include this text: #policheck-false-positive. This feedback helps reduce false positives in future scans. ✅ No issues foundMore information about PoliCheckInformation: PoliCheck | Severity Guidance | Term |
- review_scope workflow_dispatch input (default group:text) drives the review pass scope, so other doc groups can be reviewed one at a time. - Commit step now unstages generated files (index.xml, ns-*.xml, _filter.xml, FrameworksIndex/) that stub regeneration rewrites, keeping them out of the PR. Exposed by the add-at-scale run against an old base where the regenerated generated files diverged from the base. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
PoliCheck Scan ReportThe following report lists PoliCheck issues in PR files. Before you merge the PR, you must fix all severity-1 and severity-2 issues. The AI Review Details column lists suggestions for either removing or replacing the terms. If you find a false positive result, mention it in a PR comment and include this text: #policheck-false-positive. This feedback helps reduce false positives in future scans. ✅ No issues foundMore information about PoliCheckInformation: PoliCheck | Severity Guidance | Term |
|
Learn Build status updates of commit a9a9ba2: ✅ Validation status: passed
For more details, please refer to the build report. |
1 participant
What
Hardens the
auto-api-docs-writeragentic workflow that (A) fillsTo be added.placeholders and (B) reviews/improves a scope of existing mdoc XML docs, opening a PR via safe-outputs.Changes
Restored orchestration discipline (
3be9831). The orchestrator does the terminal fix → validate → commit → PR itself in the foreground. The only sub-agents are the pass-A writer + 3 reviewers; there is no terminal background "fixer" agent (the prior no-PR failure mode: backgrounding a fixer then ending the turn, killing the session before the PR). Pinned the whole run toclaude-sonnet-4.6viaengine.modeland dropped the cosmetic per-role routing — per-roletask(model=…)is not plumbed through in the gh-aw sandbox (every call was Sonnet regardless), so per-role routing is documented as local-only.Branch-clobber fix (
39fdf15+10fe960). gh-aw's checkout puts the agent on the dispatch ref; withrecreate-ref: truethe agent's commit branch becomes the PR head and is force-recreated — which destroyed this workflow's own source branch when dispatched from adev/*branch. Fixed at two layers: a host pre-agent step renames the working branch to a throwaway PR branch before the agent runs, and the prompt now says "commit on the branch you're already on, never the dispatch ref."Configurable base/dest (
e30f07b). Newworkflow_dispatchinputsdocs_base_branch(defaultmain) anddocs_head_branch(defaultautomation/write-api-docs). gh-aw propagates the base expression to the safeoutputs config, the safe_outputs checkout ref, and the handler env; stub-regen aligns todocs_base_branch; the host step checks outdocs_head_branch. This lets the writer be tested against an older docs state where many placeholders still exist. Backward compatible —schedule/pushruns are unchanged.Validation
CI run
28257558423(dispatched from this branch) → PR #158: both passes ran, the dev branch was left intact (not clobbered), and the PR landed onautomation/write-api-docs. Pass A added 6 new member docs (SKColorFilter.CreateOverdraw×2,SKImageFilter.CreateCrop×3,CreateEmpty); Pass R caught obsolete-example migrations, a removed-type reference, and a copy-paste summary/returns bug across the text-API docs.