Skip to content

fix(workflows): harden code scanning issue reconciliation - #2817

Draft
Jamie Kim (jkim323) wants to merge 6 commits into
mainfrom
chore/deps/npm-dependencies-2026-08
Draft

fix(workflows): harden code scanning issue reconciliation#2817
Jamie Kim (jkim323) wants to merge 6 commits into
mainfrom
chore/deps/npm-dependencies-2026-08

Conversation

@jkim323

@jkim323 Jamie Kim (jkim323) commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

Pull Request

Description

Harden automated GitHub code-scanning issue management so issue creation and reconciliation fail safely when ownership, input completeness, or CodeQL analysis health cannot be verified.

This change:

  • validates grouped alert records and rejects duplicate rule IDs before mutation
  • indexes automation-owned issues across open and closed states using a strict first-line marker, GitHub Actions app author, and required ownership labels
  • reopens verified closed issues when their rule returns instead of creating duplicates
  • adds dry-run and enforce reconciliation modes for closing issues whose rules are no longer present
  • skips reconciliation when CodeQL analysis health or issue-index completeness is uncertain
  • uses the repository default branch for CodeQL queries and generated source links
  • documents the marker grammar and ownership requirements in the GitHub code-scanning skill

Related Issue(s)

Closes #2763

Type of Change

Select all that apply:

Code & Documentation:

  • Bug fix (non-breaking change fixing an issue)
  • New feature (non-breaking change adding functionality)
  • Breaking change (fix or feature causing existing functionality to change)
  • Documentation update

Infrastructure & Configuration:

  • GitHub Actions workflow
  • Linting configuration (markdown, PowerShell, etc.)
  • Security configuration
  • DevContainer configuration
  • Dependency update

AI Artifacts:

  • Reviewed contribution with hve-builder and addressed all actionable findings
  • Copilot instructions (.github/instructions/*.instructions.md)
  • Copilot prompt (.github/prompts/*.prompt.md)
  • Copilot agent (.github/agents/*.agent.md)
  • Copilot skill (.github/skills/*/SKILL.md)
  • Copilot hook (.github/hooks/*/*.json)
  • Eval spec added/updated for changed AI artifacts (evals/)

Other:

  • Script/automation (.ps1, .sh, .py)
  • Other (please describe): hardened workflow automation and supporting skill guidance

Sample Prompts (for AI Artifact Contributions)

User Request:

Retrieve GitHub code-scanning alerts and create or reconcile automation-owned tracking issues safely.

Execution Flow:

  1. Retrieve grouped code-scanning alerts for the repository default branch.
  2. Verify CodeQL analysis health and build a complete index of automation-owned tracking issues.
  3. Validate alert records before any issue mutation.
  4. Create or reopen one verified issue per live rule.
  5. Report stale issue candidates in dry-run mode or close them in enforce mode.
  6. Skip mutation when scan health, ownership, uniqueness, or candidate completeness is uncertain.

Output Artifacts:

The workflow updates GitHub issues and emits concise health, indexing, creation, reopening, and reconciliation logs.

Success Indicators:

  • one verified automation-owned issue exists per live rule
  • duplicate or malformed input aborts before mutation
  • stale issues are reported in dry-run mode and closed only in enforce mode
  • unhealthy, stale, incomplete, or ambiguous evidence prevents reconciliation

Testing

Validated the changed workflow and skill surfaces with:

  • YAML and GitHub Actions validation
  • workflow permission, runner, dangerous-pattern, and dependency-pinning checks
  • skill structure and Markdown table validation
  • targeted CSpell validation for all three changed files
  • fixture-based tests for malformed grouped alerts, duplicate rules, issue ownership markers, open and closed issue handling, ambiguity, truncation, dry-run behavior, and enforced reconciliation
  • plugin manifest synchronization and generated documentation drift checks
  • git diff --check

Checklist

Required Checks

  • Documentation is updated (if applicable)
  • Files follow existing naming conventions
  • Changes are backwards compatible (if applicable)
  • Tests added for new functionality (if applicable)

AI Artifact Contributions

  • Used hve-builder review mode to review contribution
  • Addressed all actionable findings from the hve-builder review
  • Verified contribution follows common standards and type-specific requirements

Required Local Checks

  • Local validation aggregate: npm run validate:local
  • Documentation validation (if docs changed): npm run validate:docs
  • Spell checking: npm run spell-check
  • Link validation: npm run lint:md-links

Security Considerations

  • This PR does not contain any sensitive or NDA information
  • Any new dependencies have been reviewed for security issues
  • Security-related scripts follow the principle of least privilege

The workflow grants security-events: read only where required to inspect CodeQL analysis health. Issue mutation still uses the repository-provided GITHUB_TOKEN with explicit issues: write permission. Reconciliation defaults to dry-run and fails closed when ownership or evidence is incomplete.

Additional Notes

@jkim323 Jamie Kim (jkim323) self-assigned this Aug 31, 2026
@github-actions

github-actions Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Eval Execution

Status: Passed — no merge-blocking failures (1 advisory assertion failure(s) present)

  • Artifacts evaluated: 1
  • Specs run: 1
  • Assertions passed: 8
  • Assertions failed (blocking): 0
  • Assertions failed (advisory): 1
  • Failed specs (merge-blocking): 0
Artifact Kind Status Specs Passed Failed (blocking) Failed (advisory)
gh-code-scanning skill ⚠️ advisory-fail 1 8 0 1

Legend — ✅ clean · ⚠️ advisory failures only (non-blocking) · ⏭️ skipped · ❌ merge-blocking failure

Only Failed specs (merge-blocking) gates this PR. Advisory assertion failures are signal-quality checks captured during iteration; review them, but they do not block merge and may be acceptable.

@codecov-commenter

Codecov Comments Bot (codecov-commenter) commented Aug 31, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.24%. Comparing base (bb0ac58) to head (1fdf756).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #2817      +/-   ##
==========================================
+ Coverage   83.14%   83.24%   +0.09%     
==========================================
  Files         183      172      -11     
  Lines       34039    34350     +311     
  Branches       27        0      -27     
==========================================
+ Hits        28302    28594     +292     
- Misses       5734     5756      +22     
+ Partials        3        0       -3     
Flag Coverage Δ
docusaurus ?
pester 84.24% <ø> (+0.40%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 20 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jkim323
Jamie Kim (jkim323) marked this pull request as ready for review September 4, 2026 04:50
@jkim323
Jamie Kim (jkim323) requested a review from a team as a code owner September 4, 2026 04:50
@jkim323
Jamie Kim (jkim323) marked this pull request as draft September 4, 2026 04:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(workflows): close code-scanning issues whose alerts are no longer open

2 participants