[prompt-analysis] Copilot PR Prompt Analysis - 2026-03-12

[github-actions[bot]]

Summary

Analysis Period: Last 30 days | Total PRs: 999 | Merged: 787 (78.8%) | Closed: 212 (21.2%) | Open: 1

Overall merge rate is trending upward over the past 7 days (76.1% → 78.8%), suggesting prompt quality is improving.

---

Prompt Categories and Success Rates

Category	Total	Merged	Success Rate
Bug Fix	774	621	80.2%
Testing	102	73	71.6%
Feature Addition	48	37	77.1%
Refactor	28	25	89.3% ✅
Documentation	24	15	62.5%
Update/Upgrade	11	7	63.6%
Other	12	9	75.0%

Bug Fix dominates volume (77.5% of all PRs). Refactor prompts have the highest success rate (89.3%) despite lower volume.

---

Prompt Analysis

✅ Successful Prompt Patterns

Common characteristics in merged PRs:

• Average prompt body length: 996 words (vs 872 for closed — 14% longer)
• 99.2% include structured markdown (headers, lists) vs 91.0% of closed PRs
• 98.9% include backtick code references vs 92.0% of closed
• 74.6% reference a GitHub issue vs 65.6% of closed PRs

Top action verbs by success rate:

Verb	Total	Rate
bump	11	100%
rename	6	100%
refactor	27	96.3%
update	42	95.2%
move	9	88.9%
fix	346	86.1%

Example Merged PR Prompts

#20613 — fix: grant contents/pull-requests write to sync_actions reusable workflow call

The sync_actions job calls sync-actions.yml@main, which internally declares contents: write, pull-requests: write. The compiler always emits permissions: {} for the calling job… → Merged

#20608 — fix: enforce draft as policy in create-pull-request, not agent-overridable fallback

safe-outputs.create-pull-request.draft was treated as a default rather than a policy — the agent could include "draft": true in its message to bypass… → Merged

#20588 — Fix wasm golden tests and lint error after mcpg version bump to v0.1.14

Three wasm golden test fixtures and a lock file were still referencing the old MCP gateway image version (v0.1.8) after DefaultMCPGatewayVersion was bumped… → Merged

---

❌ Unsuccessful Prompt Patterns

Common characteristics in closed PRs:

• Average prompt body length: 872 words (shorter, less context)
• WIP prefix is the strongest closed-PR signal: 79 WIP PRs, only 26.6% merge rate
• Keywords strongly correlated with closure: wip, investigate, overriding, live, binary
• Less likely to include issue references (65.6% vs 74.6%)

Example Closed PR Prompts

#20617 — Remove workflow_call to sync_actions from release.md

The release workflow called sync-actions.yml via workflow_call before creating the GitHub release. This design causes... → Closed (likely superseded by a better-scoped fix)

#20604 — fix: auto-add github-actions[bot] to allowed bots for workflow_run triggers

workflow_run events always set github.actor to github-actions[bot], which has no repo collaborator role, causing pre_activation to silently fail... → Closed (approach reconsidered)

#20466 — Deduplicate engine setup logic across Claude/Gemini/Copilot/Codex

The secret-validation and installation-step assembly logic was duplicated verbatim across all four engine implementations... → Closed (complex refactor, scope unclear)

---

Key Insights

• Refactor prompts win: Despite being rare (28 PRs), refactor-categorized prompts have an 89.3% merge rate — the highest of any category. Targeted, scoped refactors with clear before/after descriptions get merged.
• WIP = high risk: 73.4% of WIP-prefixed PRs are closed. This suggests exploratory/draft PRs are being discarded rather than iterated upon. Agents should avoid the wip: prefix unless intentional.
• Structure and code references matter: Merged PRs are almost universally (99.2%) structured with markdown headers and inline code references. The small gap vs closed PRs (91%) suggests these alone don't guarantee success, but their absence correlates with failure.
• Issue links improve outcomes: PRs that reference an issue (fixes #N) are merged at 74.6% vs 65.6% for those without — a 9-point lift.

---

Recommendations

Based on today's analysis:

1. DO: Include structured markdown in prompts — use ## headers to separate "Problem", "Solution", and "Impact" sections
2. DO: Reference the specific code symbols being changed using backticks (e.g., `DefaultGitHubMCPServerVersion`)
3. DO: Link to related issues with Fixes #NNN to establish context and intent
4. DO: Use precision verbs like fix:, refactor:, update:, or bump: rather than vague openers like feat: or support:
5. AVOID: Using WIP prefix — it signals incompleteness and correlates strongly with closure (73.4% closed rate)
6. AVOID: Vague scope words (investigate, overriding, live) that lack clear solution descriptions

---

Historical Trends (Last 7 Days)

Date	PRs	Merged	Success Rate
2026-03-01	1000	761	76.1%
2026-03-02	1000	762	76.2%
2026-03-05	1000	775	77.5%
2026-03-06	1000	772	77.2%
2026-03-11	1000	781	78.1%
2026-03-12	999	787	78.8%

Trend: Merge rate is up +2.7 pp over the past 11 days — steady improvement. The dip on 2026-03-06 was minor and recovered strongly.

---

References:

• §22987325462

AI generated by Copilot PR Prompt Pattern Analysis · history

• expires on Mar 13, 2026, 5:07 AM UTC

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-03-13T05:07:52.984Z.

Closed by Workflow