[delight] UX Analysis Report 2026-07-09: Trial Ops & MCP Guide Improvements

[github-actions[bot]]

User Experience Analysis Report - 2026-07-09

Executive Summary

Today's analysis focused on:

• 2 documentation files
• 0 CLI commands (binary not available in sandbox)
• 2 workflow message configurations
• 1 validation file (integration test)

Overall Quality: Needs Minor Work

Key Finding: docs/src/content/docs/experimental/trial-ops.md contains a duplicate Related Documentation entry for MultiRepoOps (lines 144–145), and docs/src/content/docs/guides/mcps.md contains a corrupted template expression in the Authorization header example (line 163) that will mislead users trying to configure static API key authentication.

---

Quality Highlights ✅

Example 1: MCP Guide — Comprehensive Structure

• File: docs/src/content/docs/guides/mcps.md
• What works well: Clear progressive structure (Quick Start → Server Types → Tool Filtering → Registry → Examples), appropriate use of warning callouts, consistent code blocks with aw wrap language tag, practical examples covering all server types.
• Quote: The get_me warning block is an excellent enterprise UX pattern: "Do not call get_me to discover the agent's identity. Instead, read identity fields directly from the <github-context> block..." — specific, actionable, explains the alternative.

Example 2: TrialOps — Structured Mode Table

• File: docs/src/content/docs/experimental/trial-ops.md
• What works well: The Repository Modes table provides at-a-glance comparison across four modes with flags and descriptions. Result file structure shown as concrete JSON example reduces ambiguity.

---

Improvement Opportunities 💡

High Priority

Opportunity 1: Duplicate Related Documentation Entry — Single File Improvement

• File: docs/src/content/docs/experimental/trial-ops.md
• Current State: Lines 144–145 both link to the same [MultiRepoOps](/gh-aw/patterns/multi-repo-ops/) page. They use inconsistent separators (em dash — vs hyphen -) and slightly different descriptions ("Run workflows from separate repositories" vs "Coordinate across multiple repositories").
• Issue: Duplicate link to the same resource looks like a copy-paste error; inconsistent dash style breaks visual uniformity in a bulleted list.
• User Impact: Enterprise users scanning Related Documentation will click the same link twice or lose confidence in the document's overall accuracy. The inconsistency signals the page may not be actively maintained.
• Suggested Change: Remove the duplicate line (line 145) and keep the more descriptive, em-dash-formatted version (line 144).
• Design Principle: Trust and Reliability — consistent, accurate documentation builds confidence.

Medium Priority

Opportunity 2: Corrupted Template Expression in Authorization Example — Single File Improvement

• File: docs/src/content/docs/guides/mcps.md
• Current State: Line 163: Authorization: "****** secrets.API_TOKEN }}" — the ${{ expression opener has been replaced with six asterisks, producing a broken template. Every other secret reference in the file uses the correct ${{ secrets.NAME }} form (e.g., lines 100, 136).
• Issue: The corrupted syntax could cause a user to literally copy "****** secrets.API_TOKEN }}" into their workflow frontmatter, resulting in a runtime error with no clear explanation. It also looks like an accidental redaction artifact.
• User Impact: Breaks user trust ("is this a typo or intentional masking?"), and will cause a confusing runtime failure for anyone who copies it verbatim.
• Suggested Change: Restore to Authorization: "${{ secrets.API_TOKEN }}" for consistency with all other secret template examples in the file.
• Design Principle: Clarity and Precision — examples must be copy-safe and consistent.

---

Files Reviewed

Documentation

• docs/src/content/docs/experimental/trial-ops.md — Rating: ⚠️ Needs Minor Work
• docs/src/content/docs/guides/mcps.md — Rating: ⚠️ Needs Minor Work

CLI Commands

• (Not evaluated — binary unavailable in sandbox)

Workflow Messages

• .github/workflows/ruflo-backed-task.md — Rating: ✅ Professional
• .github/workflows/smoke-agent-all-merged.md — Rating: ✅ Professional

Validation Code

• pkg/workflow/repository_features_validation_test.go — Rating: ✅ Professional

---

Metrics

• Files Analyzed: 5
• Quality Distribution:
  • ✅ Professional: 3
  • ⚠️ Needs Minor Work: 2
  • ❌ Needs Significant Work: 0

---

🎯 Actionable Tasks

Here are 2 targeted improvement tasks, each affecting a single file:

---

Task 1: Remove Duplicate Related Documentation Entry — trial-ops.md

File to Modify: docs/src/content/docs/experimental/trial-ops.md

Current Experience

The Related Documentation section (lines 142–149) lists MultiRepoOps twice in consecutive bullets:

- [MultiRepoOps](/gh-aw/patterns/multi-repo-ops/) — Run workflows from separate repositories
- [MultiRepoOps](/gh-aw/patterns/multi-repo-ops/) - Coordinate across multiple repositories

Quality Issue

Design Principle: Trust and Reliability

Duplicate entries with inconsistent punctuation (em dash vs hyphen) signal copy-paste error and undermine confidence in the document's maintenance quality. Enterprise users notice these details and may question the accuracy of other information in the page.

Proposed Improvement

Remove line 145 (the duplicate with hyphen) and keep line 144 (em-dash style, consistent with OrchestratorOps entry on line 146).

Before:

- [MultiRepoOps](/gh-aw/patterns/multi-repo-ops/) — Run workflows from separate repositories
- [MultiRepoOps](/gh-aw/patterns/multi-repo-ops/) - Coordinate across multiple repositories
- [OrchestratorOps](/gh-aw/patterns/orchestrator-ops/) — Orchestrate multi-step initiatives

After:

- [MultiRepoOps](/gh-aw/patterns/multi-repo-ops/) — Run workflows from separate repositories
- [OrchestratorOps](/gh-aw/patterns/orchestrator-ops/) — Orchestrate multi-step initiatives

Why This Matters

• User Impact: Related Documentation sections are scanned to discover adjacent capabilities; a duplicate entry wastes the user's attention and undermines trust in the page.
• Quality Factor: Accuracy and professional presentation.
• Frequency: Every user who reads to the bottom of the TrialOps guide encounters this.

Success Criteria

• Changes made to docs/src/content/docs/experimental/trial-ops.md only
• Duplicate MultiRepoOps bullet removed
• Em-dash style consistent with remaining entries
• Quality rating improves from ⚠️ to ✅

Scope Constraint

• Single file only: docs/src/content/docs/experimental/trial-ops.md
• No changes to other files required
• Can be completed independently

---

Task 2: Fix Corrupted Authorization Header Template — mcps.md

File to Modify: docs/src/content/docs/guides/mcps.md

Current Experience

Line 163 inside the HTTP MCP Servers section contains a broken template expression:

headers:
  Authorization: "****** secrets.API_TOKEN }}"

The ${{ prefix was replaced with six asterisks. Every other secret reference in the file uses the correct form: "${{ secrets.NAME }}".

Quality Issue

Design Principle: Clarity and Precision

A user copying this example verbatim will create a workflow that passes the literal string "****** secrets.API_TOKEN }}" as an Authorization header — causing a rejected request at the MCP server with no clear error message pointing back to the docs. The obfuscation looks either like an accidental redaction artifact or an intentional masking that inadvertently broke the example.

Proposed Improvement

Restore the correct GitHub Actions template expression syntax.

Before:

  authenticated-api:
    url: "(api.example.com/redacted)
    headers:
      Authorization: "****** secrets.API_TOKEN }}"
    allowed: ["*"]

After:

  authenticated-api:
    url: "(api.example.com/redacted)
    headers:
      Authorization: "${{ secrets.API_TOKEN }}"
    allowed: ["*"]

Why This Matters

• User Impact: Broken copy-paste examples are the rejig docs #1 source of user frustration in technical documentation; they create silent runtime failures that are hard to debug.
• Quality Factor: Technical accuracy and copy-safety of examples.
• Frequency: The MCP guide is a high-traffic reference page; static API key auth is a common configuration path.

Success Criteria

• Changes made to docs/src/content/docs/guides/mcps.md only
• Line 163 Authorization value corrected to "${{ secrets.API_TOKEN }}"
• Consistent with all other ${{ secrets.* }} references in the file
• Quality rating improves from ⚠️ to ✅

Scope Constraint

• Single file only: docs/src/content/docs/guides/mcps.md
• No changes to other files required
• Can be completed independently

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• storage.googleapis.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "storage.googleapis.com"

See Network Configuration for more information.

📊 User experience analysis by Delight · 52.7 AIC · ⌖ 8.45 AIC · ⊞ 8.4K · ◷

• expires on Jul 12, 2026, 7:54 AM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Delight.

A newer discussion is available at Discussion #44776.