Skip to content

Blind exercise runs 14/15: findings dismissed for #576 #579

Description

@SUaDtL

Two adversarial blind exercises drove the shipped release skill as an operator's agent would. Nine HIGHs were fixed in #576; these are the remainder, dismissed there to land the PR and tracked here.

Repository state, not a skill defect — but it blocks releases today

ca cannot be released until the tag series is reconciled. LAST_TAG is v2.8.13 while plugins/ca/.claude-plugin/plugin.json reads 2.11.0, and CHANGELOG.md already carries [2.9.1] through [2.11.0]. Versions shipped without tags.

Consequences, both measured:

  • The commit window v2.8.13..HEAD spans commits already published under those versions, so a changelog roll would re-publish entries that already shipped.
  • The CHANGELOG: footer gate flags 52 commits, of which 38 predate the published [2.11.0] section. Their only stated remedy is amending or rebasing published history.

#576 added a STOP that detects the gap and refuses rather than composing a duplicate section. Clearing it is a deliberate maintainer action: tag the missing versions at the commits they shipped from, so LAST_TAG and the manifest agree again.

Proof gate is stale (expected)

.codearbiter/reports/agent-lane-proof.json records run 13's hash. The skill changed materially in #576, so check_skill_proof_fresh.py is RED. It is a declared pre-tag command, not a CI job, so it blocks the next /ca:release rather than the merge — which is the gate working. Re-record it by running a blind exercise against the shipped skill and updating the artifact.

Skill findings not taken

  • MEDIUM — CONTEXT.md present-but-silent is an unhandled third state. Pre-flight reads it for the default-branch name and authorizes the git symbolic-ref fallback only when the file is ABSENT. This repo's CONTEXT.md exists and names no branch, which reads literally as a STOP. Both exercises guessed main via the fallback and flagged it.
  • MEDIUM — Phase 2 step 1 is a single ~2,400-word paragraph containing the only git tag in the lane, buried roughly 80% of the way in after the sentence "The tag command itself appears below". "Below" is the same paragraph. Highest-consequence step, least navigable.
  • MEDIUM — Pre-flight contradicts itself on ordering. Bullet 3 requires a clean tree; bullet 6 runs rebuild, which can dirty it, then says to commit through commit-gate. Recoverable, but the phase violates its own earlier gate.
  • LOW — --latest[=false] is not runnable shell. Bracket notation inside a command template; pasted, gh receives a flag literally named --latest[=false].
  • LOW — declared pre-tag commands hardcode python3. The skill's own interpreter-resolution rule governs its helpers but does not reach operator-declared commands, which fail on exactly the Windows host the rule exists for.
  • LOW — medium-documents is cited with no path while every other include is fully pathed. It resolves, but only by guessing the sibling directory.
  • LOW — ${CLAUDE_PLUGIN_ROOT} is unquoted in ~20 invocations; a plugin root under a path containing spaces breaks all of them.

Note on method

Every HIGH after run 1 of this campaign has been in a SEAM rather than a mechanism, and three consecutive runs found their worst defect in the fix for the previous run. Run 15's top finding — eval "$(show-row …)" executing declared shell — was introduced by run 14's remediation one commit earlier. Worth keeping in view when scheduling the next pass: the loop is productive but does not obviously converge.

Refs #563, #564, #576.

Metadata

Metadata

Assignees

No one assigned

    Labels

    sev:medTribunal/triage: medium severity

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions