Skip to content

skills: restore bang-escape warning (bug still fires in CI harness) - #318

Merged
max-sixty merged 1 commit into
mainfrom
hourly/review-24641241367
Apr 20, 2026
Merged

max-sixty merged 1 commit into
mainfrom
hourly/review-24641241367

Conversation

@tend-agent

Copy link
Copy Markdown
Collaborator

PR #317 removed the !-escape heredoc warning from running-in-ci citing a local verification that Claude Code no longer rewrites ! in heredocs. That verification doesn't hold in the GitHub Actions CI harness (Claude Code 2.1.114). The Bash tool still rewrites every ! to \! in this run.

Reproduction in this run (24641241367):

$ cat << 'OUTER'
unquoted-1: Hello!
OUTER
unquoted-1: Hello\!

$ cat << 'INNER'
single-2: Hello!
INNER
single-2: Hello\!

$ echo 'plain-single: select(.x != "y")'
plain-single: select(.x \!= "y")

$ echo "plain-double: select(.x != \"y\")"
plain-double: select(.x \!= "y")

All four contexts — <<EOF, << 'EOF', plain single-quoted, and double-quoted — still rewrite the character. The bug isn't heredoc-specific; it's at the Bash-tool preprocessor layer.

Why this matters: the removed warning was the guard against bot comments shipping with literal \! corruption. With it gone, a future bot writing "Thanks for the suggestion!" via gh ... --body will publish "Thanks for the suggestion\!" — the exact user-visible failure mode the warning prevented. No corrupted comments have slipped through in the past ~4 hours since #317 merged, but that's luck, not correctness.

Changes:

  • running-in-ci: restore the !-escape paragraph; expand it to cover plain single/double-quoted args (the case that bites --jq 'select(.x != …)' commands — already hit 3 times, recorded in the evidence gist).
  • review-reviewers: restore "unescaped bangs" to the corruption-signals list.

Gate assessment:

  • Evidence level: High / Critical. Structural bug in the tool layer; reproduction is deterministic (4/4 contexts fail in this run).
  • Occurrences: 1 structural reproduction + 3 cumulative --jq occurrences in the evidence gist.
  • Change type: Targeted fix (restore removed paragraph, one-word addition in review-reviewers).
  • Both gates pass.

Caveat for the maintainer: if the bug really is fixed in your local Claude Code but not in the CI-pinned version, the right long-term fix may be bumping whatever version the runner installs rather than keeping workaround guidance. Either way, removing the warning while the bug still fires in CI was premature.

PR #317 dropped the heredoc bang-escape warning on the premise that the
underlying Claude Code bug was fixed, but the bug still reproduces in
this CI harness (Claude Code 2.1.114 on GitHub Actions runners) — every
context tested, including both heredoc forms, plain single-quoted, and
double-quoted args, still rewrites `!` to `\!`. Without the warning, a
bot comment containing an exclamation mark would ship corrupted. Restore
the paragraph and broaden it to cover plain-argument `--jq` filters
(where `!=` has bitten three times). Also re-add "unescaped bangs" to
the review-reviewers corruption signals list.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@tend-agent tend-agent added the claude-behavior Behavioral issues found by review-reviewers label Apr 19, 2026
@tend-agent

Copy link
Copy Markdown
Collaborator Author

Confirmed in the wild — the corruption this PR predicted has now shipped publicly.

tend-triage run 24641926108 on max-sixty/worktrunk posted issues/2309#issuecomment-4277108328 at 2026-04-19T23:53:23Z. The rendered comment opens Thanks for the detailed report\! — a literal backslash before the bang, exactly the failure mode this PR's body warns about. Same pipeline: model wrote cat > /tmp/issue-2309-comment.md << 'EOF' followed by plain Thanks for the detailed report!, then gh issue comment --body-file. Single-quoted heredoc, plain bang in the model's tool input (verified in the run's session JSONL) — yet the file landed with the backslash, and the comment was posted as written.

So the count is now 1 structural reproduction (the prior run on tend) + 3 cumulative --jq occurrences in the evidence gist + 1 corrupted public comment on a community-reported bug (worktrunk #2309 was filed by nicolasff, a non-bot reporter). The "no corrupted comments slipped through" caveat in the PR body no longer holds; the gap closed within ~6 hours of #317 merging.

Live demonstration also fired in this run while I was scanning bot output — a Python heredoc with if n != 406: was rewritten to if n \!= 406: and crashed with SyntaxError: unexpected character after line continuation character. Same Bash-tool layer, same rewrite, in the very environment running the analysis.

Recording the worktrunk #2309 incident in the worktrunk evidence gist under counter bot comment posted with literal \! corruption.

@max-sixty
max-sixty merged commit a93075b into main Apr 20, 2026
2 of 3 checks passed
@max-sixty
max-sixty deleted the hourly/review-24641241367 branch April 20, 2026 02:30
max-sixty pushed a commit that referenced this pull request Apr 21, 2026
## Summary

Extend the bang-escape workaround in `running-in-ci` to cover PR and
issue
titles. PR #318 restored the warning for comment bodies (via
`--body-file`);
titles are still uncovered because `gh pr create` has no `--title-file`
flag,
so a conventional-commits breaking-change marker in a title like
`feat(hooks)` followed by `!:` gets preprocessor-rewritten and ships
with
a literal backslash before the bang.

## Evidence

First wild post-#318 occurrence: `worktrunk-bot` opened

[max-sixty/worktrunk#2359](max-sixty/worktrunk#2359)
at 2026-04-21T18:40:19Z. Raw API confirms the title is stored with a
backslash before the bang:

```
$ gh api repos/max-sixty/worktrunk/pulls/2359 --jq '.title' | cat -A
feat(hooks)\!: rename pre-start/post-start to pre-create/post-create$
```

The corresponding commit message on the same branch is clean —
`git commit` uses `-F`/file internally, so the `!` survives there.
Only `gh pr create --title "feat(hooks)!: ..."` hit the preprocessor.

## Fix

Add one paragraph after the existing `--body-file` guidance documenting
the
command-substitution workaround. `$(cat /tmp/pr-title.txt)` is evaluated
by
bash after the preprocessor's string scan, so the bang in the file stays
literal. Same pattern covers `gh pr edit --title`, `gh issue create
--title`,
and `gh issue edit --title`.

## Gate assessment

- **Evidence level**: Critical — public-facing PR title corruption, 1
occurrence (#2359).
- **Cumulative this month**: prior occurrence was in a comment body

([worktrunk#2309](max-sixty/worktrunk#2309 (comment)))
  which PR #318 fixed; this is a different shape not yet covered.
- **Structural vs. stochastic**: structural. Any bot PR with `!` in its
title
  reproduces deterministically until the skill documents the workaround.
- **Change type**: targeted fix (one paragraph, no structural changes).
- **Both gates pass**: Critical + structural + proportionate fix.

Evidence gist:
[review-reviewers evidence: max-sixty/worktrunk
2026-04](https://gist.github.com/44ab1483b29406255dd36141650c894d).

Triggering run:
https://github.com/max-sixty/tend/actions/runs/24741474472

Co-authored-by: continuous-bot <269947486+continuous-bot@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
max-sixty pushed a commit that referenced this pull request May 4, 2026
… `!` in a comment body (#381)

## Problem

`tend-triage` run
[25295135366](https://github.com/max-sixty/worktrunk/actions/runs/25295135366)
on `max-sixty/worktrunk` shipped a public comment ([issue #2564 comment
4367561809](max-sixty/worktrunk#2564 (comment)))
with two simultaneous corruption signals:

1. A markdown link with literal `${GITHUB_REPOSITORY}` that did not
interpolate:
`[src/main.rs:1300](https://github.com/${GITHUB_REPOSITORY}/blob/main/src/main.rs#L1300)`.
The link is broken — clicks 404.
2. Literal `\!` backslash-bang corruption on `debug_assert\!` and
`bail\!` (Rust syntax that should render as `debug_assert!` / `bail!`).

Session-log inspection confirmed the bot used `gh issue comment 2564
--body "$(cat <<'EOF' ... EOF)"`. The single-quoted heredoc form fails
on **both** axes: the bash preprocessor's `!` → `\!` rewrite still fires
(it operates at the string level before bash parses the heredoc), *and*
`${GITHUB_REPOSITORY}` does not expand (single-quoted heredoc disables
interpolation).

## Why the existing guidance didn't catch it

`running-in-ci` already has two relevant warnings:

- The `$GITHUB_REPOSITORY` paragraph offers two ways to fix unexpanded
variables: unquoted `<<EOF` *or* the Write tool.
- The `!` paragraph mandates the Write tool when the body contains an
exclamation mark.

Neither paragraph addresses the **joint case** — when a body contains
both a `!` and a `${GITHUB_REPOSITORY}` reference. Unquoted `<<EOF`
fixes the variable but not the `!` (preprocessor still mangles).
Write-tool composition fixes the `!` but not the variable (Write does
not interpolate). The bot picked single-quoted `<<'EOF'`, which fails
both.

The `!` rule says "use Write tool"; the URL rule offers Write tool as an
alternative — but neither tells the bot how to embed
`$GITHUB_REPOSITORY` in a Write-tool-composed file. So when both gotchas
applied at once, neither rule's recipe was complete.

## Fix

One sentence added to the `$GITHUB_REPOSITORY` paragraph: when the body
contains a `!` (Write mandatory), write an `OWNER_REPO` placeholder into
the file and substitute via `sed -i "s|OWNER_REPO|$GITHUB_REPOSITORY|g"
/tmp/body.md` before `gh ... --body-file`. Plus an explicit warning that
single-quoted `<<'EOF'` with both a `!` and `${GITHUB_REPOSITORY}` in
the body fails on both counts.

The `sed` recipe is preprocessor-safe: `$GITHUB_REPOSITORY` is
`owner/repo` (no `!`), and the sed command itself contains no `!` (uses
`|` as delimiter because `/` appears in the value).

## Gate assessment

- **Confidence**: Critical — clearly wrong public output (broken
markdown link + visible backslash escapes shown to a community
reporter). 1 occurrence sufficient per Gate 1.
- **Magnitude**: Targeted fix to existing guidance, one sentence added.
Normal evidence bar.
- **Structural vs stochastic**: **Structural.** The same conditions
reproduce — the existing guidance offers Write-tool and unquoted-heredoc
as alternatives without addressing the joint case. A bot facing a body
with both `!` and `${GITHUB_REPOSITORY}` will plausibly pick `<<'EOF'`
again.
- **Recurrence history**: 4 prior `\!`-shipping incidents in the last
month's evidence gist (issue #2309 comment, PR #2359 title, PR #2361
title, the `<\!--` HTML-comment case). The `${GITHUB_REPOSITORY}`
non-expansion case is new this run. Both gates pass.

## Evidence

- Bot comment:
max-sixty/worktrunk#2564 (comment)
- Triage run:
https://github.com/max-sixty/worktrunk/actions/runs/25295135366
- Evidence gist (this month):
https://gist.github.com/9675d1510da32c68a68c1d45137b21e0
- Prior `\!` shipping incident write-up: tend [PR
#318](#318)

---------

Co-authored-by: tend-agent <270458913+tend-agent@users.noreply.github.com>
tend-agent added a commit that referenced this pull request May 10, 2026
Remove links to specific past PRs (#154, #212, #263, #203, #268, #392, #375, #305, #318) from review-reviewers and running-in-ci, restating each as a structural rule (what shape is accepted, what shape is not). Extend "Authoring skills" in the root CLAUDE.md with a parallel "No specific past-case references" bullet so future skill edits don't re-introduce them.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
max-sixty pushed a commit that referenced this pull request May 10, 2026
## Summary

Drops individual past-case pointers from skills and adds a CLAUDE.md
rule to keep them out going forward.

- `plugins/tend-ci-runner/skills/review-reviewers/SKILL.md`: rewrites
the four Non-issue entries (including the new cross-repo `CheckSuite`
mark-reads entry) as structural rules — "what shape is accepted, what
shape is not" — without naming the rejected PRs as precedent.
- `plugins/tend-ci-runner/skills/running-in-ci/SKILL.md`: removes the
`#305` pointer from the rollup-poll explanation and the `PR #318`
reference from the line-wrapping example body.
- `CLAUDE.md`: adds a "No specific past-case references" bullet to the
existing "Authoring skills" list, parallel to the existing "No specific
past-run references" rule.

The cross-repo upstream-bug references to `anthropics/claude-code#37157`
in `running-in-ci` and `review-runs` are left in place — they track an
open external bug whose resolution removes the workaround they justify,
which is forward-looking rather than precedent. Happy to remove those
too if you'd prefer no cross-repo issue links at all.

## Test plan

- [ ] Confirm the rewritten Non-issue entries still convey enough
constraint to prevent re-derivation of the rejected proposal shapes
(authorship-keyed guards on review/mention, pre-check filters keyed on
`reason`/`subject_type`/sender).
- [ ] Confirm the new CLAUDE.md bullet's wording matches the intended
bar.

---------

Co-authored-by: tend-agent <270458913+tend-agent@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

claude-behavior Behavioral issues found by review-reviewers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants