Skip to content

Capture the Claude Code CLI stderr the SDK discards - #8778

Open
izaitsevfb wants to merge 2 commits into
mainfrom
iz2/claude-capture-cli-stderr
Open

izaitsevfb wants to merge 2 commits into
mainfrom
iz2/claude-capture-cli-stderr

Conversation

@izaitsevfb

@izaitsevfb izaitsevfb commented Sep 10, 2026 •

Copy link
Copy Markdown
Contributor

✴️ iz2: @claude failures on pytorch/pytorch are currently undiagnosable — the CLI's error text is discarded before anyone can read it. This recovers it.

The problem

claude-code-action drives the CLI through the Agent SDK's query() without a stderr handler (base-action/src/run-claude-sdk.ts). When the child dies, all the action can report is:

SDK execution error: Error: Claude Code process exited with code 1

The child's own stderr is dropped, and on that path upload_execution_log writes a 2-byte file. On pytorch/pytorch, failures matching that shape (job duration 34–70s) ran 13 / 3 / 10 / 7 on Sep 6–9 against a 1–3/day baseline before Sep 6. No issue in anthropics/claude-code-action or anthropics/claude-code matches — unsurprising, since the error text isn't available to quote.

The change

path_to_claude_code_executable is a documented action input: when set, the action skips its own install and runs that binary. Pointing it at a thin wrapper recovers stderr without forking the action.

The CLI version is read from the pinned action revision at runtime rather than duplicated here, because the action hardcodes claudeCodeVersion as a constant in run.ts (v1.0.141 → 2.1.169, v1.0.220 → 2.1.267); a copy in this file would go stale on a pin bump.

The wrapper is an exec form, so the SDK's child is the CLI — no extra PID, signals and exit codes pass through, and the streamed-stdin prompt is untouched.

Where the stderr goes

Verbatim to a private S3 prefix — not to the job log, and not redacted.

The CLI runs with live AWS credentials in its environment, so pytorch/pytorch's public job logs are the wrong destination: they would require a scrubber, which is lossy on the exact text being read and cannot be shown to be complete.

ossci-raw-job-status grants public read via an explicit prefix allowlist — log/, review-logs/, additional_info/, test_run/, test_run_summary/, failed_test_runs/, ossci_tutorials_stats/. Anything outside it is account-only, so claude_cli_stderr/ is private by construction. Note this is deliberately not under review-logs/, where the execution log goes: that prefix is public.

This also captures the whole stream rather than a truncated tail; the job log carries only a pointer to the object.

Dependency

Needs s3:PutObject on arn:aws:s3:::ossci-raw-job-status/claude_cli_stderr/* added to the gha_workflow_claude_code inline policy (claude_code_s3_upload), which is scoped per prefix and currently covers review-logs/, claude_code_usage/, the autorevert-advisor prefixes and pr_review_*. Until that grant lands the upload fails and the step is a no-op — it is continue-on-error, so nothing else is affected.

Blast radius

This workflow runs every @claude in the pytorch, meta-pytorch and executorch repos, so the setup step is best-effort by construction: continue-on-error, with the action input gated on steps.stderr-capture.outcome == 'success'. Any failure — version lookup, download, install, or a binary not reporting the expected version — leaves the input empty and the action installs and runs the CLI exactly as it does today. The install is bounded (timeout-minutes: 4, curl --max-time) so a stalled download cannot consume the invocation's budget.

Precise scope of that guarantee: it covers a failure of the setup step. It does not cover a wrapper misbehaving once Claude is already running, and it does not undo a partial install.

Test plan

Exercised end-to-end against a mirror repo whose caller was temporarily pinned to this branch:

control result
Bogus CLI flag, forcing a fast failure job failed; capture contained error: unknown option '--iz2-nonexistent-flag' — a sub-second exit-1, matching the production shape, with the real text recovered
Normal @claude review succeeded, 25s
Version lookup broken via a bogus action SHA job succeeded, with the action printing its own Installing Claude Code v2.1.169... — confirming the input was empty and default behaviour restored
Both re-run against the exec wrapper crash captured; normal review succeeded in 16s and posted a completed task

Those runs exercise the capture mechanism — wrapper, version lookup, and fallback — which the S3 destination does not change.

Plus a local harness on wrapper generation: exit-code propagation through exec, streamed stdin surviving the wrapper, and a runner path containing a space.

Known residual

The action SHA appears twice — the uses: line and CLAUDE_ACTION_SHA. Bumping only one installs a CLI the action does not expect, and both configurations pass the version-format and executable checks, so neither falls back. A comment binds them; a lint asserting they match would be the mechanical fix, and I can add one here if reviewers prefer that to a follow-up.

claude-code-action drives the CLI through the Agent SDK query() with no stderr
handler, so when the child dies the action can only report "Claude Code process
exited with code 1" and the child error text is lost; the uploaded execution log
is 2 bytes on that path. pytorch/pytorch has hit this 1-13 times a day since
2026-09-06 and not one instance is diagnosable or reportable upstream.

path_to_claude_code_executable is a documented action input, so a thin wrapper
recovers the stderr without forking the action. The CLI version is read from the
pinned action revision at runtime rather than duplicated, because the action
hardcodes claudeCodeVersion as a constant and a copy here would go stale on a
pin bump.

Best-effort by construction: any failure in the setup step leaves the input
empty, the action installs and runs the CLI as it does today, and the run is
unaffected. This workflow is shared by pytorch, meta-pytorch and executorch, so
a bug here has to degrade to "no capture" rather than "no @claude".

Captured stderr is printed only when Claude failed, since these logs are public
on pytorch/pytorch and the failing case is the one nobody can see today.
Credential shapes are redacted and workflow-command parsing is stopped around
the output.
@vercel

vercel Bot commented Sep 10, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
torchci Ignored Ignored Sep 10, 2026 11:46pm UTC

Request Review

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 10, 2026
Redaction was the wrong tool. The CLI runs with live AWS credentials in its
environment, so printing raw stderr into a pytorch/pytorch job log (public)
needed a scrubber -- and a scrubber is lossy on exactly the text we are trying
to read, and unfalsifiable: you cannot demonstrate it caught everything. The
first version of it here missed AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN
outright, since neither has a matchable prefix.

Writing somewhere non-public removes the need for one. ossci-raw-job-status
grants public read by an explicit prefix allowlist (log/, review-logs/,
additional_info/, test_run/, test_run_summary/, failed_test_runs/,
ossci_tutorials_stats/), so claude_cli_stderr/ is private by construction.
Deliberately NOT under review-logs/, where the execution log goes -- that prefix
is public.

Also gets the whole stream rather than a 200-line tail, and drops it from the
job log entirely; only a pointer to the object is printed.

Depends on s3:PutObject for claude_cli_stderr/* being added to the
gha_workflow_claude_code inline policy, which is scoped per prefix. Until that
lands the upload fails and the step is a no-op.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant