Capture the Claude Code CLI stderr the SDK discards - #8778
Open
izaitsevfb wants to merge 2 commits into
Open
izaitsevfb wants to merge 2 commits into
izaitsevfb wants to merge 2 commits into
Conversation
claude-code-action drives the CLI through the Agent SDK query() with no stderr handler, so when the child dies the action can only report "Claude Code process exited with code 1" and the child error text is lost; the uploaded execution log is 2 bytes on that path. pytorch/pytorch has hit this 1-13 times a day since 2026-09-06 and not one instance is diagnosable or reportable upstream. path_to_claude_code_executable is a documented action input, so a thin wrapper recovers the stderr without forking the action. The CLI version is read from the pinned action revision at runtime rather than duplicated, because the action hardcodes claudeCodeVersion as a constant and a copy here would go stale on a pin bump. Best-effort by construction: any failure in the setup step leaves the input empty, the action installs and runs the CLI as it does today, and the run is unaffected. This workflow is shared by pytorch, meta-pytorch and executorch, so a bug here has to degrade to "no capture" rather than "no @claude". Captured stderr is printed only when Claude failed, since these logs are public on pytorch/pytorch and the failing case is the one nobody can see today. Credential shapes are redacted and workflow-command parsing is stopped around the output.
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
Redaction was the wrong tool. The CLI runs with live AWS credentials in its environment, so printing raw stderr into a pytorch/pytorch job log (public) needed a scrubber -- and a scrubber is lossy on exactly the text we are trying to read, and unfalsifiable: you cannot demonstrate it caught everything. The first version of it here missed AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN outright, since neither has a matchable prefix. Writing somewhere non-public removes the need for one. ossci-raw-job-status grants public read by an explicit prefix allowlist (log/, review-logs/, additional_info/, test_run/, test_run_summary/, failed_test_runs/, ossci_tutorials_stats/), so claude_cli_stderr/ is private by construction. Deliberately NOT under review-logs/, where the execution log goes -- that prefix is public. Also gets the whole stream rather than a 200-line tail, and drops it from the job log entirely; only a pointer to the object is printed. Depends on s3:PutObject for claude_cli_stderr/* being added to the gha_workflow_claude_code inline policy, which is scoped per prefix. Until that lands the upload fails and the step is a no-op.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
✴️ iz2:
@claudefailures on pytorch/pytorch are currently undiagnosable — the CLI's error text is discarded before anyone can read it. This recovers it.The problem
claude-code-actiondrives the CLI through the Agent SDK'squery()without astderrhandler (base-action/src/run-claude-sdk.ts). When the child dies, all the action can report is:The child's own stderr is dropped, and on that path
upload_execution_logwrites a 2-byte file. On pytorch/pytorch, failures matching that shape (job duration 34–70s) ran 13 / 3 / 10 / 7 on Sep 6–9 against a 1–3/day baseline before Sep 6. No issue inanthropics/claude-code-actionoranthropics/claude-codematches — unsurprising, since the error text isn't available to quote.The change
path_to_claude_code_executableis a documented action input: when set, the action skips its own install and runs that binary. Pointing it at a thin wrapper recovers stderr without forking the action.The CLI version is read from the pinned action revision at runtime rather than duplicated here, because the action hardcodes
claudeCodeVersionas a constant inrun.ts(v1.0.141→2.1.169,v1.0.220→2.1.267); a copy in this file would go stale on a pin bump.The wrapper is an
execform, so the SDK's child is the CLI — no extra PID, signals and exit codes pass through, and the streamed-stdin prompt is untouched.Where the stderr goes
Verbatim to a private S3 prefix — not to the job log, and not redacted.
The CLI runs with live AWS credentials in its environment, so pytorch/pytorch's public job logs are the wrong destination: they would require a scrubber, which is lossy on the exact text being read and cannot be shown to be complete.
ossci-raw-job-statusgrants public read via an explicit prefix allowlist —log/,review-logs/,additional_info/,test_run/,test_run_summary/,failed_test_runs/,ossci_tutorials_stats/. Anything outside it is account-only, soclaude_cli_stderr/is private by construction. Note this is deliberately not underreview-logs/, where the execution log goes: that prefix is public.This also captures the whole stream rather than a truncated tail; the job log carries only a pointer to the object.
Dependency
Needs
s3:PutObjectonarn:aws:s3:::ossci-raw-job-status/claude_cli_stderr/*added to thegha_workflow_claude_codeinline policy (claude_code_s3_upload), which is scoped per prefix and currently coversreview-logs/,claude_code_usage/, the autorevert-advisor prefixes andpr_review_*. Until that grant lands the upload fails and the step is a no-op — it iscontinue-on-error, so nothing else is affected.Blast radius
This workflow runs every
@claudein the pytorch, meta-pytorch and executorch repos, so the setup step is best-effort by construction:continue-on-error, with the action input gated onsteps.stderr-capture.outcome == 'success'. Any failure — version lookup, download, install, or a binary not reporting the expected version — leaves the input empty and the action installs and runs the CLI exactly as it does today. The install is bounded (timeout-minutes: 4,curl --max-time) so a stalled download cannot consume the invocation's budget.Precise scope of that guarantee: it covers a failure of the setup step. It does not cover a wrapper misbehaving once Claude is already running, and it does not undo a partial install.
Test plan
Exercised end-to-end against a mirror repo whose caller was temporarily pinned to this branch:
error: unknown option '--iz2-nonexistent-flag'— a sub-second exit-1, matching the production shape, with the real text recovered@claudereviewInstalling Claude Code v2.1.169...— confirming the input was empty and default behaviour restoredexecwrapperThose runs exercise the capture mechanism — wrapper, version lookup, and fallback — which the S3 destination does not change.
Plus a local harness on wrapper generation: exit-code propagation through
exec, streamed stdin surviving the wrapper, and a runner path containing a space.Known residual
The action SHA appears twice — the
uses:line andCLAUDE_ACTION_SHA. Bumping only one installs a CLI the action does not expect, and both configurations pass the version-format and executable checks, so neither falls back. A comment binds them; a lint asserting they match would be the mechanical fix, and I can add one here if reviewers prefer that to a follow-up.