reuse transcript blobs across turn-end checkpoints

[Soph]

This should address some of the issues reported with codex hitting hook timeouts, for example #957

Summary

Stop hooks were taking ~30 s on 18 MB image-heavy transcripts (reported against Codex, but the code path is agent-agnostic). Two root causes inside finalizeAllTurnCheckpoints:

1. For each of N turn-checkpoints, the full redacted transcript was re-chunked and re-zlib-compressed, doubled when v1+v2 dual-write is enabled. Pure-Go zlib on ~18 MB of base64 image data is the bottleneck; go-git v6 does not short-circuit before compression when the loose object already exists on disk.
2. replaceTranscript / updateCommittedFullTranscript unconditionally rewrote all transcript blobs even when the stored content was identical.

Changes

• PrecomputedTranscriptBlobs new type on UpdateCommittedOptions: chunk blob hashes + content-hash blob, computed once. Content-addressed hashes are identical for v1 (full.jsonl) and v2 (raw_transcript) paths, so one precompute serves both stores.
• PrecomputeTranscriptBlobs(ctx, repo, transcript, agentType) constructor in checkpoint/committed.go.
• finalizeAllTurnCheckpoints now calls PrecomputeTranscriptBlobs once above the per-checkpoint loop and threads the result into every UpdateCommittedOptions. On precompute failure the loop falls back to the old per-checkpoint path (best-effort, logged).
• v1 replaceTranscript (committed.go): accepts precomputed blobs; short-circuits when existing content_hash.txt matches the new sha256.
• v2 updateCommittedFullTranscript (v2_committed.go): same short-circuit against raw_transcript_hash.txt; writeTranscriptBlobs + new writeContentHashFromPrecompute accept precomputed blobs.

Expected effect on a 30 s, 18 MB, 3-checkpoint finalize

• Chunk + zlib of redacted transcript: 2N → 1 (v1 + v2, one pass).
• Content-hash blob: 2N → 1, dropping to 0 on no-op re-finalize.
• Estimated ~10–15 s removed from the reported 30 s case.
• compactTranscriptForV2 is genuinely per-checkpoint (per-checkpoint startLine), so it stays in the loop — not addressed here.

Risk notes

• Short-circuit only fires on byte-identical transcript — redaction output is deterministic so this is safe.
• Fallback path preserved: if PrecomputeTranscriptBlobs fails, every checkpoint re-chunks and re-blobs as before.
• No behavior changes visible from the public API (ReadSessionContent etc.) — verified by TestUpdateCommitted_PreservesMetadata + roundtrip tests.
• No change to on-disk tree shape, ref layout, or commit-message trailers.

Potential next steps

In decreasing ratio order, not shipped in this PR:

1. Parse-on-first-use transcript cache. lifecycle.go handleLifecycleTurnEnd runs ExtractPrompts + ExtractAllModifiedFiles + CalculateTokenUsage + parseTranscriptForCheckpointUUID — each parses the same bytes independently. A sync.Once-guarded shared []TranscriptLine would collapse 3–4 parses into 1 without touching public agent interfaces. Estimated ~3–5 s/turn on image-heavy transcripts. Every agent, every turn — not just stop.
2. Hoist compact.Compact if we can amortize per-startLine. compactTranscriptForV2 is currently per-checkpoint because of startLine. Worth a follow-up to check whether the expensive part (the JSONL walk) can run once with all chunks shared, and only the startLine slice differs.
3. In-memory-bytes overloads for ExtractPrompts / parseTranscriptForCheckpointUUID. Both currently re-read the same transcript file from disk. ~0.5–1 s/turn. Subsumed by (1) if it lands.
4. Audit .entire/metadata/<sid>/full.jsonl writers/readers. lifecycle.go:357 writes ~18 MB to disk every turn, but finalize re-reads the original agent transcript, not this copy. Some resume/explain flows may still depend on it — scope before deleting.
5. Streaming / lazy JSONL parser with json.RawMessage for image content blocks. Biggest win for 18 MB base64-heavy transcripts but largest diff; diminishing returns if (1) lands first.

Test plan

• Run mise run check to confirm fmt/lint/test:ci pass on this branch.
• Sanity-check on a local repo: start a session, run multiple turn-end commits, git stop, confirm entire/checkpoints/v1 tree and /full/current tree both contain the finalized transcript and content hash.
• If possible, reproduce the customer's 18 MB, multi-image, 3-checkpoint scenario and measure stop-hook wall time before/after.

Want me to push the branch and open the PR with this as the body?

---

Note

Medium Risk
Touches checkpoint persistence logic for both v1 and v2, adding new short-circuit paths based on stored content hashes; mistakes could leave transcripts stale or mismatched across refs. Changes are localized and covered by new roundtrip/short-circuit tests plus a fallback-to-old-behavior path if precompute fails.

Overview
Speeds up turn-end checkpoint finalization by precomputing transcript chunk blobs once and reusing their content-addressed hashes across all UpdateCommitted calls (and across v1+v2 dual-writes).

Adds a PrecomputedTranscriptBlobs option and PrecomputeTranscriptBlobs helper, updates v1 replaceTranscript and v2 /full/current updates to skip re-chunking/re-compressing when the transcript content is unchanged (via existing content-hash files), and threads the precomputed blobs through finalizeAllTurnCheckpoints. Includes new tests covering precompute reuse and the content-hash short-circuit behavior for both v1 and v2.

^{Reviewed by Cursor Bugbot for commit da55218. Configure here.}

[Copilot]

Pull request overview

This PR improves stop-hook performance by avoiding repeated transcript chunking/zlib work across multiple turn-end checkpoint finalizations, and by short-circuiting transcript rewrites when content is unchanged. It fits into the checkpoint “finalize” flow by reducing redundant blob creation and reuse across both v1 (entire/checkpoints/v1) and v2 (refs/entire/*) storage paths.

Changes:

• Precompute transcript chunk blob hashes + content-hash blob once per turn and reuse across all UpdateCommitted calls in finalizeAllTurnCheckpoints.
• Add v1/v2 short-circuiting based on stored content-hash files to skip chunking/blob creation when transcript bytes are identical.
• Add tests for precomputed blob reuse and no-op updates (roundtrip + stability checks).

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
cmd/entire/cli/strategy/manual_commit_hooks.go	Precomputes transcript blobs once and threads them into per-checkpoint UpdateCommittedOptions.
cmd/entire/cli/checkpoint/checkpoint.go	Extends UpdateCommittedOptions with PrecomputedBlobs and defines PrecomputedTranscriptBlobs.
cmd/entire/cli/checkpoint/committed.go	Adds v1 short-circuiting + uses precomputed chunk/content-hash blobs; introduces PrecomputeTranscriptBlobs.
cmd/entire/cli/checkpoint/v2_committed.go	Adds v2 short-circuiting and reuses precomputed chunk/content-hash blobs.
cmd/entire/cli/checkpoint/committed_update_test.go	Adds v1 tests around precompute roundtrip and “short-circuit” scenarios.
cmd/entire/cli/checkpoint/v2_precompute_test.go	Adds v2 tests around precompute roundtrip and “short-circuit” scenarios.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit da55218. Configure here.}