Run git cleanup's branch comparisons in parallel - #90
Open
mattmenefee wants to merge 1 commit into
Open
Conversation
mattmenefee
force-pushed
the
speed-up-git-cleanup
branch
3 times, most recently
from
August 4, 2026 21:14
692f4f4 to
901ad59
Compare
`git pc` took around thirty seconds in a repository holding a few hundred local branches, nearly all of it inside `cleanup` rather than the pull. The loop spends one `git cherry` per branch, and each of those rebuilds a patch-id table covering every commit that branch sits behind the base — so a branch forked seven thousand commits ago costs more than ten times one forked last week, and that table is rebuilt from scratch for every branch. On top of that, each iteration piped `git cherry` output into `grep` just to look for a leading `+`, paying a second process per branch for a test the shell can do itself. The comparisons never depended on one another, so they now fan out across `xargs -P` sized to the machine's processors. Workers only classify and print a verdict; deletion stays serial and happens after every verdict is in, so no ref vanishes while another process is still walking the ref table. Verdicts are sorted by ref name before anything is deleted, since workers finish out of order and the list of deleted branches is this command's audit trail. The `grep` is gone in favor of scanning the output words for a bare `+`, which is exact as long as `git cherry` stays unadorned — the comments record that constraint along with why branches travel NUL-delimited and what the `cleanup-worker` argument is guarding against. Splitting classification from deletion adds failure modes the serial loop could not have, so four guards come with it. The reader re-checks the base, current and repository base branches before deleting, because "a protected branch never reaches the delete" used to be visible in a single loop and is now a claim about a whole pipeline. It also ignores any verdict whose ref is not under refs/heads/, so a torn line cannot arrive as a bare or half-formed name. A fan-out that exits non-zero deletes nothing rather than acting on a partial verdict list. And the worker count rejects zero, which means unbounded to GNU xargs, and caps at sixteen, since `getconf` reports host processors rather than any cgroup quota. One behavior does change. A symbolic ref under refs/heads/ used to be kept or deleted purely on name order: its comparison only failed if its target happened to sort ahead of it and had already been deleted, so the same repository answered differently depending on what the two were called. Deciding every branch against one pre-deletion snapshot makes that reachable consistently, and the answer chosen is to skip symbolic refs outright — a symref is not a branch, and which of the two names `git branch -D` would take is the kind of question this command should decline. Nothing else moves. Old and new produce byte-identical stdout, stderr and exit codes on the repository that prompted this, on a 370-branch repository where 250 branches are deleted in one run, and for branches merged by rebase, branches carrying merge commits, branch names containing quotes, a tag shadowing the base branch's name, an empty candidate set, a detached HEAD, a branch held by a linked worktree, a base living only on origin, and a base that does not resolve. The run drops from roughly twenty-three seconds to under four.
mattmenefee
force-pushed
the
speed-up-git-cleanup
branch
from
August 5, 2026 21:31
901ad59 to
7d0ba15
Compare
9 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
git pctook ~30 seconds in a repository with a few hundred local branches. Profiling showed nearly all of it insidecleanuprather than the pull — the pull itself is ~1sgit cherryper branch, and each one rebuilds a patch-id table covering every commit that branch sits behind the base. Cost scales with how stale a branch is: a branch forked ~7,000 commits ago measured 271ms against 22ms for one forked last week, and that table is rebuilt from scratch for every branchgit cherryoutput intogrepjust to test for a leading+, paying a second process per branch for something the shell can do itselfxargs -Psized to the machine's processors. Workers only classify and print a verdict; deletion stays serial and runs after every verdict is collected, so no ref disappears while another process is still walking the ref tablegrepis replaced by scanning the output words for a bare+. That is exact only whilegit cherrystays unadorned — adding-vwould put commit subjects in the stream and a subject containing a lone+would read as unmerged. The comments record that constraint, along with why branches travel NUL-delimited (ref names may contain quotes, whichxargswould otherwise treat as syntax) and what thecleanup-workerargument guards against (without itxargsbinds the first branch of every batch to$0, where the loop over$@never reaches it — roughly one branch in twenty would go silently uncompared)Splitting classification from deletion introduces failure modes the serial loop could not have, so four guards come with it:
refs/heads/, so a torn or truncated line cannot reachgit branch -Das a bare or half-formed name0(which means unbounded to GNU xargs) and caps at 16, sincegetconfreports host processors rather than any cgroup quotaOne deliberate behavior change
A symbolic ref under
refs/heads/used to be kept or deleted purely on name order. Its comparison only failed if its target happened to sort ahead of it and had already been deleted — soaaa-alias→zzz-targetwas deleted, whilezzz-alias→aaa-targetsurvived, in otherwise identical repositories. Deciding every branch against one pre-deletion snapshot makes the case reachable consistently, and symbolic refs are now skipped outright: a symref is not a branch, and which of the two namesgit branch -Dwould take is the kind of question this command should decline.Everything else is unchanged. Given this alias once force-deleted every branch in a master-based repository, that was verified by differential testing rather than inspection.
Test plan
Both implementations were extracted from the actual parsed alias (
git config --get alias.cleanup) withgit branch -Dstubbed to anecho, then run side by side. Comparison iscmpon raw stdout and stderr plus exit code — not sorted, so ordering regressions cannot hide.unique merge commits'and"— handled identically (confirmed refs cannot contain spaces, so the space-delimited verdict format is safe)origin— identical0, normal, and oversizedgetconfoutputrefs/heads/, both name orderings, with real deletions inspecting.git/refs/heads/rather thanfor-each-ref(which does not list a dangling symref)git branch -Drefuses, and the exit code matches the serial versionxargsaborting mid-run, andxargsabsent entirely — both delete nothing, reportcomparison failed, and exit 1tr '\n' '\0'andxargs -0 -Pbehave as required on BSD/macOS, and that git's config parser passes the escapes through without introducing literal newlines or tabsReview
Reviewed by four expert agents in isolated worktrees. Findings folded in:
cleanup-workercomment stated the wrong rationale and the wrong failure mode. Corrected: it guards against silent per-batch loss of the first branch, not an empty-input edge case that never triggersx;touch$IFS/tmp/pwn;y, backtick and$(...)payloads,--force,-D, 6000-byte names, a hostile clone controllingorigin/HEAD) could not achieve injection or forge a verdict. Their protected-branch recheck and worker-count hardening are included abovegit branch -Dstructurally cannot observe refs disappearing mid-run, which is how the symref divergence above went unnoticed; also that a partial fan-out failed silently while returning 0. Both are fixed and covered by real-deletion tests. Their remaining checks came back clean: 36 hostile-but-legal ref names, 3000 branches againstARG_MAX, batch boundaries at 19/20/21/40/400, octopus and orphan-root merges, hostileIFS, and every base-resolution path