Git delivery watcher ignores its 5-minute interval and scans continuously

[nichenke]

🤖 Agent-filed: investigated and written by Claude Code on behalf of @nichenke. Every code citation below was verified against aa309c0, and the CPU numbers were measured on the reporter's machine.

git_delivery_watcher() is documented and coded to scan "every five minutes" but runs back-to-back, holding a sustained fraction of a CPU core indefinitely.

Mechanism

1. git_delivery_watcher() (token_meter/app.py:7081) waits with _git_delivery_wake.wait(GIT_DELIVERY_INTERVAL_S), where the interval is 5 * 60 (app.py:306).
2. publish_source_inventory() (app.py:3051) unconditionally calls _git_delivery_wake.set() (app.py:3075).
3. watcher() (app.py:9024) loops every 0.5s (time.sleep(0.5) at the loop tail, app.py:9160) and calls publish_source_inventory(sources) (app.py:9086) whenever refresh_known_source_activity() (app.py:7595) returns a new list object.

threading.Event.wait(timeout) returns immediately when the event is already set — verified on Python 3.9.6, where wait(300) on a set event returned in 0.0000s. So each scan is followed immediately by the next and the five-minute interval never applies.

Step 3 fires whenever the active session's transcript mtime has advanced, which is most 0.5s ticks while any agent session is streaming output. That frequency is inferred from the mtime comparison in refresh_known_source_activity; I did not instrument the per-tick rate.

The .set() call arrived with the Git activity feature (cc162d7, 2026-09-04). The 0.5s fast-path refresh predates it (27e4f76, 2026-08-14).

Impact

Continuous git rev-list --topo-order --reverse --max-count=2001 (services/git_delivery.py:484-485) plus --max-count=2001 reflog reads (:542), across up to MAX_REPOSITORIES = 50 repositories.

Measured: 137:01 of CPU time over 4:00:16 wall clock — 57% of one core — sustained, with git subprocesses spawning continuously and ~/.token-meter/git-delivery.sqlite3 rewritten every few seconds.

Removing the _git_delivery_wake.set() at app.py:3075 locally reduced sustained CPU from 57% to 32% of one core, and dropped git subprocess spawns from continuous to 2 per 40s.

Workload

Per-pass cost is bounded — eligible = candidate_rows[:MAX_REPOSITORIES] (git_delivery.py:625) caps a pass at 50 repositories and MAX_COMMITS_PER_SCAN caps it at 5,000 commits. The runaway is therefore frequency, not pass size. Concurrent session count is what drives frequency, because each session writing its transcript triggers another publish_source_inventory() and another .set().

The reporting machine runs:

Factor	Count	Bears on
Concurrent agent sessions	12 Claude Code CLI + 8 Codex app-servers	Trigger frequency — the driver of this bug
Git repositories under scanned roots	106	Every pass hits the 50-repo cap
Worktrees	170, plus 26 under .claude/worktrees/	Each resolves to its own repo_key, so they count as candidates
Claude transcripts	2,799 across 147 project directories, 1.3 GB	Inventory size; bears on #3, not on this bug
Codex session files	184	Same as above

A single-session user with a handful of repositories would trigger the scan rarely, and each pass would finish well under the caps — which may be why this has not surfaced before.

Suggested fix

Enforce an interval floor rather than dropping the early-wake behaviour, so source changes can still trigger a prompt scan without allowing back-to-back passes:

last_scan = 0.0
while True:
    ...
    remaining = GIT_DELIVERY_INTERVAL_S - (time.monotonic() - last_scan)
    if remaining > 0:
        _git_delivery_wake.wait(remaining)
        _git_delivery_wake.clear()
        continue
    git_delivery_service().scan(candidates)
    last_scan = time.monotonic()

Related

#3 (closed) covered the 0.5s watcher poll itself and estimated an intrinsic baseline near 34% of one core. This report is a separate mechanism: the Git delivery watcher's own interval being bypassed. The residual 32% remaining after the local workaround is consistent with #3's baseline estimate, but I did not profile it successfully (macOS sample output was unsymbolized), so I am not attributing it here.

Environment

• Revision aa309c0, macOS 26.6.2 arm64, Python 3.9.6 (system Python, via the com.token-meter.server LaunchAgent)

[bhavsarpratik]

@nichenke, the fix is ready in PR #31: #31. It addresses the concurrent-session trigger described here; repository/worktree count still affects per-pass work, but repeated inventory wakes can no longer bypass the five-minute scan floor.

Could you try it under the same large-cache workload and share the Token Meter server CPU plus Git subprocess frequency you observe?

Fresh isolated clone:

git clone --branch codex/fix-git-delivery-scan-floor --single-branch https://github.com/splunk/token-meter.git token-meter-issue-30
cd token-meter-issue-30
./scripts/install
curl -fsS http://127.0.0.1:8722/health
curl -fsS http://127.0.0.1:8722/menubar

Existing checkout with GitHub CLI:

gh pr checkout 31
./scripts/install
curl -fsS http://127.0.0.1:8722/health
curl -fsS http://127.0.0.1:8722/menubar

No sudo is needed. To restore the main-branch runtime afterward, switch back to main, pull it, and run ./scripts/install again.

[nichenke]

Tested #31 on the reporting machine. The fix works and I think it's ready to merge.

Measured the live pre-fix server first: 55% of one core over a 601s window, delivery database rewritten every 3.5s. That lines up with the 57% in the original report.

Then ran both revisions through the same harness, back to back under the same machine load. Each pass runs the real watcher() and git_delivery_watcher() threads against the machine's actual session corpus and repositories, with the ledger redirected to a temp directory, and wraps the subprocess runner to count git spawns exactly. 420s per pass:

	delivery scans	git subprocesses	CPU (1 core)
pre-fix	10	6348 (15.1/s)	84.6%
PR #31	1	2521 (6.0/s)	66.4%

One correction to my original report: a scan at this scale takes about 42s, so pre-fix the watcher was scanning every 42s continuously rather than at the wake rate. Scan duration was the limiter, not the wake signal. The effect is the same but the frequency I implied was too high.

Both passes include one cold scan against an empty ledger, which costs roughly 2500 spawns on its own. The nine additional warm scans in the pre-fix pass are what the floor removes.

Worth noting where the remaining cost sits: git subprocess CPU fell from 16.9% to 7.2% of a core, but the Python thread CPU only fell from 67.7% to 59.2%. The bulk of what is left is the 0.5s source watcher from #3, not git scanning. Absolute percentages are inflated because the harness ran alongside the live server; the scan and spawn counts are the reliable part.

Also filed #33 for something that turned up while verifying. The 50-candidate budget is mostly spent on deleted paths and duplicate worktrees.