Skip to content

fix(server): stop per-session toolsets on session delete - #3884

Open
EronWright wants to merge 1 commit into
docker:mainfrom
EronWright:contrib/stop-toolsets-on-delete
Open

fix(server): stop per-session toolsets on session delete#3884
EronWright wants to merge 1 commit into
docker:mainfrom
EronWright:contrib/stop-toolsets-on-delete

Conversation

@EronWright

@EronWright EronWright commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

fix(server): stop per-session toolsets on session delete

Problem

A session materialised via teamloader owns a team whose toolsets may hold
external resources — notably stdio MCP subprocesses. DeleteSession cancels the
runtime context but never calls team.StopToolSets, and BatchDeleteSessions has
the same gap (LocalRuntime.Close only stops background agents; StopToolSets was
otherwise reached only via the transient GetAgentToolCount). So a deleted session's
MCP subprocesses leak until the server process exits.

Fix

Track the per-session team on activeRuntimes and route both delete paths through
one scheduleTeardown helper. It keeps the entry in deletedSessions, waits for the
session's stream to drain, stops the toolsets, then drops the entry. Teardown runs in
a background goroutine that outlives the originating request, on a context detached
with WithoutCancel (so it still carries trace/logging values). The drain and
StopToolSets carry separate budgets — a session that exhausts the 5 min drain
must still get a live context to stop its toolsets with.

team is nil for attached runtimes (AttachRuntime), whose toolset lifecycle belongs
to the embedder — a no-op there.

Two things beyond the original revision, from review feedback:

  • BatchDeleteSessions is now WaitStopped-able. It previously never stored into
    deletedSessions, so WaitStopped returned nil immediately while teardown was
    still running. It now registers the same teardown as DeleteSession.
  • WaitStopped means "fully torn down". It waits on a completion channel closed
    after StopToolSets instead of polling the streaming mutex. This also closes an
    existing race where WaitStopped could win the TryLock, delete the entry and
    return nil before teardown had run.

Why this is safe

Teams are never shared between sessions: loadTeamWithConfig
teamloader.LoadWithConfig builds a fresh team and fresh toolset instances per call
(loaderdefaults.Opts() hands in a new toolsets.NewDefaultToolsetRegistry() each
time). The only process-wide singletons — plan, and todo with shared: true — do
not implement Stop, so StopToolSets cannot reach across sessions.

There is already precedent in-tree: GetAgentToolCount loads a throwaway team, starts
every toolset and unconditionally stops them while arbitrary sessions are live. If
teams or toolsets were shared, that endpoint would already be killing live sessions'
MCP servers on every call.

StopToolSets is also safe to call twice — Agent.StopToolSets skips
!toolSet.IsStarted(), and mcp.Toolset.Stop documents itself idempotent.

Test

Four new tests in pkg/server/session_manager_test.go, all of which fail without the
fix:

  • TestDeleteSession_StopsSessionToolSets — the leak itself.
  • TestBatchDeleteSessions_StopsSessionToolSets — the batch path, incl. WaitStopped.
  • TestWaitStopped_WaitsForToolSetTeardown — teardown parks on an in-flight stream and
    completes once it drains; WaitStopped returns only after the toolsets are stopped.
  • TestDeleteSession_StopsSessionToolSetsOnlyOnce — no double-stop.

task test and task lint green locally, plus go test ./pkg/server/ -race.
Verified against a multi-tenant host: the per-session subprocess count returns to zero
on delete and after server shutdown (previously it stayed pinned until exit).

Noted, not fixed here

  • LocalRuntime.Close never calls StopToolSets, so runtimeForSession's
    failed-construction defer run.Close() does not deliver on its "an unclosed runtime
    would leak its tool sets" comment. Real, but a pkg/runtime bug.
  • SessionManager has no Close/Shutdown, so toolsets of sessions that are never
    deleted still leak at process exit.
  • Team.StopToolSets is fail-fast: the first agent error aborts the loop, leaving
    later agents' toolsets running.

@EronWright
EronWright requested a review from a team as a code owner August 2, 2026 00:08
@aheritier aheritier added area/core Core agent runtime, session management kind/fix PR fixes a bug (maps to fix:). Use on PRs only. labels Aug 2, 2026
@aheritier

Copy link
Copy Markdown
Collaborator

The idea is interesting @EronWright Could you rebase your PR and validate that linters and tests are passing.
Before merging it I would like to get a review by a some people like @krissetto @simonferquel or @rumpl because even if the idea looks good I am not sure if it could have functionally some side effects in some cases

@aheritier
aheritier requested a review from docker-agent August 4, 2026 07:10

@docker-agent docker-agent left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assessment: 🟡 NEEDS ATTENTION

One medium-confidence finding was identified in the new code added by this PR.

Comment thread pkg/server/session_manager.go
@aheritier

Copy link
Copy Markdown
Collaborator

There are linter + tests failures @EronWright

@aheritier
aheritier marked this pull request as draft August 10, 2026 11:53
@EronWright
EronWright force-pushed the contrib/stop-toolsets-on-delete branch from 68e6493 to b652962 Compare August 14, 2026 18:52
A session materialised via teamloader owns a team whose toolsets may hold
external resources — notably stdio MCP subprocesses. DeleteSession cancelled
the runtime context but never called team.StopToolSets, so those subprocesses
leaked until the server process exited; BatchDeleteSessions had the same gap.

Track the per-session team on activeRuntimes and route both delete paths
through one scheduleTeardown helper: it keeps the entry in deletedSessions,
waits for the session's stream to drain, stops the toolsets, then drops the
entry. The drain and StopToolSets carry separate budgets, so a session that
exhausts the drain still gets a live context to stop its toolsets with.

BatchDeleteSessions now registers the same teardown, so a batch-deleted
session is observable through WaitStopped just like a singly-deleted one —
it previously never stored into deletedSessions, making WaitStopped return
nil while teardown was still running. WaitStopped in turn waits on a
completion channel closed after StopToolSets rather than polling the
streaming mutex, so it means "fully torn down" and can no longer return
before teardown has run.

Attached runtimes (AttachRuntime) leave team nil, so stopping is a no-op
there — their toolset lifecycle belongs to the embedder.

Claude-Session: https://claude.ai/code/session_01BiXE2cM4DxDQNYhKfsPn5K
@EronWright
EronWright force-pushed the contrib/stop-toolsets-on-delete branch from b652962 to e0785f3 Compare August 14, 2026 18:56
@EronWright
EronWright marked this pull request as ready for review August 14, 2026 19:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/core Core agent runtime, session management kind/fix PR fixes a bug (maps to fix:). Use on PRs only.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants