fix(registry): add large-thread-count feature to raise slab MAX_THREADS#3
Closed
Maxence Caron (mc-ditto) wants to merge 3 commits intodeprecate-macros-for-ditto-loggingfrom
Closed
Conversation
Adds a `large-thread-count` feature (off by default) that raises the sharded-slab MAX_THREADS limit from 4 096 to 131 072. This prevents tracing_subscriber from panicking on high-core nodes where large numbers of native threads are spawned (e.g. by the Ditto SDK in load-gen pods). When the feature is disabled the upstream default of 4 096 is used via the Config trait's default impl, so there is no behaviour change for existing consumers. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… enabled Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…l init Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Nick Robinson (nickrobinson)
pushed a commit
that referenced
this pull request
Apr 29, 2026
…irectives (tokio-rs#3487) ## Motivation `EnvFilter` maintains two `RwLock`-protected `HashMap`s (`by_id` and `by_cs`) for tracking per-span field-level match state. These are only populated when `has_dynamics` is `true` — i.e., when the filter includes span-name or field-level directives like `my_crate[span_name]=debug`. However, the span lifecycle callbacks (`on_new_span`, `on_enter`, `on_exit`, `on_close`, `on_record`) unconditionally acquire the `by_id` read lock (or write lock for `on_close`) on every invocation, even when `has_dynamics` is `false` and the maps are guaranteed empty. For the common case of pure `target=level` directives (e.g. `RUST_LOG=info,hyper=warn`), these locks are acquired and released on every span enter/exit/close/record with no effect. ## Problem Under high concurrency this causes measurable lock contention. A GDB backtrace captured from a Tokio-based HTTP server (80+ async workers, 3 `EnvFilter` instances as per-layer filters) during a load test showed ~25 threads contending on `RwLock::read` inside `EnvFilter::on_enter`, `on_exit`, and `cares_about_span`, with threads blocked in `read_contended` by `on_close`'s write lock: ``` #0 std::sys::sync::rwlock::futex::RwLock::read_contended #1 RwLock::read (self=0x55e0c7694178) #2 RwLock<HashMap<SpanId, MatchSet<SpanMatch>>>::read #3 EnvFilter::on_enter at filter/env/mod.rs:585 ``` ## Solution Add an early `if !self.has_dynamics { return; }` guard to each of the five span lifecycle methods. When dynamic directives are present, the existing code path runs unchanged. When they're absent (the common case), the lock is never touched.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
large-thread-countfeature flag (off by default) totracing-subscriberSlabConfigraisesMAX_THREADSfrom 4 096 to 131 072, preventing panics when large numbers of native threads are spawned on high-core nodessharded-slabfork or workspace-level[patch.crates-io]overrideTest plan
cargo check -p tracing-subscriberpasses with and without--features large-thread-count🤖 Generated with Claude Code