fix: harden client telemetry (#587, #588) - #601
Conversation
yhmo
commented
Sep 9, 2026
- Treat ttl_seconds as an ignored push_config field (fix: treat ttl_seconds as an ignored push_config field #587)
- Harden client telemetry control flow: make very large heartbeat intervals safe and cancellable across platforms, fence command batches across telemetry transport rebinding, canonicalize ACK IDs, wake the unsupported backoff on channel replacement, and bound show_errors truncation including UTF-8 payloads (fix: harden client telemetry control flow #588)
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: yhmo The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
Tick the box to add this pull request to the merge queue (same as
|
There was a problem hiding this comment.
🟢 Approval recommended
The changes are well-scoped, appear correctness-focused, and are backed by targeted unit tests that exercise the newly hardened control-flow paths.
Pull request overview
This PR hardens the SDK’s client-side telemetry pipeline by making heartbeat scheduling and command processing more robust (especially under extreme intervals and transport rebinding), while also relaxing client validation for server-only push fields.
Changes:
- Clamp and chunk very large heartbeat intervals to keep waits cancellable and safe across platforms.
- Fence command execution across telemetry transport rebinding and canonicalize ACK correlation IDs.
- Improve
show_errorspayload bounding (including UTF-8-safe truncation) and treatttl_secondsas an ignoredpush_configfield.
File summaries
| File | Description |
|---|---|
src/impl/ClientTelemetry.cpp |
Implements heartbeat interval clamping/chunked waits, channel-rebind fencing, ACK ID canonicalization, unsupported-backoff wakeup on channel replacement, and bounded UTF-8-safe show_errors truncation; removes ttl_seconds validation from push_config. |
test/ut/TestClientTelemetry.cpp |
Adds/updates unit tests covering max heartbeat intervals, ACK ID behavior, generation-switch fencing, backoff interruption on rebind, immediate probe on replacement during in-flight heartbeat, show_errors truncation bounds (incl. UTF-8), and ttl_seconds ignored behavior. |
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 0
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #601 +/- ##
===========================================
+ Coverage 53.47% 88.02% +34.55%
===========================================
Files 52 389 +337
Lines 4432 16580 +12148
Branches 0 1834 +1834
===========================================
+ Hits 2370 14595 +12225
+ Misses 2062 1985 -77
🚀 New features to boost your workflow:
|
- Treat ttl_seconds as an ignored push_config field (milvus-io#587) - Harden client telemetry control flow: make very large heartbeat intervals safe and cancellable across platforms, fence command batches across telemetry transport rebinding, canonicalize ACK IDs, wake the unsupported backoff on channel replacement, and bound show_errors truncation including UTF-8 payloads (milvus-io#588) - Wake the chunked heartbeat wait on a corrective interval decrease so a server push takes effect immediately instead of after the current chunk; an increase still applies at the next natural wake - Swap the accumulated collectors out of the heartbeat lock in O(1) and build the snapshot outside the lock, so the per-bucket sort and serialization no longer block RecordOperation with many collections; note the request serialization in SendHeartbeat as a follow-up - Re-queue the original reply for fenced redeliveries instead of minting a success ACK, so an already-reported failure is not contradicted; clamp the pushed heartbeat interval as uint64 and cap the reported latency-history window start at the retained range - Keep the UNIMPLEMENTED backoff from probing more often than the configured interval (floor preserved), reject negative pushed intervals instead of wrapping them, and use underscore-suffixed Impl member names consistent with the rest of the repository Signed-off-by: yhmo <yihua.mo@zilliz.com>