Skip to content

fix: harden client telemetry (#587, #588) - #601

Merged
sre-ci-robot merged 1 commit into
milvus-io:masterfrom
yhmo:fix
Sep 10, 2026
Merged

fix: harden client telemetry (#587, #588)#601
sre-ci-robot merged 1 commit into
milvus-io:masterfrom
yhmo:fix

Conversation

@yhmo

@yhmo yhmo commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

Copilot AI lite review requested due to automatic review settings September 9, 2026 07:30
@sre-ci-robot

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: yhmo

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@mergify

mergify Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

  • Queue this pull request

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The changes are well-scoped, appear correctness-focused, and are backed by targeted unit tests that exercise the newly hardened control-flow paths.

Pull request overview

This PR hardens the SDK’s client-side telemetry pipeline by making heartbeat scheduling and command processing more robust (especially under extreme intervals and transport rebinding), while also relaxing client validation for server-only push fields.

Changes:

  • Clamp and chunk very large heartbeat intervals to keep waits cancellable and safe across platforms.
  • Fence command execution across telemetry transport rebinding and canonicalize ACK correlation IDs.
  • Improve show_errors payload bounding (including UTF-8-safe truncation) and treat ttl_seconds as an ignored push_config field.
File summaries
File Description
src/impl/ClientTelemetry.cpp Implements heartbeat interval clamping/chunked waits, channel-rebind fencing, ACK ID canonicalization, unsupported-backoff wakeup on channel replacement, and bounded UTF-8-safe show_errors truncation; removes ttl_seconds validation from push_config.
test/ut/TestClientTelemetry.cpp Adds/updates unit tests covering max heartbeat intervals, ACK ID behavior, generation-switch fencing, backoff interruption on rebind, immediate probe on replacement during in-flight heartbeat, show_errors truncation bounds (incl. UTF-8), and ttl_seconds ignored behavior.
Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/impl/ClientTelemetry.cpp Outdated
Comment thread src/impl/ClientTelemetry.cpp
Comment thread src/impl/ClientTelemetry.cpp Outdated
@codecov

codecov Bot commented Sep 9, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 87.00000% with 39 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.02%. Comparing base (a0592aa) to head (128339d).
⚠️ Report is 158 commits behind head on master.

Files with missing lines Patch % Lines
src/impl/ClientTelemetry.cpp 87.00% 39 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           master     #601       +/-   ##
===========================================
+ Coverage   53.47%   88.02%   +34.55%     
===========================================
  Files          52      389      +337     
  Lines        4432    16580    +12148     
  Branches        0     1834     +1834     
===========================================
+ Hits         2370    14595    +12225     
+ Misses       2062     1985       -77     
Files with missing lines Coverage Δ
src/impl/ClientTelemetry.cpp 85.09% <87.00%> (ø)

... and 405 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread src/impl/ClientTelemetry.cpp Outdated
Comment thread src/impl/ClientTelemetry.cpp
Comment thread src/impl/ClientTelemetry.cpp
Comment thread src/impl/ClientTelemetry.cpp Outdated
Comment thread test/ut/TestClientTelemetry.cpp
Comment thread src/impl/ClientTelemetry.cpp
@mergify mergify Bot added the ci-passed label Sep 10, 2026
Comment thread src/impl/ClientTelemetry.cpp
Comment thread src/impl/ClientTelemetry.cpp Outdated
Comment thread test/ut/TestClientTelemetry.cpp Outdated
Comment thread src/impl/ClientTelemetry.cpp Outdated
Comment thread test/ut/TestClientTelemetry.cpp Outdated
Comment thread src/impl/ClientTelemetry.cpp Outdated
@mergify mergify Bot added the ci-passed label Sep 10, 2026
- Treat ttl_seconds as an ignored push_config field (milvus-io#587)
- Harden client telemetry control flow: make very large heartbeat
  intervals safe and cancellable across platforms, fence command
  batches across telemetry transport rebinding, canonicalize ACK IDs,
  wake the unsupported backoff on channel replacement, and bound
  show_errors truncation including UTF-8 payloads (milvus-io#588)
- Wake the chunked heartbeat wait on a corrective interval decrease so
  a server push takes effect immediately instead of after the current
  chunk; an increase still applies at the next natural wake
- Swap the accumulated collectors out of the heartbeat lock in O(1)
  and build the snapshot outside the lock, so the per-bucket sort and
  serialization no longer block RecordOperation with many collections;
  note the request serialization in SendHeartbeat as a follow-up
- Re-queue the original reply for fenced redeliveries instead of
  minting a success ACK, so an already-reported failure is not
  contradicted; clamp the pushed heartbeat interval as uint64 and cap
  the reported latency-history window start at the retained range
- Keep the UNIMPLEMENTED backoff from probing more often than the
  configured interval (floor preserved), reject negative pushed
  intervals instead of wrapping them, and use underscore-suffixed Impl
  member names consistent with the rest of the repository

Signed-off-by: yhmo <yihua.mo@zilliz.com>
@mergify mergify Bot added ci-passed and removed ci-passed labels Sep 10, 2026
@yhmo yhmo added the lgtm label Sep 10, 2026
@sre-ci-robot
sre-ci-robot merged commit 81c295a into milvus-io:master Sep 10, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants