Skip to content

feat(pubsub): add OpenTelemetry events and logging for hedged publish - #9465

Open
feywind wants to merge 1 commit into
pubsub/hedging-coordinator-and-schedulerfrom
pubsub/hedging-telemetry-and-logging
Open

feywind wants to merge 1 commit into
pubsub/hedging-coordinator-and-schedulerfrom
pubsub/hedging-telemetry-and-logging

Conversation

@feywind

@feywind feywind commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Overview

This PR is Layer 4 of 7 in Stack #9467, which implements Publish Hedging for @google-cloud/pubsub to mitigate high-percentile (p99/p99.9) publish tail latencies by sending backup (hedged) publish RPCs when an in-flight batch takes longer than a configurable hedgeDelay threshold.

Stack Structure

  1. feat(gax): support cancelling API calls via CallOptions.signal #9462 - feat(gax): support AbortSignal cancellation in CallOptions and createApiCall
  2. feat(pubsub): add HedgingOptions validation and token bucket #9463 - feat(pubsub): add HedgingOptions validation and HedgingTokenBucket
  3. feat(pubsub): add CancellationSharer and HedgingScheduler #9464 - feat(pubsub): add CancellationSharer and HedgingScheduler
  4. feat(pubsub): add OpenTelemetry events and logging for hedged publish #9465 - feat(pubsub): add OpenTelemetry events and logging for hedged publish (this PR)
  5. feat(pubsub): integrate publish hedging into MessageQueue #9466 - feat(pubsub): integrate publish hedging into MessageQueue
  6. test(pubsub): add unit tests for publish hedging queue integration #9635 - test(pubsub): add unit tests for publish hedging queue integration
  7. test(pubsub): add publish hedging system tests #9636 - test(pubsub): add publish hedging system tests

Summary of Changes in This PR

This layer adds OpenTelemetry span events and debug sub-logging for hedged publish attempts, matching the event names used in the Java client implementation:

  • OpenTelemetry Span Events (handwritten/pubsub/src/telemetry-tracing.ts):
    • PubsubEvents.publishStartHedged(message) - records 'publish start (hedged)' on the message's parentSpan when a hedged attempt is dispatched.
    • PubsubEvents.publishEndHedged(message) - records 'publish end (hedged)' on the message's parentSpan when a hedged attempt wins the race.
  • Debug Sublogger (handwritten/pubsub/src/publisher/message-queues.ts):
    • Added logs.publishHedged = baseLogs.pubsub.sublog('publish-hedged') for logging hedged attempt dispatches and token-bucket rate limiting.
  • Unit Tests (handwritten/pubsub/test/telemetry-tracing.ts):
    • Verified that publishStartHedged and publishEndHedged record the expected span events on producer spans.

@product-auto-label product-auto-label Bot added the api: pubsub Issues related to the Pub/Sub API. label Sep 29, 2026
@feywind
feywind added this pull request to stack #9467 September 29, 2026 02:09

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces telemetry and logging support for hedged publishing in the Pub/Sub client. Specifically, it adds a new sublog 'publishHedged' to the message queue logs, implements 'publishStartHedged' and 'publishEndHedged' event tracking methods in 'PubsubEvents', and includes corresponding unit tests to verify that these new hedged events are correctly recorded on the publisher span. There are no review comments to address.

@feywind
feywind force-pushed the pubsub/hedging-telemetry-and-logging branch from 03711de to c001507 Compare October 19, 2026 21:57

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: pubsub Issues related to the Pub/Sub API.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant