Skip to content

feat(pubsub): add HedgingOptions validation and token bucket - #9463

Open
feywind wants to merge 1 commit into
pubsub/hedging-gax-abortfrom
pubsub/hedging-options-and-token-bucket
Open

feywind wants to merge 1 commit into
pubsub/hedging-gax-abortfrom
pubsub/hedging-options-and-token-bucket

Conversation

@feywind

@feywind feywind commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Overview

This PR is Layer 2 of 7 in Stack #9467, which implements Publish Hedging for @google-cloud/pubsub to mitigate high-percentile (p99/p99.9) publish tail latencies by sending backup (hedged) publish RPCs when an in-flight batch takes longer than a configurable hedgeDelay threshold.

Stack Structure

  1. feat(gax): support cancelling API calls via CallOptions.signal #9462 - feat(gax): support AbortSignal cancellation in CallOptions and createApiCall
  2. feat(pubsub): add HedgingOptions validation and token bucket #9463 - feat(pubsub): add HedgingOptions validation and HedgingTokenBucket (this PR)
  3. feat(pubsub): add CancellationSharer and HedgingScheduler #9464 - feat(pubsub): add CancellationSharer and HedgingScheduler
  4. feat(pubsub): add OpenTelemetry events and logging for hedged publish #9465 - feat(pubsub): add OpenTelemetry events and logging for hedged publish
  5. feat(pubsub): integrate publish hedging into MessageQueue #9466 - feat(pubsub): integrate publish hedging into MessageQueue
  6. test(pubsub): add unit tests for publish hedging queue integration #9635 - test(pubsub): add unit tests for publish hedging queue integration
  7. test(pubsub): add publish hedging system tests #9636 - test(pubsub): add publish hedging system tests

Summary of Changes in This PR

This layer introduces the public configuration surface (HedgingOptions on PublishOptions), bounds/compatibility validation, and the fixed-point token bucket rate limiter (HedgingTokenBucket):

  • HedgingOptions & ResolvedHedgingOptions (handwritten/pubsub/src/publisher/hedging.ts, handwritten/pubsub/src/default-options.ts):
    • hedgeDelay?: Duration - delay before sending a hedged attempt (default: 1000ms, valid range: [100ms, 10s]).
    • maxTokens?: number - maximum capacity of the token bucket (default: 50, valid range: 1..250 integer).
    • refillRatio?: number - token refill amount per successful publish batch (default: 0.1, valid range: [0.001, 0.2]).
    • Defaults are exported under defaultOptions.publish.hedging in src/default-options.ts.
  • HedgingTokenBucket (handwritten/pubsub/src/publisher/hedging.ts):
    • Fixed-point integer token bucket (HEDGE_TOKEN_SCALE = 1000, matching the Java client implementation) that starts empty (0 tokens), refills by refillRatio on each successful batch up to maxTokens, and consumes 1.0 token per hedged attempt.
    • Provides static helper methods validateAndResolveHedgingOptions, resolveInitialRpcTimeout, and resolveTotalTimeout.
  • Publisher Integration & Exports (handwritten/pubsub/src/publisher/index.ts, handwritten/pubsub/src/index.ts):
    • Added hedging?: HedgingOptions to PublishOptions, wired validation/bucket initialization into Publisher.prototype.setOptions, and exposed a tokenBucket getter on Publisher.
  • Unit Tests (handwritten/pubsub/test/publisher/hedging.ts, handwritten/pubsub/test/publisher/index.ts):
    • Comprehensive tests for defaults, custom configuration, bounds validation, timeout/ordering incompatibilities, and fixed-point token bucket math.

@product-auto-label product-auto-label Bot added the api: pubsub Issues related to the Pub/Sub API. label Sep 29, 2026
@feywind
feywind added this pull request to stack #9467 September 29, 2026 02:09

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces publish hedging support to the Google Cloud Pub/Sub Node.js client, adding a fixed-point HedgingTokenBucket rate limiter and validating hedging options against RPC and total timeouts. Feedback on the changes highlights a critical bug in Publisher.setOptions where calling the method without explicitly passing hedging silently clears any previously configured hedging options. It is recommended to preserve existing hedging settings when merging options and to add a corresponding unit test to verify this behavior.

Comment thread handwritten/pubsub/src/publisher/index.ts Outdated
Comment thread handwritten/pubsub/test/publisher/index.ts
options,
);

const resolvedHedging = HedgingTokenBucket.validateAndResolveHedgingOptions(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you need to also include this.options?.hedging before this.settings?.hedging so the currently active running state of the publisher in memory is checked

@feywind
feywind force-pushed the pubsub/hedging-options-and-token-bucket branch from 01ca69c to 369e51c Compare October 19, 2026 21:58

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: pubsub Issues related to the Pub/Sub API.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants