Skip to content

Feat: Configurable relayer clock-drift window + legible drift error - #600

Merged
ducnmm merged 5 commits into
devfrom
fix/relayer-clock-drift-window
Aug 18, 2026
Merged

Feat: Configurable relayer clock-drift window + legible drift error#600
ducnmm merged 5 commits into
devfrom
fix/relayer-clock-drift-window

Conversation

@hungtranphamminh

@hungtranphamminh hungtranphamminh commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Summary

Why

The off-chain relayer verifies an Ed25519 signature over a canonical message that includes a UNIX timestamp, and rejects requests whose timestamp is outside a freshness window. That window was a hardcoded ±300s, and a drifted client just got an opaque 401 — indistinguishable from a bad signature. In multi-region / serverless fleets with imperfect NTP, operators had no way to tune the tolerance and no way to tell clock skew from an auth failure. Reported in #571 (and duplicate #573).

What

  • Relayer: the timestamp-drift window is configurable via AUTH_MAX_CLOCK_DRIFT_SECS (default 300, hard-capped at 900), and a drift rejection is self-describing.
  • SDKs (TS + Python): surface that rejection as an actionable clock-drift error instead of an opaque 401.

Solution

  • Configurable + bounded window. AUTH_MAX_CLOCK_DRIFT_SECS is parsed defensively: values outside 0..=900 (and negative/garbage) fall back to the default. Unset → 300 freshness (same as before).
  • Replay invariant. Freshness is symmetric (|age| <= drift). A future-dated request can be first accepted at T - drift and stay fresh until T + drift. The replay-nonce Redis TTL is therefore 2 * drift + 300s (default 900, max 2100), so the nonce record outlives the full remaining freshness interval. Nonce one-time-use stays independent of the clock tolerance. Redis-down fail-closes.
  • Legible, oracle-safe error. Drift rejections return 401 + x-auth-error: ERR_TIMESTAMP_OUT_OF_BOUNDS, emitted only on the timestamp branch — before any signature/nonce/account check. CORS exposes that one header so the browser SDK can read it. Other 401s stay bare.
  • SDKs detect the header on 401 and raise ERR_TIMESTAMP_OUT_OF_BOUNDS / MemWalClockDriftError.

Privacy floor: unaffected — this is transport-auth freshness, no plaintext index.

Note: #571 asked for a client-side max_clock_drift_seconds. The client cannot change the server's tolerance, so the real knob is the relayer env var. The reporter's +45s repro is inside the default 300s window.

Testing

  • Auth unit tests including nonce_ttl_covers_full_future_dated_freshness_lifetime (pins 300→900 and 900→2100).
  • CORS expose-header contract test.
  • Rebased onto current dev.

Related

@jessiemongeon1

jessiemongeon1 commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Style Guide Audit

All 1 file(s) pass the style guide audit.

@harrymove-ctrl
harrymove-ctrl self-requested a review August 12, 2026 02:27

@ducnmm ducnmm left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found three blockers before this can be reviewed as the clock-drift change described:

  1. [P1] The browser SDK cannot read the new drift reason header cross-origin. clockDriftErrorFromResponse() depends on res.headers.get("x-auth-error"), but services/server/src/main.rs does not expose x-auth-error through CORS (CorsLayer has no expose_headers entry). For the hosted relayer/browser SDK path, Fetch therefore hides the header and users still receive the opaque 401 this PR intends to fix. Please expose only this machine-readable header and add coverage for the CORS response contract.

  2. [P1] The branch contains 22 unrelated historical commits and is now conflicting with dev. GitHub reports 35 changed files / 1,483 additions, including old app, Move, gRPC, docs, and dependency changes unrelated to #571. The actual feature is only the final three commits (cb4c9450, 3ecc5a29, 85e7d254). Please rebuild/rebase the branch from current dev with only those commits, resolve overlap with recently merged SDK/manual and server changes, and rerun the full required CI matrix. The current two analysis jobs are not sufficient acceptance evidence for the rebased result.

  3. [P2] The operator-facing configuration is undocumented. AUTH_MAX_CLOCK_DRIFT_SECS is introduced as the deployment control, but it is absent from services/server/.env.example and docs/reference/environment-variables.md. Without this, operators cannot discover its default/cap or safely configure it. Please document the default 300s, allowed 0–900 range, fallback behavior, and replay-window tradeoff. Also fix the existing style-guide audit failures once the unrelated docs are removed/rebased.

The core bounded-window and nonce-TTL derivation look directionally sound. I did not continue with an approval-level review because the stale/conflicting branch means most of the displayed diff is not the proposed feature and its final integration result does not yet exist.

@hungtranphamminh
hungtranphamminh force-pushed the fix/relayer-clock-drift-window branch from 85e7d25 to 7163486 Compare August 12, 2026 16:46
@hungtranphamminh

Copy link
Copy Markdown
Contributor Author

Thanks for the review — all three blockers are addressed. The branch has been rebuilt from current dev, so the displayed diff is now only the feature (14 files, +449/−36) and the full CI matrix is green.

1. [P1] Browser CORS now exposes the reason header. Extracted the relayer CORS into relayer_cors() and added .expose_headers(["x-auth-error"]) — only that header. Fetch now lets the browser SDK read x-auth-error, so the hosted/browser path gets the actionable clock-drift error instead of the opaque 401. Covered by a router-level test (relayer_cors_exposes_only_x_auth_error) that drives a real cross-origin request and asserts Access-Control-Expose-Headers is exactly x-auth-error.

2. [P1] Branch rebuilt from current dev. Root cause: the branch was cut from a commit that dev later rewrote, so it carried 22 unrelated historical commits. Rebuilt fresh off current dev with only the feature work, now four logical commits — server + CORS, SDK, release metadata, docs. Resolved the overlap with the merged SDK/manual and server changes (including a new Config test fixture that needed the added field). Full required CI ran on the rebased result and passes.

3. [P2] AUTH_MAX_CLOCK_DRIFT_SECS documented. Added to services/server/.env.example and docs/reference/environment-variables.md: default 300s, allowed 0–900 range, fallback-to-default on out-of-range/unparseable, independence from nonce replay protection, and the replay-window trade-off (a wider window lengthens the interval a captured request could be replayed while the nonce store is unavailable). The earlier style-guide audit violations were all in the unrelated docs that the rebuild removed.

The bounded-window and nonce-TTL-derivation are unchanged from what you already found sound. Ready for another look — handing the ball back.

@ducnmm ducnmm left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed rebuilt exact head 7163486b19f4b18f67e9a50c5772e4a4e4b6cbf7. The three previous findings are resolved: the branch now contains only the 14 feature files on current dev, x-auth-error is exposed through the main relayer CORS contract with coverage, and the operator configuration is documented. All required CI checks are green and git diff --check is clean.

One replay-safety blocker remains:

[P1] The derived nonce TTL does not cover the full remaining freshness interval for future-dated requests (services/server/src/auth.rs:81-83,270-285). Freshness is symmetric: a timestamp at now + drift is accepted, and that same signed request remains fresh until server time reaches timestamp + drift, i.e. for up to 2 × drift after its first accepted use. But nonce_ttl_secs stores the nonce for only drift + 300 seconds. For every configured drift above 300 (for example 900), the nonce expires after 1200s while the captured request remains timestamp-fresh for up to 1800s, allowing the same signature/nonce to be accepted again during the final 600s. This breaks the stated replay invariant precisely when operators widen the new setting.

The previous constants (300s symmetric drift, 600s nonce TTL) happened to cover the worst case because TTL was 2 × drift, not merely because it was greater than one drift. Please derive the TTL from the full symmetric lifetime, e.g. 2 * drift + a positive buffer (bounded arithmetic is trivial with the 900s cap), and update the tests to model the future-dated worst case explicitly: after first acceptance at timestamp - drift, the nonce record must outlive freshness through timestamp + drift. Pin values at default and max settings so this cannot regress.

No other blockers found in the rebuilt diff. Keeping changes requested only for this replay window issue.

@ducnmm ducnmm mentioned this pull request Aug 13, 2026
@hungtranphamminh
hungtranphamminh force-pushed the fix/relayer-clock-drift-window branch from 7163486 to 9920c27 Compare August 13, 2026 14:21
@hungtranphamminh

Copy link
Copy Markdown
Contributor Author

Good catch — you're right, and it's fixed.

The nonce TTL was derived from a single drift (drift + 300), but freshness is symmetric: a request signed for T can be first accepted as early as now = T - drift (future-dated) and stays fresh through now = T + drift. So the record has to survive the full 2 * drift lifetime, not one drift. At drift = 900 the old TTL (1200s) expired while the request was still fresh through 1800s — the 600s replay gap you described. The old 300/600 constants only worked because 600 happened to be 2 × 300.

Fix (services/server/src/auth.rs): nonce_ttl_secs now returns 2 * drift + NONCE_TTL_BUFFER_SECS — 900s at the default, 2100s at the 900s cap (bounded, no overflow). The test now models the future-dated worst case explicitly: nonce first written at T - drift, asserted to still exist past T + drift, with values pinned at both the default and the max so it can't regress. The old drift + buffer derivation fails this test, which is the point.

The doc comments on NONCE_TTL_BUFFER_SECS and nonce_ttl_secs are updated to state the 2 * drift reasoning. Only auth.rs + types.rs changed; the fix is folded into the server commit, so it's still four clean commits. Full suite green (529 server tests). Ready for another look.

… error [#571]

The signed-request timestamp freshness tolerance was a hardcoded ±300s and
rejected drifted clients with an opaque 401. Make it configurable and bounded,
make a drift rejection distinguishable from a bad signature, and expose the
reason header cross-origin so browser clients can read it.

- AUTH_MAX_CLOCK_DRIFT_SECS: default 300, clamped 0..=900; garbage/negative/
  over-cap fall back to the default. Byte-identical to the old behavior when
  unset. Freshness predicate extracted to is_timestamp_fresh().
- Nonce replay TTL now derives from the window (nonce_ttl_secs = window + 300)
  so the "nonce TTL > window" replay invariant holds for every window value.
- Drift rejections return 401 + x-auth-error: ERR_TIMESTAMP_OUT_OF_BOUNDS via a
  constant-time helper, emitted ONLY on the timestamp branch (which runs before
  any identity/account check), so it opens no enumeration oracle.
- CORS: expose x-auth-error on the relayer CORS layer (extracted to
  relayer_cors()). Fetch hides response headers not in Access-Control-Expose-
  Headers, so without this the browser SDK could never read the reason. Only
  this header is exposed.

Tests: bounded-config + boundary (incl. 901) + nonce-TTL-derivation + reason-
header + CORS-expose contract exercise the real helpers. 529 server tests pass.
…or [#571]

When the relayer rejects a signed request whose timestamp is outside its
accepted clock-drift window, it returns 401 + x-auth-error:
ERR_TIMESTAMP_OUT_OF_BOUNDS. Detect that and raise a clear "synchronize the
client clock" error instead of an opaque 401. Backward-compatible: a response
without the header behaves exactly as before.

- TS: shared clockDriftErrorFromResponse() in utils, used on both the Relayer
  (memwal.ts) and manual (manual.ts) request paths; error carries
  serverCode ERR_TIMESTAMP_OUT_OF_BOUNDS.
- Python: new MemWalClockDriftError (subclasses MemWalError, exported) + test.
#571]

- Changeset: @mysten-incubation/memwal patch for the clock-drift error surfacing.
- Python CHANGELOG: entry under Unreleased for MemWalClockDriftError (no version
  bump — batched into the next Python release).
Add the new operator control to services/server/.env.example and
docs/reference/environment-variables.md: default 300s, allowed 0-900 range,
fallback-to-default behavior, independence from nonce replay protection, and
the replay-window trade-off of widening it.
@ducnmm
ducnmm force-pushed the fix/relayer-clock-drift-window branch from 9920c27 to 1ea82a8 Compare August 18, 2026 03:35
Move MemWalClockDriftError out of the shipped 0.1.5 notes. Document the
symmetric nonce TTL (2 × window + 300s) and that Redis-down fail-closes.
Require HTTP 401 before treating x-auth-error as clock drift.

@ducnmm ducnmm left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed after rebase onto current dev (0cc3b1a9). Hung left; Henry picked this up.

Previous P1 is closed: nonce_ttl_secs = 2 * drift + 300, tested at first-accept T - drift through freshness T + drift, pinned 300→900 and 900→2100. CORS exposes only x-auth-error. Operator docs are present. Reason header stays on the timestamp branch only.

Follow-up also moved the Python changelog to Unreleased (it had landed under shipped 0.1.5), corrected the TTL wording in the PR body / env docs, and requires HTTP 401 before treating the header as clock drift.

Approve. Merge once CI is green.

@ducnmm
ducnmm merged commit 3a926f6 into dev Aug 18, 2026
21 of 22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants