Skip to content

fix(datagen): make default temporal ranges deterministic - #8121

Merged
westonpace merged 1 commit into
mainfrom
claude/lance-issue-7913-boj8f5
Jul 31, 2026
Merged

fix(datagen): make default temporal ranges deterministic#8121
westonpace merged 1 commit into
mainfrom
claude/lance-issue-7913-boj8f5

Conversation

@westonpace

Copy link
Copy Markdown
Member

Closes #7913

Problem

The default temporal generators in lance-datagen (rand_date32, rand_date64, rand_timestamp) derived their one-year sampling range from Utc::now() at generator construction time. The RNG stream itself is deterministic, but it was mapped onto a moving wall-clock-dependent range, so the same generation parameters and RNG seed produced different arrays depending on when the generator was created. This also affected the corresponding paths through rand_type / rand_field, breaking reproducibility of saved fuzz inputs involving temporal columns.

Fix

  • Anchor the default sampling range to a fixed instant: the 365 days ending at 2024-01-01T00:00:00Z (exclusive), via a shared default_temporal_range() helper. Callers that need a time-relative range can use the existing *_in_range variants, as the updated doc comments now point out.
  • Add test_rng_temporal_deterministic, pinning exact expected values for rand_date32, rand_date64, and all four timestamp resolutions under the default seed. Pinned values fail on any reintroduction of wall-clock dependence (a generate-twice comparison could falsely pass when both constructions land in the same millisecond).
  • Adjust the date/timestamp predicates in the test_query_date / test_query_timestamp integration tests from > DATE '2024-01-01' to > DATE '2023-07-01': with the fixed range ending at 2024-01-01 the old literals would trivially match zero rows, and a mid-range literal keeps the filters actually splitting the generated data.

Verification

  • cargo test -p lance-datagen — 12 passed, including the new pinned-value test
  • cargo test -p lance --features slow_tests --test integration_tests -- test_query_date test_query_timestamp — 6 passed
  • cargo check --workspace --tests --benches — clean
  • cargo fmt --all and cargo clippy -p lance-datagen --tests --benches -- -D warnings — clean

🤖 Generated with Claude Code

https://claude.ai/code/session_01GzLGemSfdM8i7PRi8YwJAZ


Generated by Claude Code

The default sampling ranges for rand_date32, rand_date64, and
rand_timestamp were anchored to Utc::now(), so the same RNG seed
generated different values depending on when the generator was created.
This broke reproducibility of the rand_type / rand_field paths, e.g.
for saved fuzz inputs involving temporal columns.

Anchor the default one-year range to a fixed instant (the 365 days
ending at 2024-01-01T00:00:00Z) instead. Callers that need a
time-relative range can use the *_in_range variants.

Also pin exact expected values in a new test so any future
nondeterminism or accidental anchor change fails loudly, and adjust
query test predicates that assumed generated temporal values were near
the current date.

Fixes #7913

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GzLGemSfdM8i7PRi8YwJAZ
@github-actions github-actions Bot added the bug Something isn't working label Jul 31, 2026

@lance-gatekeeper lance-gatekeeper Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The wall-clock dependency breaks same-seed reproducibility in default temporal generation. The fixed one-year range removes that dependency across Date32, Date64, and every timestamp resolution while leaving explicit-range APIs for rolling data. Targeted generator and query tests pass, and I found no correctness, compatibility, or performance issue that should block this change.

@codecov

codecov Bot commented Jul 31, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@Xuanwo Xuanwo left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@westonpace
westonpace merged commit 68c2171 into main Jul 31, 2026
41 checks passed
@westonpace
westonpace deleted the claude/lance-issue-7913-boj8f5 branch July 31, 2026 18:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

lance-datagen temporal generators are nondeterministic with a fixed RNG seed

3 participants