Skip to content

Add focused benchmark for Parquet pruning setup cache - #24323

Open
kosiew wants to merge 2 commits into
apache:mainfrom
kosiew:schema_caching-90-21554
Open

Add focused benchmark for Parquet pruning setup cache#24323
kosiew wants to merge 2 commits into
apache:mainfrom
kosiew:schema_caching-90-21554

Conversation

@kosiew

@kosiew kosiew commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

ParquetPruningSetupCache in #21566 needs a focused benchmark that exercises it under deliberately cache-favourable conditions.

This benchmark provides a workload where many Parquet files share the same physical schema and use the same predicate, target partition count, and query, creating repeated opportunities to reuse pruning setup. It is intended to measure end-to-end scan cost for this specific workload shape, rather than represent general DataFusion performance or predict improvements in broader benchmarks such as ClickBench.

What changes are included in this PR?

  • Adds a Criterion benchmark that creates 128 Parquet files sharing one physical schema and repeatedly executes SELECT id FROM t WHERE id >= 0.
  • Keeps file generation, runtime creation, table registration, and initial correctness validation outside the timed Criterion loop.
  • Validates that the query returns all 16,384 expected rows before benchmarking.
  • Enables Criterion's async_tokio feature so the asynchronous query execution can be benchmarked with the Tokio runtime.
  • Registers the new parquet_pruning_setup_cache benchmark target.
  • Documents that the benchmark is intentionally cache-favourable and that its results should not be interpreted as general DataFusion or ClickBench performance results.

Are these changes tested?

The benchmark includes a correctness check that verifies the query returns 128 × 128 = 16,384 rows before the timed benchmark begins.

No additional tests are included in this patch.

Are there any user-facing changes?

No. This PR adds a focused benchmark and benchmark configuration only; it does not change user-facing DataFusion behavior or public APIs.

LLM-generated code disclosure

This PR includes LLM-generated code and comments. All LLM-generated content has been manually reviewed.

kosiew added 2 commits August 13, 2026 19:50
- Adds `benchmarks/benches/parquet_pruning_setup_cache.rs` with 128 same‑schema files.
- Predicate hits all files, exercising cache reuse and validating 16,384 rows.
- Updates `benchmarks/Cargo.toml` to include bench target and Criterion Tokio feature.
…seline scope and remove ClickBench generalization claim
@kosiew
kosiew force-pushed the schema_caching-90-21554 branch from 52f889c to 5b62f3e Compare August 13, 2026 11:51
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.14%. Comparing base (c08832d) to head (5b62f3e).
⚠️ Report is 12 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #24323      +/-   ##
==========================================
- Coverage   81.29%   81.14%   -0.16%     
==========================================
  Files        1110     1112       +2     
  Lines      385205   386933    +1728     
  Branches   385205   386933    +1728     
==========================================
+ Hits       313145   313963     +818     
- Misses      53580    54475     +895     
- Partials    18480    18495      +15     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@kosiew
kosiew marked this pull request as ready for review August 13, 2026 12:21
@kosiew
kosiew requested a review from alamb August 13, 2026 12:21

//! Benchmarks end-to-end Parquet scan cost for a cache-favourable workload.
//!
//! The 128 files share one physical schema and use the same predicate and target

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what would happen if the schema is not the same? or schema is already merged?

@kosiew kosiew Aug 14, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what would happen if the schema is not the same?
Different physical file schemas produce distinct ParquetPruningSetupCache keys in #21566, so their setup is not reused across schema variants.

or schema is already merged?
Merged table schema is also not the targeted scenario in #21554.

This benchmark's purpose is to evaluate whether addressing #21554 improves performance: that issue targets avoiding repeated per-file schema adaptation, predicate rewrites, and pruning-predicate construction when many files have the same physical schema.

Schema evolution/merged-schema behavior is a separate workload and would dilute this focused benchmark.

Existing clickbench_partitioned benchmarks do not show significant improvement, I think because:

  • pruning setup work constitutes a small percentage of the work
  • the scenarios do not reuse cache

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants