Optimize boolean operations #9284
Draft
+698
−181
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
parquet
Contributor
Author
|
run benchmark filter_kernels boolean_kernels arrow_reader arrow_reader_clickbench |
|
🤖 Hi @Dandandan, thanks for the request (#9284 (comment)).
Please choose one or more of these with |
|
🤖 |
|
🤖: Benchmark completed Details
|
|
🤖 |
|
🤖: Benchmark completed Details
|
|
🤖 |
|
🤖: Benchmark completed Details
|
|
🤖 |
|
🤖: Benchmark completed Details
|
Contributor
Author
|
run benchmark boolean_kernels filter_kernels |
|
🤖 |
|
🤖: Benchmark completed Details
|
|
🤖 |
|
🤖: Benchmark completed Details
|
Contributor
Author
|
run benchmark boolean_kernels |
|
🤖 |
|
🤖: Benchmark completed Details
|
Contributor
Author
|
run benchmark boolean_kernels |
|
🤖 |
|
🤖: Benchmark completed Details
|
|
🤖 |
|
🤖: Benchmark completed Details
|
|
🤖 |
|
🤖: Benchmark completed Details
|
|
🤖 |
|
🤖: Benchmark completed Details
|
|
🤖 |
|
🤖: Benchmark completed Details
|
Contributor
Author
|
run benchmark filter_kernels boolean_kernels arrow_reader_clickbench |
|
🤖 |
Contributor
Author
|
FYI @alamb this is starting to look good perf wise |
Dandandan
commented
Jan 28, 2026
| } | ||
|
|
||
| // both buffers have the same offset, we can use UnalignedBitChunk for both | ||
| let left_chunks = UnalignedBitChunk::new(left, left_offset_in_bits, len_in_bits); |
Contributor
Author
There was a problem hiding this comment.
I think we actually don't have to use this part (use the byte aligned one above and set the correct offset).
Dandandan
commented
Jan 28, 2026
| result.truncate(chunks.num_bytes()); | ||
| } | ||
| let src = src.as_ref(); | ||
| let chunks = UnalignedBitChunk::new(src, offset_in_bits, len_in_bits); |
Contributor
Author
There was a problem hiding this comment.
Probably not needed now that we have a fast byte aligned version.
Dandandan
commented
Jan 28, 2026
| } | ||
|
|
||
| #[inline] | ||
| fn fold<B, F>(mut self, init: B, mut f: F) -> B |
Contributor
Author
There was a problem hiding this comment.
The idea here is to implement an improved fold implementation so from_trusted_len_iter is fast (I used AI assistance to come up with the implementation, but it seems to look jormal)
Dandandan
commented
Jan 28, 2026
| let mut dst = buffer.data.as_ptr(); | ||
| for item in iterator { | ||
| let mut dst = buffer.data.as_ptr() as *mut T; | ||
| iterator.for_each(|item| { |
Contributor
Author
There was a problem hiding this comment.
for_each uses fold so can use a more efficient implementation if it is available.
|
🤖: Benchmark completed Details
|
|
🤖 |
|
🤖: Benchmark completed Details
|
|
🤖 |
Contributor
Author
|
run benchmark coalesce_kernels |
|
🤖: Benchmark completed Details
|
|
🤖 |
|
🤖: Benchmark completed Details
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Rationale for this change
What changes are included in this PR?
Are these changes tested?
Are there any user-facing changes?