GH-45819: [C++] Add OptionalBitmapAnd utility#49848
Open
Shockp wants to merge 1 commit intoapache:mainfrom
Open
GH-45819: [C++] Add OptionalBitmapAnd utility#49848Shockp wants to merge 1 commit intoapache:mainfrom
Shockp wants to merge 1 commit intoapache:mainfrom
Conversation
Shockp
force-pushed
the
feat-optional-bitmap-and
branch
3 times, most recently
from
7fd57ca to
28d69c7
Compare
pitrou
requested changes
Apr 27, 2026
github-actions
Bot
added
awaiting committer review
Shockp
force-pushed
the
feat-optional-bitmap-and
branch
from
4973fdd to
2741418
Compare
Contributor
Author
|
Reverted the testing submodule, added the out_offset default, implemented the zero-copy slicing for byte-aligned offsets, and updated the tests to use BitmapFromVector for non-trivial inputs. Let me know how it looks! @pitrou Quick question on the API: I noticed the other functions in bitmap_ops.h (like BitmapAnd, BitmapOr, etc.) don't currently have out_offset = 0 as a default. Would it be worth unifying those for consistency in a separate, follow-up PR, or is it safer to leave the existing signatures as-is? |
Shockp
force-pushed
the
feat-optional-bitmap-and
branch
2 times, most recently
from
20af324 to
9e05f24
Compare
This commit introduces the `OptionalBitmapAnd` utility, which provides an optimized bitwise AND operation for Arrow bitmaps. Key changes: - Added `OptionalBitmapAnd` function in `bitmap_ops.h` and `bitmap_ops.cc`. - Implemented optimizations to avoid allocations and use slicing when bitmaps are byte-aligned and either the left or right bitmap is missing. - Added comprehensive unit tests in `bitmap_test.cc` covering all permutations of offsets, lengths, and missing bitmaps.
Shockp
force-pushed
the
feat-optional-bitmap-and
branch
from
960feb9 to
8972ab0
Compare
Contributor
Author
|
Sorry, I had some conflicts with my local branch while trying to pass the lint CI. I just squashed all the changes of the PR in a single commit. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rationale for this change
In Arrow, null bitmaps are optional and represented by
nullptrwhen a column contains no nulls. Previously, conjoining two optional bitmaps required downstream code to manually handle thenullptrchecks and memory allocations. This change centralizes that logic into a single, highly optimized utility function.What changes are included in this PR?
OptionalBitmapAndtocpp/src/arrow/util/bitmap_ops.handbitmap_ops.cc.nullptr: Returnsnullptr.nullptr: Returns a copy of the right buffer.nullptr: Returns a copy of the left buffer.BitmapAndfunction.Are these changes tested?
Yes. I added a new test case (
OptionalBitmapAnd) tocpp/src/arrow/util/bitmap_test.ccthat explicitly verifies the correct buffer allocation and bitwise output for all four memory states.Are there any user-facing changes?
No. This simply exposes a new C++ utility for internal development.
Closes #45819
OptionalBitmapAndutility #45819