Skip to content

HBASE-29722 (backport portion of HBASE-27558) Coproc - data integrity issues for scan with heavy filters #7470

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
virajjasani merged 2 commits into from
Nov 21, 2025
Merged

HBASE-29722 (backport portion of HBASE-27558) Coproc - data integrity issues for scan with heavy filters #7470

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
76f6ac0
HBASE-29722 Fix data integrity issues for scan with heavy filters (ba…
virajjasani Nov 19, 2025
aab0cde
add test
virajjasani Nov 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3469,12 +3469,18 @@ private void scan(HBaseRpcController controller, ScanRequest request, RegionScan
limitReached = sizeLimitReached || timeLimitReached || resultsLimitReached;
Copy link
Copy Markdown
Contributor

@stoty stoty Nov 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not in the scope of this backport ticket, but why do we check only those two limits ?
is batch size not applicable here ? Why do we ignore the heap size limit ?

Maybe worth looking into this in another JIRA.


if (limitReached || !moreRows) {
// With block size limit, we may exceed size limit without collecting any results.
Copy link
Copy Markdown
Contributor

@apurtell apurtell Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please improve this comment.

Copy link
Copy Markdown
Contributor Author

@virajjasani virajjasani Nov 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment is from backport

// In this case we want to send heartbeat and/or cursor. We don't want to send heartbeat
// or cursor if results were collected, for example for cell size or heap size limits.
boolean sizeLimitReachedWithoutResults = sizeLimitReached && results.isEmpty();
// We only want to mark a ScanResponse as a heartbeat message in the event that
// there are more values to be read server side. If there aren't more values,
// marking it as a heartbeat is wasteful because the client will need to issue
// another ScanRequest only to realize that they already have all the values
if (moreRows && timeLimitReached) {
// Heartbeat messages occur when the time limit has been reached.
if (moreRows && (timeLimitReached || sizeLimitReachedWithoutResults)) {
// Heartbeat messages occur when the time limit has been reached, or size limit has
// been reached before collecting any results. This can happen for heavily filtered
// scans which scan over too many blocks.
builder.setHeartbeatMessage(true);
if (rsh.needCursor) {
Cell cursorCell = scannerContext.getLastPeekedCell();
Expand Down