Skip to content

[DNM] add uint64 checksum log for blk data - #804

Closed
JacksonYao287 wants to merge 1 commit into
eBay:masterfrom
JacksonYao287:add_uint64_checksum_for_blk_data
Closed

JacksonYao287 wants to merge 1 commit into
eBay:masterfrom
JacksonYao287:add_uint64_checksum_for_blk_data

Conversation

@JacksonYao287

@JacksonYao287 JacksonYao287 commented Sep 15, 2025 •

Copy link
Copy Markdown
Member

this PR add uint64_t checksum for push_data/on_push_data_received and on_fetch_data_received/handle_fetch_data_response, so that we can directly check whether data is corrupted during network transmission from the log.

next step, we need to drop the received data if it is found corrupted.

@codecov-commenter

codecov-commenter commented Sep 15, 2025 •

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 0% with 32 lines in your changes missing coverage. Please review.
✅ Project coverage is 55.24%. Comparing base (1a0cef8) to head (dca9acf).
⚠️ Report is 264 commits behind head on master.

Files with missing lines Patch % Lines
src/lib/replication/repl_dev/raft_repl_dev.cpp 0.00% 32 Missing ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #804      +/-   ##
==========================================
- Coverage   56.51%   55.24%   -1.28%     
==========================================
  Files         108      110       +2     
  Lines       10300    13281    +2981     
  Branches     1402     1935     +533     
==========================================
+ Hits         5821     7337    +1516     
- Misses       3894     5104    +1210     
- Partials      585      840     +255     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@JacksonYao287
JacksonYao287 force-pushed the add_uint64_checksum_for_blk_data branch from abf6629 to c0a10cf Compare September 15, 2025 08:14
@JacksonYao287
JacksonYao287 marked this pull request as draft September 15, 2025 11:21
@JacksonYao287
JacksonYao287 force-pushed the add_uint64_checksum_for_blk_data branch from c0a10cf to 1d52b1f Compare September 15, 2025 12:02
@JacksonYao287
JacksonYao287 marked this pull request as ready for review September 15, 2025 12:02
@JacksonYao287
JacksonYao287 force-pushed the add_uint64_checksum_for_blk_data branch from 1d52b1f to dca9acf Compare September 15, 2025 12:06
@JacksonYao287

JacksonYao287 commented Sep 16, 2025 •

Copy link
Copy Markdown
Member Author

[09/15/25 05:00:00-07:00] [D] [test_raft_repl_dev_replica_1] [1169944] [raft_repl_dev.cpp:1398:handle_fetch_data_response] [traceID=48] [rdev1:640b63d5-675f-4139-8a33-fc71c9bc0fc0] Data Channel: the sha256 value of the data received in handle_fetch_data_response for rreq=[dsn=47 term=1 lsn=-1 op=HS_DATA_LINKED local_blkid=[[{blk#=294958 count=2 chunk=4},] ] state=[BLK_ALLOCATED | DATA_RECEIVED | ]] is 13456295685418586948

[09/15/25 05:00:00-07:00] [D] [test_raft_repl_dev_replica_0] [1169527] [raft_repl_dev.cpp:1354:operator()] [traceID=n/a] [rdev1:640b63d5-675f-4139-8a33-fc71c9bc0fc0] Data Channel: the sha256 value of the data in on_fetch_data_received for dsn=47 is 13456295685418586948

[09/15/25 04:59:45-07:00] [D] [test_raft_repl_dev_replica_2] [1169638] [raft_repl_dev.cpp:960:on_push_data_received] [traceID=48] [rdev1:640b63d5-675f-4139-8a33-fc71c9bc0fc0] Data Channel: the sha256 value of the data received in on_push_data_received for rreq=[dsn=47 term=1 lsn=-1 op=HS_DATA_LINKED local_blkid=[[{blk#=163922 count=2 chunk=3},] ] state=[BLK_ALLOCATED | DATA_RECEIVED | ]] is 13456295685418586948

[09/15/25 04:59:45-07:00] [D] [test_raft_repl_dev_replica_0] [1169464] [raft_repl_dev.cpp:856:push_data_to_all_followers] [traceID=48] [rdev1:640b63d5-675f-4139-8a33-fc71c9bc0fc0] Data Channel: the sha256 value of the data pushed for rreq=[dsn=47 term=1 lsn=-1 op=HS_DATA_LINKED local_blkid=[[{blk#=294997 count=2 chunk=3},] ] state=[BLK_ALLOCATED | ]] is 13456295685418586948

@JacksonYao287

Copy link
Copy Markdown
Member Author

above is the produced log for push_data_to_follower/ on_push_data_received , on_fetch_data_received/ handle_fetch_data_reponse . all the member will calculate the checksum of the sent/received data and put it in the log.

@xiaoxichen xiaoxichen changed the title add uint64 checksum log for blk data [DNM] add uint64 checksum log for blk data Sep 24, 2025
@xiaoxichen

Copy link
Copy Markdown
Collaborator

DNM before we have upgrade compatibility implemented

@JacksonYao287
JacksonYao287 deleted the add_uint64_checksum_for_blk_data branch July 23, 2026 02:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants