MPE preprint version - #215

Closed

FarmersWrap wants to merge 61 commits into

texttron:mainfrom

FarmersWrap:preprint

Commits on Dec 10, 2025

rewrote passage chunking
Ryan Yu
committed
cdaf034
added logic for left padding
Ryan Yu
committed
1536517

Commits on Dec 11, 2025

added search
Ryan Yu
committed
dccfaf8

Commits on Dec 14, 2025

changed the tokenizer logic
Ryan Yu
committed
1769715
added train collator debug
Ryan Yu
committed
16d604a
traincollator is done
Ryan Yu
committed
61dbf6e
fixed some comments
Ryan Yu
committed
6ec22a9
modified chunkedencoder
Ryan Yu
committed
40eddf8
Modified forward and model
Ryan Yu
committed
0ebcf37
Modified inference on chunked passage, in progress
Ryan Yu
committed
e7e3bc3
fixed a chunk size not passed to model
Ryan Yu
committed
2d9939f

Commits on Dec 15, 2025

changed eos to sep
Ryan Yu
committed
ed3e302

Commits on Dec 16, 2025

added logs
Ryan Yu
committed
add3832

Commits on Dec 19, 2025

added some scripts
Ryan Yu
committed
249dd9d

Commits on Dec 22, 2025

added tests
Ryan Yu
committed
9efb43b
added tests
Ryan Yu
committed
9c37e29
added log
Ryan Yu
committed
a18c578

Commits on Dec 23, 2025

review
Ryan Yu
committed
f0ee786
added collator helper functions
Ryan Yu
committed
3c1752d
added padding helper
Ryan Yu
committed
1fa3c1e
Tested the collator
Ryan Yu
committed
224e185
Reviewed forward and maxsim
Ryan Yu
committed
a60b20b

Commits on Dec 24, 2025

dataset uses random negative for cases have less negatives
Ryan Yu
committed
32a1976
added some prints
Ryan Yu
committed
1b4163e
removed one breakpoint
Ryan Yu
committed
fc16311

Commits on Dec 29, 2025

Added random chunks
Ryan Yu
committed
22868d2
Added full randomization
Ryan Yu
committed
d389bc6
removed useless variables
Ryan Yu
committed
338fe92
Refactored the randomization
Ryan Yu
committed
190f937

Commits on Jan 6, 2026

added search time passage encoding with chunks
Ryan Yu
committed
0a9626f

Commits on Jan 10, 2026

Added random chunk size for eval, and tests for prechunked passages and random chunking
Ryan Yu
committed
b88e5c1

Commits on Jan 26, 2026

Move loss compute to trainer
Ryan
committed
cd858d1
add logs
Ryan Yu
committed
417a201
try to fix masks
Ryan Yu
committed
fa5362d
removed some logs
kevin-zkc
committed
fe01725
Removed many logs
Ryan
committed
6793707
removed emb printing
Ryan
committed
e853354

Commits on Jan 27, 2026

one more fix for chunk size
Ryan
committed
5338771
add debug log
Ryan Yu
committed
5f48c8a
fix hash
Ryan Yu
committed
493ed71

Commits on Jan 29, 2026

added script generator
Ryan Yu
committed
3e50628
updated the scripts
Ryan Yu
committed
69840a2
modified script
Ryan Yu
committed
d6dbff8
changed the scripts and log
Ryan Yu
committed
82efea9
report to wandb
Ryan Yu
committed
8bbcf87

Commits on Jan 30, 2026

reevaluated the code
Ryan Yu
committed
cc26a6b

Commits on Feb 6, 2026

added the chunking
Ryan Yu
committed
6056cc5
add a script
Ryan Yu
committed
ac92ddb
add train
Ryan Yu
committed
6df28f6

Commits on Feb 13, 2026

Added diff
Ryan Yu
committed
ca544cf
added tests/test_chunking.py
Ryan Yu
committed
23b06aa

Commits on Feb 27, 2026

Added mpe
Ryan Yu
committed
da49386
deleted some scripts
Ryan Yu
committed
b1c28ed
deleted some scripts-2
Ryan Yu
committed
19e0519
deleted non-necessary code
Ryan Yu
committed
ee39a3e
deleted more scripts
Ryan Yu
committed
9d31443
dataset package
Ryan Yu
committed
376d1da
added req
Ryan Yu
committed
54913b0
uses sdpa for longembed
Ryan Yu
committed
c1bd75c

Commits on Feb 28, 2026

changed readme
Ryan Yu
committed
dc6a4fb

Commits on Jul 18, 2026

chore: gitignore decrypted BrowseComp-Plus queries.tsv

Your Name
and
claude
committed
1c1ec6d