review F: whole-repository review docs/31 (F1-F10) - #67
Merged
Conversation
…ansfer flag, script resolution, NaN calibration query, rescorer standardisation, output-over-input, and four regressions from A/D/E Package F of docs/31_code_review_2026-09-08_full.md: the six confirmed correctness and security defects, plus the regressions packages A, D and E introduced. F1. prescan read the infinite-bounds sentinel that rt-im-train writes for "calibration unavailable, search the whole gradient" as "cannot be screened" and dropped the candidate. On the documented FASTA/MS2PIP path, where no confident seed exists and every window row is infinite, that discarded the entire library and exited 0 with a zero-row survivors table; the single-label bail could not see it, being gated on a survivor count. An unbounded window now screens over the whole gradient, a candidate with no run_windows row is treated the same, both are counted and warned about, and screening every candidate away is an error. F2. A present-but-wrong-typed is_transferred was swallowed as "no transfers", silently removing every match-between-runs identification from peptides.tsv and proteins.tsv while the parquet still carried them. mbr_worker.py writes that column through pandas, so a nullable dtype or int8 reaches the stage. Present columns are read in their declared type and a mismatch is an error; only an absent column falls back. This is the absent-versus-malformed rule quant and audit already follow. F3. sidecar::resolve_script tried the working directory before the directory beside the binary, the ordering python::resolve_script_dir was hardened against and documents at length. That resolver only claims a directory holding mbr_worker.py or deeplc_worker.py, so a directory with any of the other ten workers reached resolve_script still relative and was executed from the working directory. An absolute directory is taken as given, then the executable's directory, then <exe>/scripts, and the working directory last. F4. Loess::predict indexed before the start of its grid for a non-finite query: both boundary comparisons are false, partition_point returns 0, and grid_x[j - 1] underflows. One library row with a null predicted_irt is enough, because the parquet reader maps a null f32 to NaN. It returns NaN now, and rt-im-train treats a non-finite library iRT as "no calibrated RT" and counts those rows. F5. The rescorer's in-memory TSV backend standardised with median/IQR while the parquet and streaming backends used mean/std, so one pool scored differently depending on rescore.handoff and on the 4 GB streaming threshold, with nothing logged. All three use mean/std, which leaves the shipped parquet default and every published benchmark unchanged. The MUMDIA_NN_FOLD_KEYS companion is length-checked instead of sliced short, which used to leave the tail rows in no fold, unscored, emerging from the rank normalisation at a plausible mid-rank score that satisfies the completeness contract. The backend-size estimate counts feature columns by name rather than subtracting a hardcoded three. F6. refuse_output_over_input existed and was wired into two of eighteen stages, so compete --features f.parquet --out f.parquet replaced the widest artifact of the run with the competed subset at exit 0. It guards every output of search-seed, rt-im-train, extract, features, compete, rescore, quant and audit. F7 (regression, package D). The LOESS boundary extrapolation slope was the pointwise local slope at the sparsest, most one-sided point of the fit: unbounded, free to be negative and so to invert the iRT-to-RT map, multiplying a distance that is unbounded by construction. It is the secant of the fitted curve over its end decile, clamped non-negative and to at most four times the global least-squares slope. The test uses noisy anchors rather than a noiseless quadratic. F8 (regression, package E). A stop arriving between the reap and the end of publish_exit could pass a recycled process id to the tree kill. The waiter retires the id the instant wait returns, before it reads the output directory. F9 (regression, package E). The conversion lock spun without pause on an undeletable stale lock, mistook clock skew and a peer's partial file for evidence about its own holder, could be held by two processes at once, and left every interrupted conversion's partial mzML behind for ever. Take-overs are bounded and paced, waiting has a deadline, the holder is identified by a token it reads back and only removes a lock that is still its own, a future modification time counts as fresh, the partial-file probe matches this destination only, and abandoned partials are swept under the lock. F10 (regression, package A). Dropping an unpredicted candidate with everything sharing its pair key also removed positional isomers that predicted correctly, bounded only by the library being emptied. The direct misses and the collateral are counted separately and exceeding 2% of the library is an error naming the sidecar. The key stays position-free deliberately: a positional key would stop matching a reverse decoy to its target, trading a sensitivity defect for an FDR one, and a test pins that trade. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sixth stacked review PR, and the first from a whole-repository review rather than a diff:
docs/31_code_review_2026-09-08_full.md, work package F. Base isreview/e-followup(#66); merge #61, A, B, C, D, E, then this.Twelve parallel finders plus a verification sweep produced twenty-one findings at
21b8ac7. Eleven were re-verified against the source before being recorded. Package F closes the six confirmed correctness and security defects and the four regressions this review series introduced; the remaining ten are listed in the document as a second tier.Confirmed defects
prescanread the(-inf, +inf)sentinel thatrt-im-trainwrites for "calibration unavailable, search the whole gradient" as "cannot be screened" and dropped the candidate. On the documented FASTA/MS2PIP path that discarded the entire library at exit 0, with a zero-row survivors table and a NaN target/decoy ratio; the single-label bail is gated onsurv.len() > 1000so it never fired. An unbounded window now screens over the whole gradient, a candidate with no window row is treated the same, both are counted and warned about, and screening everything away is an error naming the likely causes.is_transferredwas swallowed as "no transfers", silently removing every match-between-runs identification from both TSV reports while the parquet still carried them.mbr_worker.pywrites it through pandas, so a nullable dtype or int8 reaches the stage. Present means readable now; only absent falls back.a_malformed_transfer_column_is_an_error_not_a_silent_loss_of_every_transfer,a_table_that_never_saw_mbr_reports_no_transfers_without_complaintsidecar::resolve_scripttried the working directory first, the orderingpython::resolve_script_dirwas hardened against; that resolver only claims a directory holdingmbr_worker.pyordeeplc_worker.py, so a directory with any of the other ten workers arrived here relative and ran from the CWD. Absolute as given, then exe dir, then<exe>/scripts, CWD last.the_shipped_directory_beside_the_binary_wins_over_the_working_directory+2Loess::predictindexedgrid_x[-1]for a NaN query; one library row with a nullpredicted_irtreaches it, because the parquet reader maps a null f32 to NaN. Returns NaN, andrt-im-traintreats a non-finite iRT as "no calibrated RT" and counts it.a_nan_query_returns_nan_instead_of_indexing_out_of_boundsrescore.handoffand across the 4 GB threshold. All three use mean/std, leaving the shipped default and every published benchmark unchanged. The fold-key companion is length-checked (a short one left the tail unscored at a fabricated mid-rank score that satisfies the completeness contract), and the backend-size estimate counts features by name.a_short_fold_key_file_is_refused_rather_than_leaving_rows_unfoldedrefuse_output_over_inputwas wired into 2 of 18 stages, socompete --features f.parquet --out f.parquetreplaced the run's widest artifact with the competed subset at exit 0. Now on every output ofsearch-seed,rt-im-train,extract,features,compete,rescore,quant,audit.writing_the_output_over_the_input_is_refusedRegressions from this review series
boundary_extrapolation_stays_monotone_and_bounded_under_noise, on noisy anchors rather than the noiseless quadratic the original test usedpublish_exitcould tree-kill a recycled pid. The waiter retires the pid the instantwaitreturns.a_stop_after_the_reap_has_no_pid_to_killraw::testsa_large_unpredicted_fraction_is_a_failure_not_a_warning,a_positional_isomer_still_shares_its_pair_key_by_designTwo changes deliberately not made
Not in this PR
The second tier is recorded in docs/31: unbounded
LogBinsallocation, the scramble decoy PRNG carrying no peptide dependence, the vacuous NaN-RT smoke assertion, the SBOM's dangling refs, the inert DeepLC import-order test, the/3.0top-3 features, and five smaller items.Local checks
Engine
cargo fmt --check,clippy -D warnings,cargo test --workspace(280 tests, up from 266); desktopclippy+cargo test --lib(82);pytest tests/python(68 passed, 12 skipped);python -m compileall scripts;ci/gen_config_reference.pyregenerated;ci/check_doc_refs.py;ci/check_desktop_ui.py.🤖 Generated with Claude Code