fix(compression): the brain pre-fetch logs its failure instead of degrading in silence - #212
Merged
acidkill merged 1 commit intoSep 7, 2026
Conversation
…rading in silence CompressionEngine.run fetches the brain once for the whole pass, because the derived-field refresh needs its embedding config and a per-fiber lookup would contradict this path's own round-trip budget. When that fetch raised, the except set brain to None and said nothing — the only unlogged except in a file where every other one logs. The fallback itself is right: the refresh helper looks the brain up per fiber when it is handed None, so the pass completes. Two consequences were invisible though. The one-lookup-per-pass optimisation degrades to one lookup per fiber with nothing in the log to explain why the pass got slower. And when the storage problem persists, the warning the operator does see comes from content_refresh and blames the embedding provider for what is really a brain-fetch failure. A logger.warning with exc_info before the fallback fixes both, and changes nothing about when the code runs. Two tests: a pass whose get_brain raises must log the warning and still return a report, and a pass that fetches its brain must stay quiet.
acidkill
approved these changes
Sep 7, 2026
acidkill
left a comment
Owner
There was a problem hiding this comment.
Reviewed the full diff. Fail-soft stays fail-soft, but the degraded brain pre-fetch now warns with the cause instead of blaming the embedder downstream. Green CI.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CompressionEngine.runnow logs a warning with the cause when it fails, rather than falling back in silence.Why
This is follow-up 2 from the review on #193, which described the problem better than I can paraphrase: the pre-fetch is "the only unlogged
except" in a file where every other one logs, and although it is "correctly fail-soft — the helper re-fetches per fiber", two consequences are invisible.The first is a performance cliff with no signal. The whole point of the pre-fetch is one brain lookup per pass instead of one per fiber; when it fails, the pass quietly reverts to the per-fiber shape it was written to avoid, and nothing in the log accounts for the slowdown.
The second is worse, because it misdirects. If the storage problem persists, the operator does eventually see a warning — but it comes from
content_refreshand blames the embedding provider. The actual fault is a brain fetch, in a different module, that said nothing.A
logger.warning(..., exc_info=True)before the fallback fixes both. It changes nothing about when any code runs.Changes
engine/compression.py: alogger.warningwithexc_info=Truein theexceptaround the brain pre-fetch, beforebrain = None, and a comment recording why the branch stays fail-soft.tests/unit/test_compression_brain_prefetch_logging.py(new):test_failed_brain_prefetch_is_loggedswapsget_brainfor a coroutine that raises and asserts both the warning and thatrunstill returns a report;test_successful_brain_prefetch_stays_quietis the positive control.Test plan
pytest tests/unit/test_compression_brain_prefetch_logging.py— 2 passed.engine/compression.pyreverted tomainand the tests kept,test_failed_brain_prefetch_is_loggedfails onAssertionError: expected a 'Brain pre-fetch failed' warning; got: [], whilst the positive control still passes.pytest tests/ -m "not stress" -n 4against a live SurrealDB v3.2.0 — 7285 passed, 48 skipped, 1 xfailed, which ismain's 7283 plus the two tests added here. Two tests intests/unit/test_dashboard_brains_scope.pyfail on this branch and onmainalike: they want a live database and collide with one another under-n. Both pass when that file is run on its own.ruff check src/ tests/clean;ruff format --check src/ tests/reports 740 files already formatted.mypy src/ --ignore-missing-imports— success, no issues found in 354 source files.main.CHANGELOG.mduntouched — left to the release entry, as with fix(storage): bind datetimes in time comparisons so they select by value #191–fix(memory): refresh content-derived fields on compress, restore, and refine #193.Related issues
Follow-up 2 from the #193 review. Follow-up 1 from the same review — the
GRAPH_ONLY_PLACEHOLDERconstant — is a separate PR, since it touches a different set of files for a different reason.Verified by
@RobertSigmundsson