feat(decay): record one aggregate telemetry row per decay pass - #187
Merged
Merged
Conversation
Closes #183. Collapsing superseded change-log updates is lossless for replication but discards the weight trajectory, and the only thing recording it was the replication queue. This gives decay its own home for that data. One aggregate row per pass, never a row per edge. The pass touches every synapse, so per-edge rows would reintroduce exactly the unbounded growth that made change_log a problem — an AST test asserts the write cannot appear inside a per-entity loop. DecayReport already carried most of it; what was missing: - why a processed synapse was NOT decayed. "processed" minus "decayed" used to be an unexplained number, so a healthy pass (most edges simply not due) and a starved one (a gate stuck shut) rendered identically. Now counted per gate: pinned, idle-gate, bookmark. - the weight distribution either side of the pass, bucketed. Denser near zero, where the prune threshold sits and where a mis-tuned rate does its damage. - the knobs the pass ran with. A distribution is uninterpretable without them: the same shape means different things at different decay rates. Off by default, like TraceConfig. Pruned from consolidation on retention and record count — and that prune deliberately does NOT check `enabled`, so turning telemetry off still cleans up what it accumulated. The write is fail-soft: telemetry that can break the pass it observes is worse than no telemetry, and the caller has nothing to do about it. New table is purely additive, so no SCHEMA_VERSION bump — ensure_schema is idempotent and runs before apply_migrations on every start. It carries brain_id, so it is registered in _BRAIN_SCOPED_TABLES; a test caught that omission, without which its rows would have outlived the brain. Indexed on (brain_id, ran_at) from day one, because every query here filters on time.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #183.
Why
Collapsing superseded change-log updates (#182) is lossless for replication, but it
discards the weight trajectory — and the only thing recording that was the replication
queue itself. This gives decay its own home for the data.
Shape: one row per pass, never per edge
The decay pass touches every synapse, so a row per edge would reintroduce exactly the
unbounded growth that made
change_loga problem, under a new name. An AST test assertsthe write cannot appear inside a per-entity loop — a comment saying so would not survive
the next refactor.
What was missing from DecayReport
DecayReportalready carried the counters and duration. Three things it did not:Why a processed synapse was not decayed.
processedminusdecayedwas anunexplained number, so a healthy pass (most edges simply not due yet) and a starved one
(a gate stuck shut) rendered identically. Now counted per gate: pinned, idle-gate,
bookmark — and surfaced in
summary()only when non-zero, so a clean pass stays quiet.The weight distribution, bucketed, either side of the pass. Buckets are denser near
zero, where the prune threshold sits and where a mis-tuned rate does its damage.
The knobs the pass ran with. A distribution is uninterpretable without them: the same
shape means different things at different decay rates.
Retention and safety
TraceConfig.deliberately does not check
enabled— otherwise turning telemetry off would strandits rows forever. A test pins that.
no telemetry, and the caller has nothing to do about the failure. This is the one place
where swallowing is correct, and it is logged.
Schema
Purely additive, so no
SCHEMA_VERSIONbump —ensure_schemais idempotent and runsbefore
apply_migrationson every start (the same precedent astraining_files,watch_state,tag_cooccurrence). Avoids the whole class of migration risk hit at v10.Two things caught during review that are worth naming:
brain_id, so it belongs in_BRAIN_SCOPED_TABLES. A test caught theomission; without it the rows would have outlived the brain.
(brain_id, ran_at)from day one, because every query here filters on time,and an unindexed filter column means a full read (v10's lesson).
Verified against a live brain
That distribution is the thing #183 asked for: it answers "is the decay rate sane" without
keeping a row per edge to get there. Test rows were removed afterwards.
Verification
google.genai) is identical onmaingreen before pushing