Skip to content

feat(decay): record one aggregate telemetry row per decay pass - #187

Merged
acidkill merged 1 commit into
mainfrom
feat/decay-telemetry
Aug 27, 2026
Merged

acidkill merged 1 commit into
mainfrom
feat/decay-telemetry

Conversation

@acidkill

Copy link
Copy Markdown
Owner

Closes #183.

Why

Collapsing superseded change-log updates (#182) is lossless for replication, but it
discards the weight trajectory — and the only thing recording that was the replication
queue itself. This gives decay its own home for the data.

Shape: one row per pass, never per edge

The decay pass touches every synapse, so a row per edge would reintroduce exactly the
unbounded growth that made change_log a problem, under a new name. An AST test asserts
the write cannot appear inside a per-entity loop — a comment saying so would not survive
the next refactor.

What was missing from DecayReport

DecayReport already carried the counters and duration. Three things it did not:

Why a processed synapse was not decayed. processed minus decayed was an
unexplained number, so a healthy pass (most edges simply not due yet) and a starved one
(a gate stuck shut) rendered identically. Now counted per gate: pinned, idle-gate,
bookmark — and surfaced in summary() only when non-zero, so a clean pass stays quiet.

The weight distribution, bucketed, either side of the pass. Buckets are denser near
zero, where the prune threshold sits and where a mis-tuned rate does its damage.

The knobs the pass ran with. A distribution is uninterpretable without them: the same
shape means different things at different decay rates.

Retention and safety

  • Off by default, like TraceConfig.
  • Pruned from consolidation on both retention window and record cap. That prune
    deliberately does not check enabled — otherwise turning telemetry off would strand
    its rows forever. A test pins that.
  • The write is fail-soft. Telemetry that can break the pass it observes is worse than
    no telemetry, and the caller has nothing to do about the failure. This is the one place
    where swallowing is correct, and it is logged.

Schema

Purely additive, so no SCHEMA_VERSION bumpensure_schema is idempotent and runs
before apply_migrations on every start (the same precedent as training_files,
watch_state, tag_cooccurrence). Avoids the whole class of migration risk hit at v10.

Two things caught during review that are worth naming:

  • the table carries brain_id, so it belongs in _BRAIN_SCOPED_TABLES. A test caught the
    omission; without it the rows would have outlived the brain.
  • indexed on (brain_id, ran_at) from day one, because every query here filters on time,
    and an unindexed filter column means a full read (v10's lesson).

Verified against a live brain

telemetry OFF:  0 rows written
telemetry ON:   exactly 1 row per pass

counters:      209,070 processed / 192,130 decayed
weight_before: {0.01-0.05: 18,661, 0.05-0.1: 12,633, 0.1-0.25: 64,034,
                0.25-0.5: 70,676, 0.5-0.75: 16,286, 0.75-1: 9,840}
config:        {decay_rate: 0.1, prune_threshold: 0.01, min_age_days: 1.0}

That distribution is the thing #183 asked for: it answers "is the decay rate sane" without
keeping a row per edge to get there. Test rows were removed afterwards.

Verification

  • full suite: 7271 passed, 0 failed
  • lint / format clean; the single mypy error (google.genai) is identical on main
  • all three docs gates (CLI reference, config reference, scattered refs) regenerated and
    green before pushing

Closes #183.

Collapsing superseded change-log updates is lossless for replication but
discards the weight trajectory, and the only thing recording it was the
replication queue. This gives decay its own home for that data.

One aggregate row per pass, never a row per edge. The pass touches every
synapse, so per-edge rows would reintroduce exactly the unbounded growth
that made change_log a problem — an AST test asserts the write cannot
appear inside a per-entity loop.

DecayReport already carried most of it; what was missing:

- why a processed synapse was NOT decayed. "processed" minus "decayed"
  used to be an unexplained number, so a healthy pass (most edges simply
  not due) and a starved one (a gate stuck shut) rendered identically.
  Now counted per gate: pinned, idle-gate, bookmark.
- the weight distribution either side of the pass, bucketed. Denser near
  zero, where the prune threshold sits and where a mis-tuned rate does
  its damage.
- the knobs the pass ran with. A distribution is uninterpretable without
  them: the same shape means different things at different decay rates.

Off by default, like TraceConfig. Pruned from consolidation on retention
and record count — and that prune deliberately does NOT check `enabled`,
so turning telemetry off still cleans up what it accumulated.

The write is fail-soft: telemetry that can break the pass it observes is
worse than no telemetry, and the caller has nothing to do about it.

New table is purely additive, so no SCHEMA_VERSION bump — ensure_schema
is idempotent and runs before apply_migrations on every start. It carries
brain_id, so it is registered in _BRAIN_SCOPED_TABLES; a test caught that
omission, without which its rows would have outlived the brain. Indexed on
(brain_id, ran_at) from day one, because every query here filters on time.
@acidkill
acidkill merged commit 7795323 into main Aug 27, 2026
9 checks passed
@acidkill
acidkill deleted the feat/decay-telemetry branch August 27, 2026 17:37
@acidkill acidkill mentioned this pull request Aug 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Decay telemetry: record weight trajectories somewhere that is not the sync queue

1 participant