feat: Incremental reprocessing of Sync events

[stevensJourney]

Part 2 of 3

Summary

Part 1 compiles replication events into the sync plan and provides behavioral equality for the compiler's JavaScript event model. This PR uses that model to make replication events participate in MongoDB storage v3 incremental reprocessing.

When a sync config is deployed, an unchanged event definition keeps its assigned stream-local id and reuses existing snapshot coverage. A new or behaviorally changed event receives a new id, and only its source tables require new snapshot work.

Motivation

Incremental storage already assigns persisted ids to bucket data definitions and parameter lookups. Those ids are stored as source-table memberships, allowing compatible definitions to reuse an existing snapshot while new definitions are processed separately.

Events previously had no equivalent membership. Storage knew that a physical table could trigger an event, but it did not know which event definition or version a particular SourceTable represented. When incremental processing split one physical table across multiple SourceTable records, storage also needed a separate event-carrier workaround to avoid firing the same event more than once.

Treating events as first-class mapped definitions gives snapshot selection, event delivery, and cleanup the same durable identity to work from.

Event mapping and identity

Each sync config now persists an event-name-to-id mapping alongside its bucket and parameter mappings. Event ids are opaque hexadecimal counters scoped to one replication stream. They are assigned from the largest id reserved by current or historical mappings, so an id is not reused for an unrelated event after its original definition is removed.

The id is not derived from event content. Instead, the compiler-model equality from Part 1 decides whether a definition in the new sync config is compatible with one in an existing config. A compatible event reuses the earlier mapping's id. A new event, or an event with changed behavior, receives the next id.

Matching includes the event name and compiled behavior. Two configs may use the same event name, but they share an id only when their compiled definitions are also compatible. Reordering unchanged event definitions does not create new ids or snapshot work. Ordering only affects which counter values are assigned when several genuinely new events are introduced together.

Persisted events are recompiled from their retained SQL when mappings are compared. This ensures compatibility uses the JavaScript compiler model rather than serialized-plan structure or raw SQL text.

Source-table membership and snapshots

MongoDB storage v3 source-table documents now store event_definition_ids alongside bucket and parameter memberships. Memberships for multiple SourceTable records representing the same physical table remain disjoint.

A reused event id can use an existing snapshot-complete source table. A new or changed event has an uncovered id, so reconciliation creates separate source-table work and snapshots the relevant table for that event. Snapshot completion checks include event memberships, and the reverse mapping from event id to sync config id ensures the work contributes only to configs using that event.

An event-only source table now has one or more event memberships but no bucket or parameter memberships. It does not need to retain source-row state after its snapshot is complete.

Event delivery and cleanup

Hydrated events remain grouped by their originating sync config so each definition can be resolved through that config's mapping. The stream then deduplicates compatible definitions by assigned id. Each event id is owned by exactly one SourceTable record for a physical table, so a source change fires that event once even when other definitions caused the table to be split across several records.

Stopped-config cleanup now removes unused event memberships in the same pass as bucket and parameter memberships. A source table is deleted when none of its memberships remain. A table retained only by live event memberships becomes event-only and its source-record collection is dropped. This removes the previous need to reparse persisted sync configs and special-case event-only tables by physical table name.

Scope and follow-up

This PR changes incremental event processing for MongoDB storage v3. Legacy and non-incremental storage continue to select events by physical table reference.

Part 3 uses the event-to-sync-config association established here to scope custom checkpoint reads to the collection associated with the active reading sync config.

Validation

Tests verify that unchanged event definitions retain their ids, reuse snapshot-complete source tables, and fire once. Changed definitions receive new ids and create snapshot work. Coverage also verifies event-only tables, split source tables, sync-config attribution, snapshot blocking, and cleanup when event memberships are retained or removed.

---

AI usage: Guided Codex GPT 5.6 to implement this. Reviewed manually and with Claude Opus 4.8.

[changeset-bot]

🦋 Changeset detected

Latest commit: 6a66e91

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 19 packages

Name	Type
@powersync/service-sync-rules	Minor
@powersync/service-core	Minor
@powersync/service-module-mongodb-storage	Minor
@powersync/service-jpgwire	Patch
@powersync/service-core-tests	Patch
@powersync/lib-services-framework	Patch
@powersync/service-module-convex	Patch
@powersync/service-module-mongodb	Patch
@powersync/service-module-mssql	Patch
@powersync/service-module-mysql	Patch
@powersync/service-module-postgres-storage	Patch
@powersync/service-module-postgres	Patch
@powersync/service-module-core	Patch
@powersync/service-image	Minor
test-client	Patch
@powersync/service-schema	Minor
@powersync/lib-service-postgres	Patch
@powersync/service-rsocket-router	Patch
@powersync/lib-service-mongodb	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR