feat: Incremental Custom Checkpoints

[stevensJourney]

Part 3 of 3

Summary

Part 2 gives each replication event a stream-local id and ties that id to the sync configs and source tables that use it. This PR uses that association to scope MongoDB storage v3 custom checkpoint requests to the exact event definition that produced them.

Custom checkpoints from active and processing sync configs can now coexist during an incremental deployment. Client reads continue to use the collection associated with the active sync config until the replacement config becomes active.

Motivation

MongoDB previously stored custom checkpoints for all replication streams in one shared collection, with records distinguished by replication stream id. That is not sufficient when an active sync config and its replacement are being processed in the same stream at the same time.

If the checkpoint event changes, Part 2 assigns the replacement definition a new event id and snapshots it separately. The processing config may begin producing custom checkpoints before it is active. Reading the greatest checkpoint across the stream would expose that processing state to clients still served by the old config.

The read must instead follow the active sync config's mapping. An unchanged event can safely share its existing collection, while a changed event needs a separate collection until activation.

Selecting the checkpoint event

Custom checkpoint mode now accepts an event name through setWriteCheckpointMode({ mode: CUSTOM, eventName }). The integration supplies the configured event name rather than an event id or resolver. MongoDB storage v3 resolves that name through the active sync config's persisted mapping whenever it reads custom checkpoint state.

This keeps storage-assigned ids internal. Integrations do not derive ids from SQL or compiled event content, and a later change to event comparison does not change the external configuration contract.

MongoDB storage v3 requires an event name in custom mode and rejects unknown names. One event owns custom checkpoints for a storage instance, so an integration using this mode must use the same configured event for its custom checkpoint flow.

Writing scoped checkpoints

Incremental replication event payloads now include the assigned event_id. An event handler copies that value to addCustomWriteCheckpoint(), allowing the write path to route the checkpoint to the event version that actually fired. MongoDB storage v3 rejects missing or unknown event ids.

Each event stores its records in custom_checkpoint_requests_${stream_id}_${event_id}. A collection contains the latest custom checkpoint for each user for that event definition. Collections and their user, operation-id, and retention indexes are created lazily, so unrelated events do not create empty checkpoint collections.

Compatible event definitions reuse the same assigned id and collection across deployments. A changed definition receives a new id and writes to a separate collection while both sync configs coexist.

Reading through the active sync config

The sync read path now passes its active hydrated sync config through checkpoint watches and checkpoint-change lookups. Storage resolves the configured event name using that config's single-config mapping and reads only the corresponding event collection.

This means checkpoints written by a changed processing event remain invisible to clients until the new sync config is activated. After activation, new reads resolve the same configured event name through the replacement mapping and begin reading its collection. When the event is unchanged, both mappings resolve to the reused id and existing checkpoint state remains available without a transition.

Lifecycle and compatibility

Request-derived custom checkpoints retain their checkpoint_requested_at metadata. MongoDB v3 compaction applies the existing retention policy across the per-event collections, while persistent source-owned checkpoints remain unaffected.

Stopped-config cleanup drops a custom checkpoint collection when no live sync config uses its event id. Clearing a replication stream also removes all of its scoped custom checkpoint collections.

The per-event collection model applies only to MongoDB storage v3. Legacy MongoDB storage versions and Postgres continue using their existing unscoped custom checkpoint storage. Managed checkpoint mode is unchanged.

Validation

Tests verify independent writes and reads for multiple event definitions, lazy collection and index creation, missing and unknown event validation, and idempotent setup after a service restart. They also verify that an unchanged event retains its checkpoint state across deployment, while a changed processing event remains isolated until activation.

Additional coverage verifies checkpoint-change streaming, request-retention compaction, stopped-config cleanup, full storage clearing, and unchanged behavior for legacy storage implementations.

---

AI usage: Guided Codex GPT 5.6 to implement this. Reviewed manually and with Claude Opus 4.8.

[changeset-bot]

🦋 Changeset detected

Latest commit: a19cbd3

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 12 packages

Name	Type
@powersync/service-core	Minor
@powersync/service-module-mongodb-storage	Minor
@powersync/service-module-convex	Patch
@powersync/service-module-core	Patch
@powersync/service-module-mongodb	Patch
@powersync/service-module-mssql	Patch
@powersync/service-module-mysql	Patch
@powersync/service-module-postgres-storage	Patch
@powersync/service-module-postgres	Patch
@powersync/service-image	Minor
test-client	Patch
@powersync/service-schema	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR