Skip to content

feat: Use Compiler for Event Definitions - #762

Draft
stevensJourney wants to merge 10 commits into
mainfrom
event-plan-compiler
Draft

feat: Use Compiler for Event Definitions#762
stevensJourney wants to merge 10 commits into
mainfrom
event-plan-compiler

Conversation

@stevensJourney

@stevensJourney stevensJourney commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

Part 1 of 3

Summary

Replication events defined through event_definitions are currently parsed and evaluated by a separate legacy implementation. Sync streams and buckets already use the sync-plan compiler.

This PR compiles replication events into the sync plan and evaluates them through the same row-projection machinery used by streams. It also introduces behavioral equality for the compiler's JavaScript event model, which the next PR will use to decide whether an event definition can be reused during incremental reprocessing.

This PR does not assign persisted event ids or change incremental-reprocessing behavior. Those storage concerns remain in part 2.

Motivation

Replication events map rows from a source table to payloads consumed by the service. Custom checkpoint requests are one use of these events:

event_definitions:
  checkpoint_requests:
    payloads:
      - SELECT user_id, checkpoint, client_id FROM checkpoints

Incremental reprocessing needs to distinguish a behavioral event change from a non-functional SQL edit. Comparing raw SQL would treat formatting and equivalent operand ordering as changes. For example, these definitions compile to equivalent event behavior:

SELECT user_id, checkpoint, client_id
FROM checkpoints
WHERE active = TRUE AND checkpoint > 0
select "user_id", "checkpoint", "client_id"
from "checkpoints"
where 0 < "checkpoint" and true = "active"

The comparator works from the compiler's JavaScript model instead of serialized SQL or a content-derived UUID. This keeps compatibility decisions independent of the serialized plan format and allows the comparison rules to evolve with the compiler.

What changed

event_definitions are now compiled for every supported sync-config edition and stored in an additive events field in the serialized sync plan. Compiled events use the shared row-projection implementation, allowing the separate SqlEventDescriptor and SqlEventSourceQuery evaluators to be removed.

Behavioral equality includes the event name, source tables, filters, and projected payload. It ignores raw SQL formatting and payload-query order. Reordered and regrouped AND and OR expressions compare as equivalent, as do swapped = and IS operands. Reversed comparisons such as a > b and b < a are normalized to the same representation.

Comparison remains conservative for constructs that are not explicitly normalized. Failing to recognize an equivalence may cause a later deployment to reprocess unnecessarily, while different behavior must never be treated as reusable.

The existing event-query restrictions remain. Each payload query must read from one physical source table and cannot depend on request parameters. An event can define at most one payload query for a given source table.

Persisted-plan compatibility

Compiled events are an additive sync-plan field and do not require a plan-version bump.

The service continues to dual-write the raw event SQL alongside the compiled plan so older service versions can ignore the new field and use their legacy event evaluator. When a new service loads an older persisted plan without compiled events, it compiles that sidecar SQL at the loading boundary and uses the compiled evaluator from then on.

Follow-up

Part 2 uses the behavioral equality added here to assign or reuse stream-local event ids, store those ids as source-table memberships, and resnapshot only the source tables associated with new or changed event definitions.

Validation

Tests cover event compilation, serialization, deserialization, hydration, and row evaluation. They verify behavioral equality across formatting, payload-query ordering, commutative operands, and reversed comparisons, while ensuring changed behavior is detected.

Compatibility tests cover both newly compiled plans and older plans containing only raw event descriptors. Validation tests cover unsupported joins, request parameters, and duplicate source tables.


AI usage: Guided Codex GPT 5.6 to implement this. Reviewed manually and with Claude Opus 4.8.

@changeset-bot

changeset-bot Bot commented Aug 21, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 0ada267

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 19 packages
Name Type
@powersync/service-sync-rules Minor
@powersync/service-core Minor
@powersync/service-jpgwire Patch
@powersync/service-core-tests Patch
@powersync/lib-services-framework Patch
@powersync/service-module-convex Patch
@powersync/service-module-mongodb-storage Patch
@powersync/service-module-mongodb Patch
@powersync/service-module-mssql Patch
@powersync/service-module-mysql Patch
@powersync/service-module-postgres-storage Patch
@powersync/service-module-postgres Patch
@powersync/service-module-core Patch
@powersync/service-image Minor
test-client Patch
@powersync/lib-service-postgres Patch
@powersync/service-rsocket-router Patch
@powersync/lib-service-mongodb Patch
@powersync/service-schema Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant