Skip to content

feat: add event-driven replication plugin using afterQuery hooks#144

Open
aadityaranjan01 wants to merge 1 commit intoouterbase:mainfrom
aadityaranjan01:feat/replication-plugin-clean
Open

feat: add event-driven replication plugin using afterQuery hooks#144
aadityaranjan01 wants to merge 1 commit intoouterbase:mainfrom
aadityaranjan01:feat/replication-plugin-clean

Conversation

@aadityaranjan01
Copy link
Copy Markdown

/claim #72

Closes #72

Summary

This PR introduces a replication plugin that propagates write queries (INSERT, UPDATE, DELETE, REPLACE) to peer StarbaseDB instances using the existing plugin system.

Replication is implemented using the afterQuery hook and ExecutionContext.waitUntil, ensuring zero impact on primary query latency.


Design Approach

Rather than introducing polling or scheduled sync, replication is implemented via query interception. This aligns with:

  • StarbaseDB’s plugin architecture
  • Cloudflare Worker runtime constraints (no persistent timers)
  • Existing patterns established by the CDC plugin

Key Properties

  • Non-blocking: replication runs in waitUntil (no added latency)
  • Fail-safe: replication errors never affect primary query execution
  • Parallel fanout: uses Promise.allSettled to isolate replica failures
  • Selective replication: optional table-level filtering
  • Minimal surface area: no new APIs, no changes to existing flows

Differentiation from CDC Plugin

While the CDC plugin focuses on emitting change events for observability, this plugin focuses on write propagation across instances, enabling simple replication setups.


Usage

const replicationPlugin = new ReplicationPlugin({
  replicas: ['https://replica.example.workers.dev'],
  authToken: 'your-token',
})

replicationPlugin.onEvent(ctx)

Notes

  • Replication is intentionally event-driven (not polling-based)
  • Replica instances must share compatible schema
  • Avoid pointing replicas back to the primary to prevent loops

Why this approach

This implementation prioritizes:

  • architectural alignment
  • minimal integration risk
  • production safety

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Replicate data from external source to internal source with a Plugin

1 participant