Status: built (M13). Wonky Sheets and Wonky Slides sync through RevLog, a
server-sequenced, revision-ordered log of row changes. It replaced the vendored
serverpod_offline_sync CRDT engine. Wonky Write keeps its own OT protocol
(wonky_write/03), which RevLog's connection code was lifted from.
What RevLog is. Every spreadsheet or deck is one file with a head revision. Clients keep a local cache of the file's rows plus a durable outbox of their own changes. A change set (an op) is sent over a Serverpod stream; the server applies it to the domain tables in one transaction, gives it the next revision and fans it out to every other open connection. There is no operational transform: Sheets and Slides address everything by stable id, so ops are column assignments with "last arrival wins" per column and "delete wins" for rows. Offline editing works as before: the outbox survives reloads and is sent when the connection returns.
How this spec is organized. Parts 1–6 are the implementation specification (decisions, data
model, change sets, protocol, server, client). Part 7 holds the performance gates. References of
the form 05 §3.3 point to part 5 §3.3 of this file. The As built section below records
where the implementation differs from parts 1–7; where they disagree, As built wins.
Code: packages/wonky_revlog (pure Dart core: schema, codec, apply rules, overlay, client state
machine, chunker; test doubles in testing.dart), packages/wonky_revlog_flutter (client cache,
transport, per-file connection, per-user client; fakes in testing.dart),
wonky_server/lib/src/platform/revlog/ (sequencer, connection, pages, hooks, headless cache, jobs)
and the bindings wonky_server/lib/src/sheets/revlog/, wonky_server/lib/src/slides/revlog/,
wonky_sheets/lib/features/revlog/, wonky_slides/lib/features/revlog/.
Differences from parts 1–7, and decisions made while implementing:
-
Client cache layout (02, 06 §2). Domain rows live in generic tables instead of one table per schema table, so the cache needs no DDL per schema and survives schema changes with a reset (
RevDomainRows,wonky_revlog_flutter/lib/src/store/):rev_row(fid, tbl, id, data)(WITHOUT ROWID) holds structure rows, one record each;rev_cells(rid, fid, tbl, scope, row_key, data)holds the rows ofcellsByRowtables (Sheets cells) with one record per sheet row: every cell of that row in one blob, so a viewport or streaming read touches one record per sheet row and a bootstrap page writes 500 records instead of 13,000;rev_cell_id(fid, h, rid)finds a cell's record from its id (an applied update or delete carries no sheet/row key):his 52 bits of the id (exact on the web), and the decoded ids are checked, so collisions only cost a decode. A bootstrap writes its records without these entries and adds them, sorted, in the transaction that marks the file complete (random-key inserts dirtied the whole index on every page); until then nothing looks cells up by id (domain writes wait for the end of the bootstrap; a page's replayed changes find their cells in the page).
fidis an integer per file (rev_fid),tblthe table's index in the schema, anddataa compact binary row encoding (RevRowWriter: the column's index and a tagged value; UUIDs as 16 bytes, integers as varints, null columns and the key columns left out). A 1M-cell file takes ~110 MB (~112 bytes per cell; the JSON layout took 700). Layout changes are cache migrations that reset server state; the outbox stays. The other tables (rev_meta,rev_local_file,rev_coverage,rev_local_op,rev_outbox,rev_outcome) are as specified; a complete file keeps no coverage keys (a new bootstrap starts from a reset). -
File ids. The file uuid is the drive item uuid and
DriveItem.contentRef(decks and spreadsheets created on a device use their own uuid for both). -
Creation is atomic (04, 05 §8.3).
RevLogService.createFileinsertsrev_file, the hook state and every initial chunk in one transaction (deterministic op idsuuid v5(file, 'init:i'));initPendingis not needed. Files created on a device (initialize: false) take the client's creation op as their first op. The cache of a file created on a device is marked complete at once, so it reopens offline. -
Wire shapes (04).
RevRemoteOpnests aRevOpView. Server-side writes use session idsserver:<actorKind>. -
Server changes (05 §7).
submitServerChangeorders changes parents first (parentsFirst) and normalizes column values per column (uuids as strings, UTC times), so server writers may passUuidValues. Records for undo:ServerChangeRecord.recordIdlinks the record to its ops (rev_op.recordId); the legacy inline/file inverse fields stay for records written before (composite assistant changes);spreadsheetUuidalso holds deck file uuids. -
Apply rules (03 §4). An update is checked only against the parents it changes (moving an element to a layout or a group no longer drops it as
parentMissing). -
Times. Planners use millisecond-precision time, the precision the codec stores and sends, so a writer's model equals everyone else's.
-
Partial caches (06 §5.4). Reopening a partly loaded file reloads it from the server (the outbox stays) instead of resuming per coverage key.
-
Several tabs (06 §2.6). A tab adopts the outboxes of tabs that have been gone for 10 minutes.
-
Rejections (06 §4.6). The model rolls back, the plan's undo step is removed when it is the top step (a buried step's undo finds nothing to revert and is skipped by the per-column rule), and one toast names the change and the reason. Save my changes as a copy is a follow-up.
-
Conflicts (06 §6). The Sheets and Slides conflict reviews keep tracking the cells and text bodies written offline and compare them after the outbox drained; the deciding op comes from
rev_local_op. Outcomes are stored and exposed assession.outcomes; a review fed directly by outcomes is a follow-up. -
Loading gates (06 §5.5). Built as specified: deleting a sheet or columns waits for the whole sheet, deleting rows waits for those rows.
-
Script user properties. Server table
script_user_property(read/written withsheets.getUserProperties/setUserProperties), with a copy and the unsent changes in the user's local store for offline runs. -
Headless models (05 §9).
HeadlessWorkbookProvider.openkeeps its lease until the request's session closes; the cache evicts only after a new entry is leased. -
Unsynced-data download (06 §8). The recovery download of a cache that can't be opened is not built yet; on such a failure the editor offers Try again. Clear local data (
RevOfflineProvider.clear) empties the cache, outboxes included, without exporting them first (06 §7.1). -
Test doubles (09 §1).
FakeRevServer(wonky_revlog/testing.dart) andFakeRevHub/FakeRevTransport(wonky_revlog_flutter/testing.dart): an in-process server with the real apply rules, used by the client, Sheets and Slides tests. -
Client-side creation (04 §5, 06 §7). The clients keep calling
drive.registerSpreadsheet(spaceUuid, parentUuid, name)/registerPresentation, which now meancreate*(fileUuid: spaceUuid, initialize: false)(with a fallback to My Drive when the parent folder is gone or no longer writable), andPendingDriveOpkeeps the kindsregisterSpreadsheet/registerPresentation. ThecreateSpreadsheetendpoint has nofileUuid/initializeparameters (the service has them);createPresentationhas both. -
Assistant runs (RD-21, 05 §8.1). Each assistant tool call is its own recorded server change; at the end of a run its records are combined into one composite record (
ServerChangeRecords.compositeinplatform/revlog/server_change_records.dart: the calls' ops move to one record id), which is the run's single undo step. Reverting it reverts the calls' changes in reverse order with the skip rule. -
MCP
changes.revertkeeps "own changes only" (Editor+, not yet reverted, 24 h), stricter than the endpoints' Editor+ rule. -
Enums (RD-5, 02 §2.1). No pure-Dart twins in
wonky_sync_models: the Sheets and Slides enums are Serverpod enums generated next to the server tables and shared throughwonky_client; the cores use enum names (RevColumnType.enumName). -
Deck repairs (06 §1.2).
LoggedDeckSession.runMaintenancewaits while another connection's op group is half applied or this tab's outbox is not empty, and retries later. -
Overlay (03 §7).
Overlay.build(...)in parts 3 and 6 is the classRevOverlay(wonky_revlog/lib/src/apply/overlay.dart) withremote,acked,rejectedandrebuild; the per-sheetcell.fileIdis a plain column without a foreign key (02 §2.1), so deleting arev_filedoesn't scan the cell table. -
Performance (R6, part 7). Server rows are read as
json_strip_nulls(row_to_json(t)); an op's prefetch is one statement for all tables (CTEs, stillFOR UPDATE); rows that did not exist at the prefetch are written with a plainINSERT, the others with the upsert of 05 §3.3; headless loads keyset per-sheet tables along their(sheetId, rowId)index; the file state caches the drive item id. The codec memoizes UUIDs per payload andBase62Uuidconverts with 16-bit limbs;RevOverlaykeeps an index of the outbox (key → ops). The client cache writes rows in batches, stores row JSON without null columns and rewrites only the outbox rows that changed. The measured gates and the misses (P7, P8, P10 as one transaction, P9's planner) are in part 7. -
Client performance (R7, part 7).
- Reopen from the cache is viewport-first (06 §5.1): the structure
(
RevLocalStore.loadStructure), the cells of the plan's first block (loadScopedAt), the outbox on top, then ready; the rest of the cells stream in from the cache through the sameRevBootstrapPlan(250 sheet rows per read), soLoadingGate,#LOADINGand the shimmer work as during a cold open. Exactness as for pages (06 §5.3): each read returns its rows with the revisionRsof the stored state (one read transaction); ops processed while streaming are buffered; the buffered changes newer thanRsare replayed on the rows read; rows the outbox touches are seeded in the overlay with that server state and rebuilt (RevFileSync._applyCached). A bootstrap from the server stops the streaming. Decks (no cell table) load everything as before. - A finished bootstrap stores its revision (
markComplete(revision:), the last op processed), so the file reopens from the cache; before, a file without ops since its download keptrevision = 0and was downloaded again. - Page writes don't hold up the model or ready (P6): they are queued in order and not
awaited, with backpressure on the oldest writes beyond 120,000 queued rows; consecutive
queued pages are written in one transaction (
applyPages), and new cell rows in one round trip to the database worker (computeWithDatabase, native; the web runs the statements). A store write that waits for the UI isolate between its statements fell behind a download. - Model and planner (P7, P9): one cell id hash per written cell and
Uuid5without per-call allocations; row and column number formats read once per row and column of a paste; plan changes ordered by rank with buckets instead of a comparison sort; cells kept out ofWorkbookModel's owner map (a sheet lookup finds a cell's sheet); the formula engine's cell keys are records instead of 110-character strings, and its range index buckets the entries added since its last rebuild by blocks of 16 rows (stabs scanned all of them); loaded rows are wrapped (RowChange.loaded) instead of copied; the chunker no longer composes two full chunks of one plan just to find they don't fit. _SheetsPlanhands out each row once (a walk per sheet instead of copying the missing rows per block), so several cell pages can be in flight (06 §5.2).
- Reopen from the cache is viewport-first (06 §5.1): the structure
(
-
Wire and server performance (R7, part 7).
- Page format version 2 (04 §3.1): binary and columnar (ids once, per-column blocks
with presence bitmaps, UUID and string dictionaries, varints, zlib where it saves ≥ 20 %),
negotiated with
RevPageRequest.format; version 1 (gzip NDJSON) is still served to clients withoutformatand decoded from servers that ignore it. Decoded rows leave null columns out unless the column has a non-null default. A 13,000-cell page: 360 → 250 KB, encode 52 → 4 ms, decode 64 → 13 ms. - Server pages (05 §7) read each table as one row of typed arrays (
array_agg, UUIDs asuuid_sendbytes) straight intoRevPageBuilder; cell pages probe the(sheetId, rowId)index once per sheet row (unnest … LATERAL … OFFSET 0), because withrowId = ANY(…)and statistics older than a big sheet Postgres scanned the whole sheet throughcell_col_idx(2.4 s per page at 1M cells); the cascade of deleted rows' cells uses the same form.RevAnalyzeanalyzes the cell table in the background after every 100,000 committed cell inserts. - Client pages (06 §5.2): up to 3 cell pages in flight, applied in request order;
decoding off the UI thread (
RevPageDecoder: an isolate per page on native, ~8 ms slices on the web). - Commits (05 §3.3): the prefetch reads typed arrays (UUIDs as text) instead of JSON
rows; inserts and updates send one binary array per column (
unnest); rows the op read areUPDATEd in place with only their changed columns (falling back to the upsert when a database cascade removed one earlier in the op); the applied form is encoded and gzipped once (an all-insert op's view reuses it); file creation answers the prefetch from memory (_CreatedRows) and builds no views. The cell table hasfillfactor = 85(RevStorage, ensured at server start and inwithWonkyServerpod, since models can't declare storage parameters), so rewritten cells can be HOT.
- Page format version 2 (04 §3.1): binary and columnar (ids once, per-column blocks
with presence bitmaps, UUID and string dictionaries, varints, zlib where it saves ≥ 20 %),
negotiated with
-
Pre-RevLog files are removed on open. The migration kept the drive items of older spreadsheets and decks but not their data, so they can't be opened (no
rev_file). When a connection or apage/ops/infocall finds norev_filefor a file uuid that names a drive item of the app's kind (bycontentRef, or by item uuid when it has none) that the caller can see,RevLegacyFiles.removeIfLegacypurges that item for everyone (DriveTrash.purge: shares, stars, stored files, aremoveddrive event) and the open is rejected as not found. New items are inserted in the same transaction as theirrev_file, so only leftovers match; items the caller can't see are kept.
This is the authoritative list of choices for the RevLog implementation. Later files refer to them as RD-n. The review questions Q1–Q3 were decided on 2026-10-18 (§3).
| Term | Meaning |
|---|---|
| File | One spreadsheet or one deck. Its id (file UUID) is the drive item UUID, the id of the root row (Spreadsheet.id / Deck.id) and DriveItem.contentRef, as today |
| Root table | spreadsheet (Sheets) or deck (Slides) |
| Domain tables | The 24 Sheets and 13 Slides tables that hold file content (02 §2), minus change_log and deck_change_log, which are removed |
RowChange |
One row-level change: {table, rowId, op: insert/update/delete, before, after}. The existing class, moved to wonky_revlog (03 §1) |
| Change set (op) | An ordered list of RowChanges plus metadata (03 §2). It is the unit of submission, sequencing and acknowledgement |
| Group | The ops that form one undo step or one server operation (groupId); a plan larger than one op is split into a group |
| Revision | A dense integer per file, assigned by the server. Revision r is the file state after applying ops 1…r; revision 1 is the creation op |
| Materialized state | The current rows of a file in the server's domain tables |
| Outbox | The client's unacknowledged change sets for a file (one in flight plus buffered), persisted before sending |
| Sequencer | The server code path that assigns revisions: RevLogService.submit |
| Binding | The per-app plug-in (Sheets or Slides) that provides the table schema, validation, normalization and hooks |
| Applied form | The normalized version of an op that the server actually committed (RD-20, 03 §4.7) |
| Record | The ops of one revertible server change or assistant run (recordId, RD-21) |
| # | Decision | Why |
|---|---|---|
| RD-1 | The server totally orders the ops of a file by revision. One sequencer per file (Postgres row lock on rev_file) across instances; one in-memory lock per file per instance in front of it, as Write does |
Same guarantee as Write's OT, without the transform |
| RD-2 | No transform. A change assigns absolute column values (after) to rows addressed by stable ids. Ops are applied in revision order, so later ops overwrite earlier ones per column (LWW by sequence). Concurrent inserts of the same id merge per column: an insert into an existing row writes only its non-null columns (the rule of today's offline-sync patch OS-4) |
Sheets and Slides address everything by id (rows, columns, cells, elements, formula references, ranges) |
| RD-3 | Delete wins, and parents must exist. A change to a row whose row or a parent row is gone is dropped and reported (dropped). Parent/child relations come from RevSchema, and the sequencer cascades explicitly: when it deletes a row, it also deletes its descendants. Database FKs with ON DELETE CASCADE remain as a safety net, except for cell.rowId/cell.columnId, which become plain columns (02 §2.1) because an FK on them would need two extra indexes on the largest table. Plans must list every descendant they delete so that undo can restore them. The planners already do (planDeleteDimension, planDeleteSheet, deleteElementTree), but only for what the model contains. So deleting rows, columns or sheets requires those parts to be loaded (LoadingGate, 06 §5.5), and a property test enforces the rule (09 §2.4) |
Today's FK projection gives the same "delete wins" result |
| RD-4 | The materialized state is in ordinary server tables. The domain tables move from packages/wonky_sync_models (database: sync) to wonky_server/lib/src/{sheets,slides}/models/rows/ as serverOnly: true tables (a shared package can only declare all/sync tables; inventory §4). That is 22 Sheets tables (the 24 .spy.yaml minus change_log and script_user_property) and 12 Slides tables (minus deck_change_log). They get fileId (FK to rev_file) and revision columns and lose spaceId |
No CRDT metadata. One row per cell |
| RD-5 | The table schema is shared as pure Dart. The types (RevSchema, RevTable, RevColumn) live in wonky_revlog. The instances live in the cores: sheetsSchema in wonky_sheets_core, slidesSchema in wonky_slides_core (02 §4). The server, the cores and the clients use them for codecs, SQL and the client cache DDL; a server guard test compares them with the generated tables. The domain enums that the cores use today from generated code (Dimension, HorizontalAlign, SheetKind, …, 108 importing files) become hand-written pure-Dart enums in wonky_sync_models with the same names and values. The server keeps Serverpod enums for its tables, and a guard test compares the value names. wonky_sync_models keeps ids and the enums, and becomes pure once Slides has moved (R8). Renaming it is optional and not part of this work |
One source for three consumers; the generated client classes are no longer needed |
| RD-6 | The op log replaces the synced change logs. rev_op stores every op: author, actor, label, summary, detail (≤ 200 cells, as today) and the compressed applied changes including before. After 30 days the payload, summary and detail of an op are pruned, but a slim row (fileId, revision, clientOpId, author, committedAt) stays for a year. That keeps resends idempotent across long offline periods (05 §10). change_log and deck_change_log disappear |
One record per change, written in the same transaction; history, cell history, revert and activity read it |
| RD-7 | Transport. One Serverpod bidirectional stream per open file (SheetsDocEndpoint.connect / SlidesDocEndpoint.connect), like Write's document.connect. Unary RPCs for bootstrap pages and op ranges. No stream message exceeds 1 MiB compressed (Serverpod has no WebSocket size limit, so we impose one; inventory write-infra §5). Payloads over 16 KiB are gzip-compressed |
Proven pattern in this codebase |
| RD-8 | Immediate fan-out. ClusterBus.post gets immediate: true, which publishes at once instead of waiting for the 1 s batch (today only Slides presenting forces a full flush). Used for rev:<fileUuid> channels and Write's doc: op messages |
Cross-instance propagation within the latency budget |
| RD-9 | Access is the drive item's ACL. AccessService is checked on connect, on every submit and on every RPC. Role changes reach open streams through the existing ACL event path (acl_events.dart → bus role message → re-check), as Write does. Spaces, memberships, reservations and subscriptions are removed |
One permission model for all apps; expired grants stop writes at once (fixes code-quality CQ-16 for these apps) |
| RD-10 | Side effects run after the ack from durable hook cursors. Activity, drive modifiedAt/search text, cell history, version state, title mirroring, comment orphaning, script triggers and linked-chart invalidation run in a per-file queue after commit, debounced about 1 s. Per file, one rev_hook_state row holds every hook's cursor and a lease, so only one instance drains a file. The hooks are at-least-once and resume after a crash (05 §6) |
Never inside the sequencer lock (Write's mistake, CQ-22); no lost side effects |
| RD-11 | The client store is a cache. It holds the server state at a known revision, the outbox, recent op metadata and view data. It is never the source of truth. When its schema version differs from the app's, the client drops the domain tables and re-downloads them, keeping the outbox (06 §2.4) | No lock-step schema releases |
| RD-12 | Opening is viewport-first from the server. A file that is not on the device opens after its first page (structure + visible rows); the rest streams in. Pages carry the revision they were read at, so they combine exactly with ops (05 §7) | Big files open in about a second |
| RD-13 | Offline creation uses client UUIDs. The client generates the file UUID and queues an idempotent drive.createSpreadsheet / drive.createPresentation (PendingDriveOp). The creation op is the file's first outbox op. No reserved spaces |
Same as Write's offline createDocument |
| RD-14 | Validation on the server; repairs stay on the client. Every submit is checked against the schema and limits (02 §4, 03 §5). The deterministic, idempotent maintenance repairs stay where they run today: Slides' 12 DeckMaintenance rules and Sheets' AnchorMaintenance, executed by an editor's client after remote batches. In addition, Sheets wires the core's WorkbookMaintenance, which exists but is not called anywhere today. The server hook RevBinding.normalize exists and is empty in this migration (03 §6) |
Repairs need the whole model, which the clients have and the sequencer should not load. A sequencer makes conflicting repairs rare, and the repairs are idempotent anyway |
| RD-15 | Undo is unchanged. sheets.rows / slides.rows invert against the current model with the per-column skip rule. Server changes keep ServerChangeRecord, but the inverse is computed from the op group's stored before values instead of a separate inverse file (05 §8) |
No behavior change for users |
| RD-16 | Slides follows Sheets with the same packages and its own binding (spec file 08). Text bodies stay last-writer-wins per body, with soft locks | One sync system for both apps |
| RD-17 | No data migration. Destructive server migrations; client replica files of the old engine are deleted on first start. Protocol bump: minClientVersion → 3.0.0 |
No users yet |
| RD-18 | Cell hints are reduced. The cell and range commit hints are removed; typing and pending stay (07 §6) |
Data arrives in about one round trip |
| RD-19 | Protected ranges are enforced by the server (decided, Q2). For Sheets ops on files with protections, the sequencer runs the core's ProtectionGuard, the same code as the client's write gate, against the server's model of the file at the previous revision. An op that writes a protected cell, row, column, rule, filter, object or sheet the author may not edit is rejected whole (protectedRange). Warning-only protections never block on the server (the client asked for confirmation). Files without protections pay nothing. Design: 05 §3.5 |
Today protection is advisory: a modified or stale client, or an offline edit made before a protection arrived, can write protected cells |
| RD-20 | Every committed op has one applied form, and only that form is ever replayed. The server normalizes the submitted changes into the applied changes (03 §4.7): • a merged insert becomes an update of the columns it wrote, with their before;• dropped changes are removed; • cascaded deletes are added. Remote ops, pages, catch-up and the author's own ack all deliver this form, or say it equals the submitted form, so the store never re-derives server state |
Exact convergence without trusting client-side re-application or capped outcome lists |
| RD-21 | Groups and records are different things. groupId ties the ≤ 5,000-change chunks of one plan together (atomic for undo and hints). recordId ties the ops of one revertible server change or one assistant run (ServerChangeRecord) together |
One id with two meanings broke groupLast (review item 29) |
| RD-22 | One sender per file per device. Several tabs of one user may open the same file. Each tab sends only the outbox rows it created (tagged with its tab id). Rows of a closed tab are adopted by the background sync of another tab, which coordinates through the Web Locks API (desktop: a process-level lock). Store writes are monotonic in revision (06 §2.6) | Multi-tab use is allowed (suite 03) |
Q1 · How does a late offline edit resolve against a newer online edit of the same cell field? Decided: last to arrive wins. Ops apply in arrival order (RD-2), for online and offline ops alike.
- Reporting. The server reports the fields whose value at apply time differed from the change's
beforeand were last written by another user (conflicts, 03 §4.5). - UX. The offline author sees "Your offline changes replaced 3 newer edits by Bo — Review", with Restore theirs per cell (06 §6). This matches Google Sheets and Write.
- Considered and not chosen: keeping the newer value for offline ops (today's semantics).
- Specs to rewrite: Sheets 10 §9–§10 and Slides 09 §6.3 (spec file 11).
Q2 · Enforce protected ranges on the server? Decided: yes, in this migration (RD-19, design in 05 §3.5). Protections stop being advisory. Suite 06 §3 and Sheets 07 §10.3 drop their "advisory" notes (spec file 11).
Q3 · Keep the generated domain-table classes on the client?
Decided: no (RD-4: serverOnly: true). The clients use RevSchema column maps and the cores'
own records, so the client protocol shrinks by about 35k generated lines. If some client code
still needs a typed row class, it gets a hand-written record in the core.
| Limit | Value | Enforced by | Error code |
|---|---|---|---|
| Changes per op | 5,000 | client chunker, server chunker (server changes), server | tooLarge |
Uncompressed op payload (incl. before) |
16 MiB (a 2 MiB script file update is 4 MiB plus overhead) | server, before decode | tooLarge |
| Stream message | 4 MiB after compression. Chunkers cut ops by count and by encoded size (target ≤ 1 MiB compressed), so a message normally stays far below the cap. There is no separate large-submit RPC: unary request bodies are capped at 512 KiB (maxRequestSize), stream messages are not (inventory write-infra §5) |
both | tooLarge |
Batches of ops (RevInit.ops, ops() responses) |
≤ 1,000 ops and ≤ 1 MiB compressed in total. When either limit is hit, the batch stops early and the client gap-fills (04 §2.2) | server | — |
| Bootstrap page | ≤ pageRows rows and ≤ 4 MiB compressed |
server | — |
| Single column value | 50,000 chars (Cell.content, SlideElement.text); 1 MiB for JSON columns; 2 MiB for ScriptFile.source |
server (RevColumn.maxLength) |
invalidChange |
| Cells per spreadsheet | sheets.maxCellsPerSpreadsheet (1,000,000; grid size counts) |
server stats, client | limitExceeded |
| Sheets per spreadsheet | sheets.maxSheetsPerSpreadsheet (200; exists today in ServerLimits, now enforced by the binding) |
server, client | limitExceeded |
| Elements per deck | No server limit (unchanged). 50,000 elements stays a client-side cap on what is kept offline without OPFS | — | — |
| Op-log retention | 30 days for payload, summary and detail; slim rows (idempotency) 365 days (opTombstoneDays) |
job | — |
| Outbox age (client) | 180 days. Older outbox rows are not resent; the client offers them as Save my changes as a copy (06 §8) | client | — |
| Editors per file | 100; later joiners become viewers (Write's rule) | server | — |
| Offline availability (client) | Opened in the last 14 days (max 50 files per app) or pinned. Files with an outbox are never evicted | client | — |
| Offline cell cap without OPFS (client) | 200,000 cells (Sheets), 50,000 elements (Slides): only limits what is kept offline | client | — |
New section wonky.revlog in config/*.yaml (read by WonkyConfig, env override
WONKY_REVLOG_<KEY>; suite 01 §6):
wonky:
revlog:
maxChangesPerOp: 5000
maxOpBytes: 16777216 # uncompressed payload incl. before
maxMessageBytes: 4194304 # one stream message, compressed (hard cap)
targetOpBytes: 1048576 # chunkers aim below this, compressed
compressAboveBytes: 16384
opsInInit: 1000 # and ≤ 1 MiB compressed in total
opRetentionDays: 30 # payload/summary/detail
opTombstoneDays: 365 # slim rows for idempotency
maxEditorsPerFile: 100
pageRows: 1000 # sheet rows per cell page
structurePageRows: 20000 # rows per structure/rows page (and ≤ 4 MiB)
hookDebounceMs: 1000
hookRetryMaxSeconds: 600
hookLeaseSeconds: 60
headlessBudgetMb: 2048 # HeadlessFileCache, both apps together
headlessMaxEntries: 100
schemaWindowDays: 14 # how long the server accepts an older compatible schema hash
sheets:
maxCellsPerSpreadsheet: 1000000 # kept
# maxActiveSpacesPerUser: removed
slides:
# maxActiveSpacesPerUser: removed (no new keys)
cluster:
busFlushMs: 1000 # kept; RevLog posts use immediate: trueProtocolInfo (suite 01 §7) replaces syncTablesHash with revSchemaHash (the hash of RevSchema
for both apps, 02 §4.3) and raises minClientVersion to 3.0.0.
All models are Serverpod .spy.yaml. Field lists of today's domain tables are in
packages/wonky_sync_models/lib/src/models/**; only the changes are listed here.
class: RevFile
table: rev_file
serverOnly: true
fields:
fileUuid: UuidValue # = drive item uuid = root row id
kind: RevFileKind # spreadsheet | deck
item: DriveItem?, relation(onDelete=Restrict) # itemId; purge deletes rev_file first (05 §10)
headRevision: int, default=0
prunedBelow: int, default=0 # ops < prunedBelow have no payload (05 §10); RevInit uses it
initPending: bool, default=false # createFile wrote the row but not all initial ops yet (05 §8.3)
purging: bool, default=false # purge in progress: refuse connects/submits (05 §10)
schemaHash: String # RevSchema hash of the last write
stats: String, default='{}' # JSON, binding-defined (§1.1)
contentCount: int, default=0 # Sheets: grid cells (Σ rows × cols); Slides: elements (informational)
createdAt: DateTime
updatedAt: DateTime
indexes:
rev_file_uuid_idx: { fields: fileUuid, unique: true }
rev_file_item_idx: { fields: itemId, unique: true }
class: RevOp
table: rev_op
serverOnly: true
fields:
file: RevFile?, relation(onDelete=Cascade) # fileId
revision: int
clientOpId: UuidValue
baseRevision: int
authorId: int? # user_profile id (null for system)
authorUuid: UuidValue? # auth user uuid (sent to clients)
sessionId: String? # client connection session id, or "server:<runId>"
onBehalfOfSessionId: String?
actorKind: ActorKind
label: String
groupId: String? # chunk group of one plan (RD-21)
groupIndex: int, default=0
groupLast: bool, default=true
recordId: UuidValue? # ServerChangeRecord / assistant run (RD-21)
offline: bool, default=false
payload: ByteData? # gzip codec JSON of the APPLIED changes incl. before (RD-20); null once pruned
changeCount: int
summaryJson: String? # null once pruned
detailJson: String? # ≤ 200 cells; null once pruned
outcomeJson: String? # full outcome (04 §2.2) for idempotent re-acks; null once pruned
appliedEqualsSubmitted: bool # true: the applied form is exactly the submitted changes
clientCreatedAt: DateTime?
committedAt: DateTime
indexes:
rev_op_rev_idx: { fields: fileId, revision, unique: true }
rev_op_client_idx: { fields: fileId, clientOpId, unique: true }
rev_op_record_idx: { fields: recordId }
rev_op_committed_idx: { fields: committedAt, type: brin } # prune jobs
class: RevHookState
table: rev_hook_state
serverOnly: true
fields:
file: RevFile?, relation(onDelete=Cascade)
cursorsJson: String # {"<hookId>": <last processed revision>, …}
leaseOwner: String? # instance id draining this file
leaseUntil: DateTime?
attempts: int, default=0
lastError: String?
nextAttemptAt: DateTime?
updatedAt: DateTime
indexes:
rev_hook_state_file_idx: { fields: fileId, unique: true }
rev_hook_state_due_idx: { fields: nextAttemptAt }
enum: RevFileKind
serialized: byName
values: [spreadsheet, deck]| Binding | JSON |
|---|---|
| Sheets | {"tables":{"sheet":3,"sheet_row":3000,"sheet_column":78,"cell":312,…},"sheets":{"<sheetId>":{"rows":1000,"cols":26,"cells":312}}}. contentCount = Σ rows × cols over grid sheets, the quantity capped by sheets.maxCellsPerSpreadsheet (Sheets 02 §3.2: "the grid size counts") |
| Slides | {"tables":{"slide":12,"slide_element":340,…}}; contentCount = elements |
tables gives per-table row counts. The client uses them to plan structure loading (06 §5.2); the sequencer uses tables.protected_range to skip the protection check on files without protections (05 §3.5).
The sequencer keeps the stats in the same transaction from the applied op's insert and delete
counts (05 §3.2); revlog.repairStats recomputes them (05 §10).
| Sheets | Slides | |
|---|---|---|
| Move from | packages/wonky_sync_models/lib/src/models/*.spy.yaml |
…/models/slides/*.spy.yaml |
| Move to | wonky_server/lib/src/sheets/models/rows/ |
wonky_server/lib/src/slides/models/rows/ |
| Tables moved | 22 (the 24 files minus change_log and script_user_property) |
12 (the 13 files minus deck_change_log) |
| Enums | 10 Sheets enums | 7 Slides enums |
Per model:
| Change | Detail |
|---|---|
database: sync |
removed → a server table; add serverOnly: true (Q3) |
spaceId (implicit) |
gone |
new file: RevFile?, relation(onDelete=Cascade) |
fileId on every domain table. Set by the sequencer, never by clients; not part of RevSchema |
new revision: int |
The revision that last wrote the row. Set by the sequencer; not part of RevSchema. Used for debugging and as the starting point of conflict attribution (03 §4.5, 05 §3.2) |
| domain FKs | unchanged relation(onDelete=Cascade, deferred), except cell.row and cell.column: they become plain rowId: UuidValue and columnId: UuidValue fields with no DB FK. The sequencer cascades cells explicitly by (sheetId, rowId) / (sheetId, columnId) using the existing indexes (05 §3.3). A DB FK on them would need (rowId) and (columnId) indexes on the largest table so that row/column deletes don't scan it. cell.sheet keeps its FK; its index (sheetId, …) serves it |
id, createdAt |
unchanged |
Enums. The server gets Serverpod enums (.spy.yaml, serialized: byName) in the same
folders. The cores and apps use hand-written pure-Dart enums with identical names and values in
packages/wonky_sync_models/lib/enums.dart (RD-5). On the wire and in column maps, enums are
names, so no conversion code is needed beyond byName. A guard test compares the value lists.
ActorKind moves to platform/revlog/models/ (the platform uses it) and also has a pure-Dart
twin.
- Keep every existing index (inventory models §1–§2).
- Add
<table>_file_idx (fileId, id)to every domain table exceptcell, for structure pages, purge and headless loads. cellis never read or deleted byfileId. Whole-file cell access goes sheet by sheet (WHERE "sheetId" = ANY(@sheets)or one sheet at a time), usingcell_row_idx (sheetId, rowId). This rule applies to headless loads, version capture, search text, content size, stats repair and purge (05 §§6, 9, 10). Acell (fileId)index would cost about 40 MB per million cells and is not needed.
| Removed | Replaced by |
|---|---|
change_log (Sheets, synced) |
rev_op |
deck_change_log (Slides, synced) |
rev_op |
script_user_property (synced, personal space) |
server table script_user_property (§2.4) |
space_subscription, space_reservation (server) |
nothing (RD-9, RD-13) |
sheets_change_log_receipt, deck_change_log_receipt (server) |
rev_hook_state |
reserved_space, sheets_dirty_space, sheets_journal_space, sheets_row_journal (client) |
the client cache tables (§3) |
crdt_*, offline_sync_* (engine, server and client) |
nothing |
SheetsMergedSignal (bus message) |
RevBusMessage (05 §5) |
Client database migrations. wonky_client/lib/migrations/ is one generated linear chain
(Sheets client migrations followed by slides-s1). Never edit it by hand.
- R5: once the Sheets synced models and client tables are gone, regenerate the client migrations. The generator produces a migration that drops them.
- R8: once the Slides ones are gone too and no
database: client/syncmodel is left, the server'sconfig/generator.yamllosesexperimental_features.databaseSync, theserverpod_offline_syncmodule and theshared_packagesentry. The client database is then removed through the generator.
class: ScriptUserProperty
table: script_user_property
serverOnly: true
fields:
user: UserProfile?, relation(onDelete=Cascade)
fileUuid: UuidValue
key: String
value: String
updatedAt: DateTime
indexes:
script_user_property_idx: { fields: userId, fileUuid, key, unique: true }Endpoints sheets.getUserProperties(fileUuid) and
sheets.setUserProperties(fileUuid, Map<String,String?> changes) (null = delete); Viewer+, since
the properties are private per user. The client cache and offline queue are in 06 §7.1. Server
scripts read and write the table directly (ServerUserProperties, today
script_service.dart:1209-1300).
| Table | Change |
|---|---|
server_change_record |
spreadsheetUuid → fileUuid. Remove inverseJson, inverseFileId, forwardFileId. Add recordId: UuidValue (= rev_op.recordId of its ops). Revert/reapply compute the changes from the ops' applied payloads (05 §8.2). Keep changeId, cellsAffected, expiresAt (≤ 30 days, within payload retention), revertedAt, revertChangeId |
cell_history |
changeLogId: UuidValue → revision: int; spreadsheetUuid → fileUuid. Unique (fileUuid, revision, rowId, colId) |
sheets_snapshot_index, slides_snapshot_index |
Add revision: int (the file revision captured). changedRangesJson / changesJson are built from rev_op.summaryJson between two capture revisions |
sheets_ai_preview_record |
Rename to ai_preview_record (AiPreviewRecord); spreadsheetUuid → fileUuid (Slides uses it too). Done (CQ-52), also server_change_record.spreadsheetUuid → fileUuid; shared code in platform/ai/ai_preview_records.dart and platform/revlog/server_change_records.dart |
item_version (platform) |
For spreadsheets and decks the captured revision goes into the existing ItemVersion.revision: int? (platform/versions/item_version.spy.yaml:13, used by Write today). VersionRestoreResult.revision reports the restore's last revision |
sheets_import_job, script_execution, script_trigger_state, script_job, script_authorization, ai_function_cache |
Unchanged; their spreadsheetUuid columns hold the file uuid |
drive_item |
contentRef = file uuid (unchanged) |
File <app>_cache_<userUuid>.db (sheets_cache_…, slides_cache_…), opened with
LocalDatabases.open(…, migrations: revCacheMigrations(schema)) (06 §2). The old
sheets_<userUuid>.db / slides_<userUuid>.db replica files are removed on first start
(06 §2.1).
-- Bookkeeping (fixed DDL, in wonky_revlog_flutter)
CREATE TABLE rev_meta (key TEXT PRIMARY KEY, value TEXT NOT NULL);
-- 'schemaHash' = RevSchema hash the domain tables were created with (06 §2.4)
CREATE TABLE rev_local_file (
file_uuid TEXT PRIMARY KEY,
item_json TEXT, -- DriveItemView JSON (offline open, titles)
role TEXT,
revision INTEGER NOT NULL, -- server state stored = state at this revision; only ever increases (06 §2.6)
complete INTEGER NOT NULL DEFAULT 0, -- 1 = every row loaded (available offline)
pinned INTEGER NOT NULL DEFAULT 0,
last_opened_at INTEGER,
size_bytes INTEGER NOT NULL DEFAULT 0,
created_locally INTEGER NOT NULL DEFAULT 0, -- waiting for the queued drive create (06 §7.1)
orphaned INTEGER NOT NULL DEFAULT 0 -- queued create failed permanently (06 §8)
);
CREATE TABLE rev_coverage ( -- bootstrap coverage while incomplete (06 §5.3–5.4)
file_uuid TEXT NOT NULL,
key TEXT NOT NULL, -- 'tbl:<table>:<fromId>:<toId>' (structure ranges) or 'row:<sheetId>:<rowId>' (cells)
revision INTEGER NOT NULL, -- page revision Rp
PRIMARY KEY (file_uuid, key)
); -- collapsed to 'sheet:<sheetId>' once a sheet's cells are all loaded
CREATE TABLE rev_local_op ( -- recent op metadata for offline history (06 §3.3)
file_uuid TEXT NOT NULL,
revision INTEGER NOT NULL,
author_uuid TEXT, session_id TEXT, actor_kind TEXT NOT NULL,
label TEXT NOT NULL, group_id TEXT, record_id TEXT,
summary_json TEXT, detail_json TEXT, offline INTEGER NOT NULL DEFAULT 0,
committed_at INTEGER NOT NULL,
PRIMARY KEY (file_uuid, revision)
);
CREATE TABLE rev_outbox (
file_uuid TEXT NOT NULL,
seq INTEGER NOT NULL, -- local order
tab_id TEXT NOT NULL, -- creating tab/process (RD-22)
client_op_id TEXT NOT NULL,
base_revision INTEGER NOT NULL,
payload TEXT NOT NULL, -- codec JSON incl. sum/det (uncompressed)
schema_hash TEXT NOT NULL, -- hash the payload was encoded with (06 §2.4)
label TEXT NOT NULL, actor_kind TEXT NOT NULL,
group_id TEXT, group_index INTEGER NOT NULL DEFAULT 0, group_last INTEGER NOT NULL DEFAULT 1,
offline INTEGER NOT NULL DEFAULT 0,
inflight INTEGER NOT NULL DEFAULT 0, -- may have been sent; resend with the same id
created_at INTEGER NOT NULL,
PRIMARY KEY (file_uuid, client_op_id)
);
CREATE INDEX rev_outbox_seq ON rev_outbox (file_uuid, seq);
CREATE TABLE rev_outcome ( -- acked outcomes awaiting the review UI (06 §6)
file_uuid TEXT NOT NULL, revision INTEGER NOT NULL, outcome_json TEXT NOT NULL,
PRIMARY KEY (file_uuid, revision)
);
-- Domain tables, generated from RevSchema (§4), one per table:
-- CREATE TABLE <table> (id TEXT PRIMARY KEY, file_uuid TEXT NOT NULL, <columns…>);
-- CREATE INDEX <table>_file ON <table> (file_uuid);
-- plus the table's client indexes from RevSchema (e.g. cell (sheetId, rowId)).
-- Column types: uuid/string/json/enumName → TEXT; int/bool/dateTime → INTEGER (dateTime = UTC ms);
-- double → REAL. No foreign keys: cascades arrive as explicit deletes in applied ops and pages.The domain tables hold server state only. Unacknowledged local changes live only in
rev_outbox and in the in-memory model (the overlay; 03 §7).
Types in packages/wonky_revlog/lib/src/schema/; instances in the cores:
wonky_sheets_core/lib/src/revlog/sheets_schema.dart and
wonky_slides_core/lib/src/revlog/slides_schema.dart (RD-5).
enum RevColumnType { uuid, string, int, double, bool, dateTime, enumName, json }
final class RevColumn {
const RevColumn(this.name, this.type, {
this.nullable = false,
this.defaultValue, // applied on insert when the column is missing (03 §4.2)
this.enumValues, // for enumName
this.maxLength, // string/json (01 §4)
this.clientWritable = true, // false: columns only the server writes
this.immutable = false, // may not change by update (cell sheetId/rowId/columnId, deterministic-id keys)
});
}
final class RevParent {
const RevParent(this.column, this.table, {this.optional = false}); // cascade on delete (sequencer)
}
final class RevTable {
const RevTable(this.name, this.columns, {
this.parents = const [],
this.plainRefs = const [], // documentation + repair hints; never cascaded
this.indexes = const [], // client cache indexes
this.rank = 0, // dependency order (parents first)
this.deterministicIds = false, // cell, basic_filter, script_build, slide_table_cell, chart_link, theme copies…
this.clientWritable = true, // false: script_build
this.isRoot = false, // spreadsheet / deck
this.coverage = RevCoverage.structure, // how bootstrap pages cover this table (06 §5.3)
});
}
enum RevCoverage { structure, cellsByRow /* Sheets cell: key = sheetId + rowId */ }
final class RevSchema {
const RevSchema(this.app, this.tables, {required this.coupled, required this.columnwise,
this.compatibleWith = const {}});
final String app; // 'sheets' | 'slides'
final Set<String> compatibleWith; // older hashes whose payloads this schema accepts (additive changes only)
RevTable table(String name); // throws on unknown
RevTable? tryTable(String name); // decoders: unknown tables/columns of a compatible newer schema are skipped
int rank(String table);
String get hash; // UUIDv5 over canonical JSON of tables/columns/types/enums
}- The cores' table constants become part of the schemas.
SheetsTables,SlidesTables,SheetsUndo.columnwiseandSlidesUndoHooksmove ontosheetsSchema/slidesSchema(coupled,columnwise). This replacesSyncedTables,SlidesSyncedTables,SyncedTableSpecandSyncTableSet. - Plan-less summaries move. Plan-less change summaries are needed by the client (undo, redo, repairs) and the server (server changes). They move from the Sheets app (
SheetsWriter.summarize,wonky_sheets/lib/features/sync/sheets_writer.dart:68) intowonky_sheets_coreasSheetsChangeSummary.ofChanges(changes, current). Slides keepsSlidesChangeSummary.of(changes, wasDeleted: …, current: …)(change_summary.dart:89) in its core.
- Server (
wonky_server/test/unit/revlog/schema_guard_test.dart): for everyRevTable, the generated server table has the same columns, types and nullability, plusfileIdandrevision. Parents match the FKs, exceptcell.rowId/columnId, which have no FK (§2.1). Server and pure enums have equal value names. Built (code-quality plan, Wave 3): also checks that every server table versioned by RevLog (fileId→rev_fileplusrevision) is in a schema. - Cores: every table name the planner uses exists in the schema;
coupled/columnwisereference existing columns;immutablecolumns are never updated by any planner op (property test).
Everything is driven by RevColumn.type:
RevValues.toWire/fromWire(codec, 03 §3.2);RevValues.toSql(server parameters with explicit casts);RevValues.toSqlite/fromSqlite(client cache).
In memory the column-map conventions stay as today: lower-case UUID strings, enum names, UTC
DateTime, double. So WorkbookModel and DeckModel ingestion doesn't change.
| Rule | Detail |
|---|---|
| Where the hash travels | Every payload ("s"), RevInit.schemaHash + RevInit.acceptedHashes (current + compatibleWith still inside schemaWindowDays), ProtocolInfo.revSchemaHash |
| Server accepts a payload | if its "s" is the current hash or in compatibleWith within the window. Otherwise schemaMismatch |
| Client, older than the server | If its hash is in RevInit.acceptedHashes: continue; the decoders skip unknown tables and columns of the newer schema (additive changes only). If not: stop sending and show "Update required" (suite 01 §7) |
| Client after an update | It re-encodes and re-stamps its outbox with the new schema on load (06 §2.4). Changes to tables or columns that no longer exist are dropped and reported once, so a stale hash never deadlocks the outbox |
| What counts as additive | A new nullable column or a new table. Each additive release adds the previous hash to compatibleWith. Anything else (rename, type change, removal) is a breaking release: bump minClientVersion |
RowChange, RowOp, MutationPlan, composeRowChanges, UndoInverter, RowsUndoEntry,
RowsUndoHandler, CoupledColumns, ColumnwiseInserts and CurrentRowReader move from
packages/wonky_sync_kit/lib/src/{rows,undo} to packages/wonky_revlog/lib/src/{rows,undo}.
Their public API stays as it is today (inventory: models §7). RowPatch becomes Overlay (§7).
SyncTxAdapter and SyncTableSet are deleted: they are replaced by RevBinding (05 §1) and
RevSchema (02 §4).
| Op | before |
after |
Meaning in RevLog |
|---|---|---|---|
insert |
— | every column the creator sets, plus id |
Create the row. If the id exists, the server writes the non-null columns of after into it (RD-2; see §4.2) |
update |
the touched columns' values as the author saw them | the touched columns' new values | Assign the named columns. before is used for conflict detection (§4.5) and undo |
delete |
the full row as the author saw it, plus id |
— | Delete the row. Children cascade (RD-3); the plan lists the children it knows |
Invariants checked by RowChange and the codec:
rowIdis a canonical lower-case UUID string.update.afteris non-empty, and its keys equal the keys ofbefore.idnever appears inupdate.after.- Columns exist in
RevSchema; values have the column's type (§3.2).
/// One op: the unit of submission, sequencing and acknowledgement.
final class ChangeSet {
ChangeSet({
required this.clientOpId, // UUIDv7, generated once, reused on every resend
required this.baseRevision, // the client's server revision when the op was cut
required this.changes, // ≤ RevLimits.maxChangesPerOp, ordered (§4.1)
required this.label, // undo label ("Paste", "Insert 3 rows"); "" for maintenance
required this.actorKind, // ActorKind name: user | mcp | ai | script | system
this.summary, // ChangeSummary JSON (Sheets: sheets, ranges, kinds, deleted…)
this.detail, // ≤ 200 cells before/after (cell history), as MutationPlan.detailJson
this.groupId, // shared by the ops of one plan / one server operation
this.groupIndex = 0, // 0-based position in the group
this.groupLast = true, // false on all but the last op of a group
this.offline = false, // true if any change was made while disconnected (reported in conflicts and the review UI, 06 §6)
this.createdAt, // client wall-clock UTC of the first change (display only)
});
}- Summary and detail. These are computed exactly as today:
MutationPlan.summaryJson/detailJsonfor plans;SheetsWriter.summarize(moved to the Sheets binding) for plan-less writes: undo, redo, AI, maintenance.
- Groups. A plan of more than
maxChangesPerOpchanges is split byMutationPlan.chunksinto one group. The summary goes on the first op of the group; the detail on the first op if it has ≤ 200 cells, otherwise none. The server storesgroupId, and history shows a group as one entry. - Composing ops. While an op is in flight, later local changes are composed into the buffered op (06 §4.2). Composing appends changes, runs
composeRowChanges, and merges summaries (union of sheets, ranges, kinds and deleted ids). Details are concatenated and capped at 200 cells. The label becomes that of the latest step, or "Edits" if they differ.offlineis OR-ed.
A change set is encoded as compact JSON. When the UTF-8 encoding is larger than
compressAboveBytes (16 KiB), it is sent gzip-compressed as bytes (04 §2).
- Row ids travel in the 22-character base-62 form (
Base62Uuid), the same form as formula reference tokens. They are decoded to canonical UUID strings before use. insertcarriesafteronly;updatecarries both arrays (the touched columns are those not equal to the "absent" sentinel);deletecarriesbeforeonly.- The metadata of §2 (
clientOpId,baseRevision,label, …) travels in the message, not in the codec JSON (04 §2).
Values are encoded per RevColumnType (02 §4):
| Type | JSON | Notes |
|---|---|---|
uuid |
base-62 string (22) | FK and plain-reference columns |
string |
string | |
int |
number (integer) | |
double |
number | Dart shortest round-trip; NaN/±Inf are rejected |
bool |
true/false |
|
dateTime |
integer (UTC milliseconds since epoch) | |
enumName |
string (enum value name) | Unknown names are rejected by the server (invalidChange); a client decoding a compatible newer schema keeps the name as-is (02 §4.3) |
json |
string (opaque JSON text, as stored) | Validated as JSON by the server; size limit per column |
| null | null |
Allowed only for nullable columns |
Round-trip property: decode(encode(cs)) == cs for every valid change set (property test,
09 §2.1). The client and server codecs are the same Dart code.
The rules are written once as pure Dart against an abstract row store. The sequencer runs this very function on prefetched rows (05 §3.2), and clients use it for the overlay (§7) and in tests. The SQL write step only persists its result, so the server's result equals the pure function's by construction; 09 §2.2 checks that the persistence is faithful.
abstract interface class RevRowStore {
Map<String, Object?>? row(String table, String id); // null = missing
void put(String table, String id, Map<String, Object?> row); // full row
void remove(String table, String id); // no cascade
Iterable<(String table, String id)> childrenOf(String table, String id); // via RevSchema parents
}
final class ApplyResult {
final List<RowChange> applied; // the APPLIED form (§4.7), incl. cascades
final List<DroppedChange> dropped; // target or a parent missing
final List<FieldConflict> candidates; // fields whose value at apply time ≠ before (§4.5)
final List<RowChange> cascaded; // children removed by cascade that the op didn't list
bool get equalsSubmitted; // applied == submitted changes (no drops, merges or cascades)
}
ApplyResult applyChangeSet(RevSchema schema, RevRowStore store, ChangeSet cs);Changes are applied in the order of the op. Planners already emit parents first for inserts and
updates and children first for deletes (PlanBuilder.build: stable sort by RevSchema.rank). The
client chunker and the composer keep that order. Rank can't order rows of one self-referencing
table (Slides slide_element.parentId: a group created in the same op as its children may come
after them), so RevChunker.split first runs parentsFirst(schema, changes)
(wonky_revlog/lib/src/apply/order.dart): an insert created in the op moves to just before the
first insert or update that references it, every other change keeps its place. The server does not
reorder; an out-of-order child insert is simply dropped (§4.2), and that is a client bug caught by
tests.
- Parents. Every non-null FK column of the row must reference an existing row (also
one inserted earlier in the same op). Otherwise the change is dropped
(
DroppedChange(reason: parentMissing)). - Absent row. Store the row:
afterplusRevSchemadefaults for missing columns. - Existing row (a deterministic id created by someone else, or a replay): write each
column of
afterwhose value is non-null (an insert that leaves a column null does not author it, OS-4 semantics). Columns whose current value is non-null and differs fromafterare conflict candidates (§4.5).
- Missing row → dropped (
rowMissing). This is "delete wins". - Immutable columns (
RevColumn.immutable) can't be updated; validation rejects such an op (§5).- These are
cell.sheetId,cell.rowIdandcell.columnId: a cell's id is derived from them, so cells never move; moving content writes other cells. - The same applies to the key columns of other deterministic-id tables:
slide_table_cellelement, row and column;chart_link.elementId.
- These are
- Parent change. If a mutable FK column is updated, the new parent must exist, otherwise the whole change is dropped (
parentMissing). Examples are an element re-parented into another group, or a slide moved between groups (groupIdis a plain reference, not an FK). - Columns. Write every column. A column whose current value differs from
beforeis a conflict candidate (§4.5).
- Missing row → nothing (idempotent; not reported).
- Present row → remove it and every descendant through
RevSchemaparents (cascade). The descendants that the op did not delete explicitly are returned ascascaded(they were created concurrently by someone else; 07 §5 shows them in the conflict review as "removed with row 7").
Plain references (Slide.layoutId, DimensionGroup.startId, pivot anchors, formula tokens,
GridRange JSON) are not cascaded. They resolve at evaluation time exactly as today
(Sheets 02 §2, Slides 02 §3), and the client repairs of §6 fix what must be fixed.
A candidate is a field (table, rowId, column) where the op's before value differs from
the value at apply time. Value equality follows RowPatch.sameColumns: numbers by value,
date-times to the millisecond, enums by name, JSON strings byte-wise. The pure function reports
every candidate; it knows nothing about authors.
Attribution is a server post-step after the apply, still inside the transaction (05 §3.2 step 3.7). For each candidate, find the op that last wrote that field:
- Start at the row's
revisioncolumn. - Walk back through
rev_oppayloads (applied changes) until one wrote the column. The walk is bounded to 50 ops per field and 500 fields per op; beyond that the author is unknown. - A field is a conflict only if its last writer is another user (by
authorUuid). The user's own other sessions and devices don't count, and neither do the user's own server changes on their behalf.
The Ack reports conflicts with theirs (the value before our write), byUser and byRev
(04 §2.2). The fields were overwritten: the last arrival wins (Q1, decided).
Given the same state and the same op, applyChangeSet produces the same applied form on the
server and on every client. Clients nevertheless never re-derive the server state with it: they
receive the applied form (§4.7, RD-20).
The applied form is the list of changes that turns the state at revision r−1 into the state at r, written so that it can be replayed literally, with no rules:
| Submitted | In the applied form |
|---|---|
| Insert of an absent row | insert with the full stored row (defaults filled) |
| Insert merged into an existing row (§4.2.3) | update of the columns it actually wrote, with their before. So undo, revert and replay never treat it as a whole-row insert or delete, and other users' columns survive |
| Update | update of the columns actually written |
| Delete | delete with the full row as stored (before) |
| Dropped change | absent |
| Cascaded descendants | deletes (full rows) after their parent's delete |
| Fix-ups (none in this migration) | appended |
Clients receive the applied form without before values (04 §2.2), since they never need
them. rev_op.payload keeps the before values for revert and attribution.
Run against the decoded op. Any failure rejects the whole op with RevRejected(invalidChange),
and the message names the first offending change.
changes.length ≤ maxChangesPerOp; payload size ≤maxOpBytes.- Every table is in the binding's
RevSchema. Clients may not write tables withRevTable.clientWritable == false(Sheetsscript_build; Slides none). - Every column exists, is writable by clients (
RevColumn.clientWritable), and has the right type and nullability (§3.2). Strings are withinmaxLength. JSON columns parse.immutablecolumns appear only in inserts. 3a. Ids belong to this file. Every id the op touches that exists on the server must belong to this file. The sequencer prefetches by id without a file filter and rejects foreign ids withinvalidChange(05 §3.2). Ids are global UUIDs, so this check is what keeps one file's op from writing another file's rows. insertof a root-table row is only allowed in the file's creation op (revision 1) and only withid == fileUuid. A root row can't be deleted through an op (deleting a file is a drive operation).actorKindisuserfor client submits.mcp,ai,scriptandsystemare only accepted from server-side callers (submitServerChange, 05 §8), with one exception: client scripts run in the browser submit withactorKind: script(Sheets 09 client runtime).- Binding checks (
RevBinding.authorize, 05 §1):- Sheets: the cell cap (
contentCount) andmaxSheetsPerSpreadsheet. - Roles for special tables:
script_triggerandscript_fileneed Editor; triggers keeprunAsUserId == author, as the server checks today inScriptService. - Slides: exactly one owner FK set on
slide_element.
- Sheets: the cell cap (
- Protected ranges (Sheets; RD-19, 05 §3.5): the op must not write protected cells, rows,
columns, rules, filters, objects or sheets that the author may not edit. Otherwise it is
rejected whole with
protectedRange. This runs inside the sequencer, after the in-memory apply (it checks the applied form against the model at the previous revision).
Server hook (empty in this migration). RevBinding.normalize(ctx, applied) may return extra
changes (fix-ups). They are applied in the same transaction and appended to the op's applied
form (§4.7), so the Ack carries them to the author (06 §4.4). The protocol and the client support fix-ups from day one so that a later binding can
use them without a protocol change. No binding returns any in this migration (RD-14).
Client repairs (as today, run by an editor's client after remote batches, debounced 1 s, only when no remote group is half-applied and the tab's outbox is empty (06 §1.2), not
undoable, label "Repair", actorKind: user (clients may not send system, §5 rule 5), summary flag "repair": true. A repair op rejected with protectedRange is not retried for that violation for 24 h; another user who may edit the protection repairs it when they open the file):
| App | Repairs | Status today |
|---|---|---|
| Sheets | AnchorMaintenance (object anchors) |
Wired in ObjectsController (objects_controller.dart:341-378) |
| Sheets | WorkbookMaintenance.plan (core mutations/maintenance.dart:15): duplicate sheet names → " (2)" on the larger id; duplicate named ranges → "_2"; frozen counts clamped; overlapping merges → keep the smaller id; unresolvable merges deleted; basic filters whose range no longer resolves deleted; pivot anchors on deleted rows/columns moved to the nearest survivor; group depths normalized |
Not wired anywhere today. R5 wires it next to AnchorMaintenance (one repair op per debounce) |
| Slides | The 12 DeckMaintenance rules (Slides 02 §10; spec file 08 §4) |
Wired in SyncedDeckSession.runMaintenance; moves to LoggedDeckSession unchanged |
Repairs are deterministic and idempotent. Two editors repairing at the same time produce the same changes, so the second op's changes are no-ops or identical values.
The client model is defined as model = applyAll(outbox ops, in order, rules of §4) over server state. The outbox is applied with the same applyChangeSet rules, not
by blindly re-writing after-values. That means:
- an outbox insert of a deterministic id merges only its non-null columns over a remote insert of the same id, so another user's
boldstays visible; - an outbox change whose row or parent was deleted remotely is dropped from the view.
When a remote op (applied form) arrives:
touched = rows the remote op changes, plus the rows the outbox changes for those ids
before = model rows for `touched` // = old server ⊕ outbox
server' = server rows for `touched` with the remote applied form replayed literally
after = applyAll(outbox ops restricted to `touched`, over server') // pure, §4 rules
patch = diff(before, after) // RowChanges; RowPatch.sameColumns → no flicker, no echo
model.applyChanges(patch) // emitted as one remote batch
Overlay.build(model, serverRows, remoteApplied, outbox, {schema}) implements this. Outbox
changes for rows outside touched are unaffected, so restricting the replay is exact.
Property test (09 §2.6, replaces remote_patch_property_test.dart): for random server
states, outboxes and remote ops, model.applyChanges(Overlay.build(...)) equals a full rebuild
applyAll(outbox, newServerState).
The protocol is generic: one set of message models for both apps, under
wonky_server/lib/src/platform/revlog/messages/ (sealed bases with their variants in the same
folder, as Write's write/messages/; inventory write-infra §4). Each app gets a thin endpoint
that names its binding.
/// wonky_server/lib/src/sheets/endpoints/sheets_doc_endpoint.dart (Slides: slides_doc_endpoint.dart)
class SheetsDocEndpoint extends Endpoint {
@override bool get requireLogin => true;
/// Opens an editing/viewing session on a spreadsheet (§2).
Stream<RevServerMessage> connect(Session session, UuidValue fileUuid, int knownRevision,
String tabId, Stream<RevClientMessage> input) async* {
yield* RevLogConnection.open(session, SheetsRevBinding.instance, fileUuid, knownRevision,
tabId, input);
}
/// Bootstrap page (§3). Viewer+.
Future<RevPage> page(Session session, UuidValue fileUuid, RevPageRequest request);
/// Ops in (from, to]: ≤ 1,000 ops and ≤ 1 MiB compressed (fewer if a limit is hit; the
/// caller continues from the last returned revision). Access rule: 05 §11.
Future<List<RevOpView>> ops(Session session, UuidValue fileUuid, int fromRevision, int toRevision);
/// Head revision, role and stats without opening a stream (background sync, Settings). Viewer+.
Future<RevFileInfo> info(Session session, UuidValue fileUuid);
}Endpoint names on the client: client.sheetsDoc.* and client.slidesDoc.*. File creation stays
with the drive endpoints (§5).
tabId is a random id per app instance (browser tab or desktop process), stable for its
lifetime. It tags the connection so that the server can skip echoes to the sending connection
(§2.2) and so that roster entries are per tab.
class: RevSubmit
extends: RevClientMessage
fields:
clientOpId: UuidValue # idempotency key (same on every resend)
baseRevision: int
payload: String? # codec JSON (03 §3) incl. "sum"/"det", when ≤ compressAboveBytes
payloadGz: ByteData? # gzip of the same JSON otherwise (exactly one of the two is set)
label: String
actorKind: ActorKind # user | script (03 §5 rule 5)
groupId: String?
groupIndex: int
groupLast: bool
offline: bool
createdAt: DateTime # client clock, display only| Message | Fields | Meaning |
|---|---|---|
RevSubmit |
above | One op (03 §2). Sent only after it is durable in the outbox (06 §4.1) |
RevRequestOps |
fromRevision, toRevision |
Gap fill. The server answers with RevRemoteOps, the same byte cap applying as for ops() |
RevPing |
t |
Keep-alive and latency, every 30 s (as Write) |
The payload JSON adds two keys to the codec object of 03 §3.1:
"sum": the summary JSON;"det": the detail JSON, or absent.
| Message | Fields | Meaning |
|---|---|---|
RevInit |
revision, role: AccessRole, sessionId, schemaHash, acceptedHashes: List<String>, stats (02 §1.1), prunedBelow, ops: List<RevOpView>?, more: bool, bootstrap: bool |
First message. ops: the ops in (knownRevision, revision] when knownRevision ≥ prunedBelow and there are at most opsInInit of them. The list is cut at 1 MiB compressed, and more: true tells the client to gap-fill the rest with ops(). bootstrap: true: the gap is too large or pruned, and the client must (re)load pages (06 §5) |
RevAck |
clientOpId, revision, duplicate: bool, applied? / appliedGz?, outcome? |
The op was committed as revision. applied (the codec JSON of the applied form without before, 03 §4.7) is present iff the applied form differs from the submitted changes (rev_op.appliedEqualsSubmitted == false). The client stores the applied form (or its own submitted changes when absent) as server state (06 §4.4). outcome is JSON for the UI (below); absent when empty |
RevRemoteOp |
revision, clientOpId, payload? / payloadGz? (applied form, no before), authorUuid, authorSessionId, actorKind, label, summaryJson, detailJson?, offline, groupId?, groupIndex, groupLast, recordId?, onBehalfOfSessionId?, committedAt |
An op committed by another connection. The sending connection never receives its own op as a RevRemoteOp: it gets RevAck (Write's rule, document_connection.dart:309). Other tabs of the same user receive it like any remote op |
RevRejected |
clientOpId?, code: RevRejectCode, message, detailsJson? (code-specific, e.g. the protections hit for protectedRange) |
The op was not committed (§4). clientOpId == null means a session-level error, and the server closes the stream after sending it |
RevRoleChanged |
role: AccessRole |
Effective role changed (sharing, expiry, editor limit) |
RevResync |
reason: String |
The server can't continue this session incrementally (for example after an admin repair). The client re-bootstraps (06 §5) |
RevPong |
t, headRevision |
headRevision lets the client detect a lost bus message and gap-fill (05 §5) |
Outcome JSON (in RevAck.outcome and RevOpView.outcomeJson; for the UI only, since the
state is carried by the applied form):
{
"dropped": [ {"t":"cell","id":"<b62>","op":1,"reason":"rowMissing|parentMissing"} ],
"conflicts": [ {"t":"cell","id":"<b62>","col":"content",
"theirs":"n:5", "byUser":"<uuid>", "byRev":4211} ], // overwritten (last arrival wins, Q1)
"cascaded": [ {"t":"cell","id":"<b62>"} ],
"truncated": false
}Each list is capped at 500 entries; truncated: true marks a capped list. Capping never affects
state.
class: RevOpView # in RevInit.ops and ops()
fields:
revision: int
clientOpId: UuidValue
payload: String? # applied form without before (null if pruned — never returned then)
payloadGz: ByteData?
authorUuid: UuidValue?
authorSessionId: String?
actorKind: ActorKind
label: String
summaryJson: String?
detailJson: String?
offline: bool
groupId: String?
groupIndex: int
groupLast: bool
recordId: UuidValue?
onBehalfOfSessionId: String?
outcomeJson: String? # lets the author recover a lost ack's outcome
committedAt: DateTime
class: RevFileInfo
fields:
fileUuid: UuidValue
revision: int
role: AccessRole
schemaHash: String
stats: String # 02 §1.1class: RevPageRequest
fields:
kind: RevPageKind # structure | rows | cells
cursor: String? # structure/rows: JSON {"t": <tableIdx>, "after": "<id>"} from the previous page; null = start
table: String? # kind == rows: the table (keyset over id)
sheetId: UuidValue? # kind == cells (Sheets)
rowIds: List<UuidValue>? # kind == cells: ≤ pageRows sheet rows
maxRows: int # server caps at structurePageRows (structure/rows) or pageRows (cells)
format: int? # newest body version the client reads (§3.1); null = a client from before negotiation
class: RevPage
fields:
revision: int # exact revision of every row in this page (05 §7)
body: ByteData # binary columnar body, version 2 (§3.1); gzip NDJSON (version 1) for clients without `format`
coverage: List<String> # coverage keys (§3.2)
excludedTables: List<String> # structure: tables the client must load with kind == rows
nextCursor: String? # structure/rows: cursor of the next page; null = doneThe body is self-describing: a binary body starts with RVP and its version byte, a version-1
body with the gzip magic. Negotiation: the client sends the newest version it reads in
RevPageRequest.format; the server serves min(format, its newest) (RevPageCodec.negotiate),
and version 1 when format is absent (a client from before negotiation). The client decodes
both versions; a version it does not know fails with a clear RevCodecException ("unsupported
page version N"), and loading reports it. Caches need no migration: a page is applied the same
way whatever its version.
Version 2 (current; RevPageCodec, RevPageBuilder in wonky_revlog/lib/src/codec/) —
binary, columnar, little-endian, unsigned varints (7 bits per byte) and zigzag varints for
signed values:
body := "RVP" 0x02 · flags (u8, 0) · str schemaHash · varint tableCount · table*
table := str name · varint n · varint columnCount · block ids · column*
column := str name · u8 RevColumnType.index · block
block := u8 enc · varint length · bytes enc: bits 0–1 presence (0 all null, 1 all
present, 2 bitmap), bit 2 dictionary,
bit 7 zlib-compressed bytes
str := varint byteLength · UTF-8
- ids:
n× 16 raw bytes (never compressed). - Presence: with a bitmap (
ceil(n/8)bytes, LSB first) the values follow for the present rows only; an all-null column is an empty block. - Values per type:
uuid16 bytes each, or a dictionary (varint d,d× 16 bytes,u8index width 1/2/4, indexes);string/json/enumNameUTF-16 lengths as varints then one UTF-8 blob (decoded once and sliced), or a dictionary (varint d,dlengths,u8width, indexes, blob of thedstrings) when values repeat;intzigzag varints;doublefloat64;boola bitmap;dateTimezigzag varints of the millisecond delta to the previous value. - Compression: a block of ≥ 256 bytes is zlib-compressed (level 1) when that saves ≥ 20 %; raw UUID blocks are never compressed (random bytes).
- Decoded rows carry
idand their non-null columns; a null column is left out (it reads as null, as in the client cache) unless the column has a non-null default, where the null is explicit. Unknown tables and columns (a newer compatible schema) are skipped; a known column of another type is an error.
A 13,000-cell page (500 sheet rows × 26) of the perf data is ~250 KB (version 1: ~360 KB); encoding takes ~4 ms and decoding ~13 ms on the desktop VM (version 1: ~52 / ~64 ms).
Version 1 (legacy): gzip NDJSON, a header line {"s", "t": [tables], "c": [[columns]…]}
then one line [tableIdx, "<b62 id>", [values aligned with c[tableIdx]]] per row, values in
the codec encoding (03 §3.2) with nulls written.
A page is authoritative for a set of rows at revision. Every row in that set is either in
the page (its state at revision) or does not exist at revision. Coverage keys are the same
strings the client stores in rev_coverage (02 §3).
| Key | Meaning |
|---|---|
tbl:<table>:<fromId>:<toId> |
Every row of <table> with id in (fromId, toId]. fromId = - for the start, toId = + for the end |
row:<sheetId>:<rowId> |
Every cell of that sheet row (Sheets) |
Structure pages walk the non-cell tables in rank order with a cursor {t, after}. A
structure page carries one tbl: key per table it touched. Tables with more than 50,000 rows for
the file (per RevInit.stats.tables) are listed in excludedTables on the first structure page;
the client loads them with rows pages instead (two in parallel, 06 §5.2).
| Code | When | Client reaction (06 §4.6, §8) |
|---|---|---|
invalidChange |
Schema/type/limit violation, a foreign id, an immutable column (03 §5) | Drop the op and the rest of its group; roll the model back; drop or trim the plan's undo step; one toast; error log |
tooLarge |
Op or message over the limits | Same as invalidChange (a chunker bug) |
limitExceeded |
Cell cap, sheet count | Same, with the server's message ("This spreadsheet has reached the limit of 1,000,000 cells") |
protectedRange |
The op writes protected cells, structure, rules, objects or sheets the author may not edit (05 §3.5). detailsJson lists the protections hit |
Same as invalidChange (drop the group, roll back, drop the undo step), with the blocked-cell message (GuardResult.blockedMessage). For offline ops, also offer Save my changes as a copy (06 §4.6). A repair op isn't retried for 24 h (03 §6) |
forbiddenForRole |
Role below Editor (ACL or editor limit), or a table needing a higher role | Keep the outbox, switch to read-only, offer Save my changes as a copy |
forbidden |
No access at all (session-level; the stream closes) | Unavailable state "You no longer have access…" (outbox kept for the copy offer) |
unknownBase |
baseRevision > head (the client is ahead of the server, e.g. a restored server database) |
Re-bootstrap; offer the outbox as a copy |
schemaMismatch |
Payload "s" not accepted (02 §4.3) |
Stop sending; "Update required" |
notFound |
The file doesn't exist (yet) | If a create for this file is queued: wait (06 §5.1). Else unavailable |
fileReadOnly |
The file is trashed or being purged | Read-only with the drive's trashed banner |
internal |
Unexpected server error | Retry with backoff, same clientOpId (idempotent) |
A rejection means the op is not in the log: none of its changes were applied.
Future<DriveItemView> createSpreadsheet(Session s, UuidValue? parentUuid, String name,
UuidValue? templateUuid, {String? locale, String? timeZone, // existing params, kept
UuidValue? fileUuid, bool initialize = true}); // new
Future<DriveItemView> createPresentation(Session s, UuidValue? parentUuid, String name,
UuidValue? templateUuid, {UuidValue? fileUuid, bool initialize = true});- Idempotency. With
fileUuidthe call is idempotent: a retry returns the existing item, and finishes initialization ifinitPending(05 §8.3). initialize: true. The server writes the content as revision 1…: the template's rows, or the default content (SpreadsheetFactorywith locale, time zone and the user's default fonts fromSheetsCreateDefaults;DeckFactoryfor decks).initialize: false. The file stays at revision 0 and the client's first op creates the rows. Used by every client-side creation that writes its own content:- offline creation (queued in
PendingDriveOp); - online creation from the Sheets/Slides apps when the client already holds the content: built-in Slides templates and blank decks, version copies, import-as-new (06 §7).
- offline creation (queued in
- The Drive app's New, MCP, AI and welcome content use
initialize: true.
| Removed | Replacement |
|---|---|
drive.registerSpreadsheet, drive.registerPresentation |
create*(fileUuid, initialize: false) |
sheets.reserveSpaces, slides.reserveSpaces, subscribe, unsubscribe, ensureSubscribed, spaceStats, sheetsSync.sync, slidesSync.sync |
nothing (info, connect) |
PendingDriveOp gains the kinds createSpreadsheet and createPresentation (args fileUuid,
parentUuid, name, initialize: false). They replace registerSpreadsheet and
registerPresentation (06 §7).
Title of a file created with initialize: false. The creation op inserts the root row with
the title the client had. If the drive item was renamed in between (a queued rename replays
before the creation op arrives), the sequencer sets the root row's title to the drive item's
current name while applying the creation op. The applied form then differs from the submitted
one, so the author learns the title from RevAck.applied. This is the only server-side
adjustment in this migration (RD-14).
client A (tab 1) server client B
|-- RevSubmit(c1, base=40) ------------>| validate, lock, apply, rev_op 41, head=41, commit
| |-- bus rev:<file> (immediate) --------------> RevRemoteOp(41)
|<---------- RevAck(c1, 41[, applied, outcome]) --|
| |-- hooks (debounced, after the ack)
A's op is committed as 41; B's op (base 40) as 42. On one instance, B receives
RevRemoteOp(41) before RevAck(42), because the server posts under the file lock in revision
order. Across instances, RevAck(42) can arrive first. In both cases B processes
revision-carrying messages strictly in order (06 §4.3): the ack waits for 41, which arrives by
bus or by gap fill. B applies 41 through the
overlay (its own op stays on top), then the ack of 42 moves 42 from the outbox into server
state. If 41 and 42 wrote the same field, the ack's outcome lists a conflict with byRev: 41
(when A is another user).
client: outbox = [c7 (inflight), c8, c9 (buffered, composed)]
|-- connect(knownRevision=40) ----------->|
|<----------- RevInit(revision=55, ops 41..55, more=false) --| (or bootstrap:true)
| apply 41..55 through the overlay; an op whose clientOpId is c7 is our own committed op → ack c7
|-- RevSubmit(c7, …) (if still unacked) ------------>| idempotent: returns the original revision
|<-------------------------------------- RevAck ------|
|-- RevSubmit(c8+c9 composed) ---------------------->|
- No age limit for transform. The server never refuses an old base, since ops need no transform. The only age limits are the idempotency horizon (slim op rows kept 365 days) and the client's 180-day outbox age (01 §4).
- Own op in the catch-up. An op in
RevInit.opsorops()whoseclientOpIdis in this tab's outbox is the tab's own committed op; its ack was lost. The client treats it as the ack and takes the outcome fromoutcomeJson.
|-- connect(0) ------------------------------->|
|<-- RevInit(revision=R0, bootstrap:true, stats) --| later ops arrive as RevRemoteOp
|-- page(structure, cursor=null) --------------->| in parallel: rows pages of excluded tables,
|-- page(cells, sheet S, top-20..top+200) ------>| then the viewport cells
|<------ RevPage(rev=R1, coverage …) ------------| apply page + buffered ops > R1 for covered rows
| … WorkbookReady after structure + first cells; the remaining pages stream in (06 §5) …
Module: wonky_server/lib/src/platform/revlog/. Bindings: sheets/revlog/, slides/revlog/.
The patterns are lifted from Write's OT server (write/ot/, inventory write-infra §1). The code
is new and generic over a binding, and the bugs listed in 01-tech-spike §6 are fixed by design.
| File | Contents |
|---|---|
rev_binding.dart |
RevBinding interface (§1) |
rev_log_service.dart |
submit, submitServerChange, createFile, revert, reapply (§3, §8) |
rev_sql.dart |
Set-based reads and writes driven by RevSchema (§3.3) |
rev_file_state.dart |
Per-instance file state: lock, revision, op ring, catch-up (§2) |
rev_log_connection.dart |
Stream connection (§4) |
rev_bus.dart |
Bus channel, messages, roster (§5) |
rev_hooks.dart |
Durable post-commit hooks (§6) |
rev_pages.dart |
Bootstrap pages (§7) |
headless_file_cache.dart |
Generic headless model cache (§9) |
rev_jobs.dart |
Prune, hook sweep, init sweep, stats repair, purge (§10) |
models/, messages/ |
02 §1, 04 §2 |
abstract class RevBinding {
RevFileKind get kind; // spreadsheet | deck
RevSchema get schema; // sheetsSchema (wonky_sheets_core) | slidesSchema (wonky_slides_core)
DriveItemKind get itemKind;
String get rootTitleColumn; // 'title' on spreadsheet / deck (04 §5 creation title)
/// Limits and roles beyond the schema (03 §5 rule 6). Throws RevRejection.
Future<void> authorize(RevTx tx, ChangeSet cs, ApplyResult applied, RevAuthor author, RevStats statsAfter, RevModelAccess models);
// RevAuthor {authUuid, role (effective), actorKind}; RevModelAccess.atPrevious() returns the headless model at revision head (before this op), loaded on first use (§3.5)
/// Fix-ups appended to the applied form (03 §6). Returns [] in this migration.
Future<List<RowChange>> normalize(RevTx tx, ApplyResult applied) async => const [];
/// rev_file.stats/contentCount from the applied form (02 §1.1).
RevStats updateStats(RevStats before, List<RowChange> applied);
/// Summary JSON for plan-less server changes, from the core
/// (SheetsChangeSummary.ofChanges / SlidesChangeSummary.of; 02 §4).
String summarize(List<RowChange> changes, CurrentRowReader current, {bool insertsAreRevivals = false});
/// Durable post-commit hooks, in run order (§6).
List<RevHook> get hooks;
/// Headless model (§9).
Object buildModel(UuidValue fileUuid, Iterable<(String table, Map<String, Object?> row)> rows);
void applyToModel(Object model, List<RowChange> applied);
int estimateBytes(Object model);
/// Tables read sheet by sheet instead of by fileId (Sheets: {'cell'}; 02 §2.2).
Set<String> get perSheetTables;
}SheetsRevBinding.instance and SlidesRevBinding.instance are registered at startup
(server.dart). This replaces registerScriptHooks()' change-log plugin and the Slides plugin
registration in SlidesServer.ensureRegistered.
final class RevFileState {
final UuidValue fileUuid; final int fileId; final RevBinding binding;
int revision; // last revision known to this instance
final RevOpRing recent; // ≤ 1,000 RevOpViews and ≤ 32 MiB of payload; larger payloads are not kept (fetched from rev_op)
Future<T> lock<T>(Future<T> Function() body); // promise-chain mutex (Write's DocumentState.lock)
Future<void> ensureAtLeast(Session s, int revision); // coalesced catch-up from rev_op
}
abstract final class RevFileStates {
static Future<RevFileState> of(Session s, RevBinding b, UuidValue fileUuid);
static RevFileLease retain(RevFileState st); // connections, server changes, headless entries
static void release(RevFileLease lease); // state disposed 60 s after the last lease
static void evict(UuidValue fileUuid); // purge (§10), tests
static void clear(); // tests
}static Future<RevCommit> submit(Session session, {
required RevBinding binding,
required Actor actor,
required RevFileState state,
required ChangeSet cs, // decoded + size-checked (§3.2 step 0)
required Role connectionRole, // the connection's effective role (ACL ∧ editor limit), or the server change's
String? sessionId, // the connection's sessionId (used to skip the echo, §4)
String? onBehalfOfSessionId,
UuidValue? recordId, // server changes (§8)
bool serverChange = false,
});
final class RevCommit {
final int revision; final bool duplicate;
final List<RowChange> applied; // applied form (with before)
final bool appliedEqualsSubmitted;
final RevOutcome outcome; // dropped, conflicts (attributed), cascaded
final RevOpView view; // bus/ring view (applied form without before + metadata)
}-
Before any lock.
- Check sizes before decoding: the message against
maxMessageBytes, the uncompressed payload againstmaxOpBytes. - Check
"s"against the accepted hashes (02 §4.3), elseschemaMismatch. - Run the static validation of 03 §5 rules 1–5 (immutable columns included).
- Check sizes before decoding: the message against
-
Idempotency fast path.
SELECT revision, payload, "appliedEqualsSubmitted", "outcomeJson" FROM rev_op WHERE "fileId"=@f AND "clientOpId"=@c. If found, returnduplicate: truewith the stored applied form and outcome. A pruned payload means the op is older than 30 days; returnapplied = nullandoutcome = null, and the client then re-bootstraps the affected file (06 §4.5). -
Access.
role = min(AccessService.effectiveRole(session, actor, item), connectionRole).effectiveRolerespects grant expiry, andconnectionRolecarries the editor limit.- Below Editor →
forbiddenForRole. - Item trashed, or
rev_file.purging→fileReadOnly.
This runs on every submit.
-
state.lock(...), then one transaction:- Lock the file.
SELECT id, "headRevision", stats, "initPending" FROM rev_file WHERE id=@f FOR UPDATE, which serializes the file across instances.cs.baseRevision > head→unknownBase. - Idempotency again, inside the transaction. A hit returns the duplicate; this closes Write's race (CQ-5).
- Catch up:
state.ensureAtLeast(head). - Prefetch (§3.3).
- Load the current full rows of every id the op touches, by id alone, without a file filter. Any row whose
fileId ≠ @f→invalidChange("foreign id"; 03 §5 rule 3a). - Check the parents the inserts and parent-changing updates need, within the file.
- Load the current full rows of every id the op touches, by id alone, without a file filter. Any row whose
- Apply in memory:
applyChangeSet(schema, prefetchedStore, cs)(03 §4) →ApplyResult(applied form, dropped, candidates, cascaded-to-compute). - Cascade (§3.3). For each deleted row, query the descendants the op did not list, recursively through
RevSchemaparents. Cells go through(sheetId, rowId)/(sheetId, columnId). Append them to the applied form as full-row deletes and tocascaded. - Attribute the conflict candidates (03 §4.5): read the needed
rev_oppayloads (≤ 50 ops per field, ≤ 500 fields), keep the fields last written by another user, and attachbyUser/byRev/theirs. - Creation title. If the op inserts the root row, set its title column to the drive item's current name (04 §5) and record the change in the applied form.
- Authorize.
statsAfter = binding.updateStats(...), thenbinding.authorize(tx, cs, applied, author, statsAfter, models): limits, special-table roles and, for Sheets, protected ranges (§3.5). A failure islimitExceeded,forbiddenForRoleorprotectedRange, and the transaction rolls back. - Normalize:
binding.normalize(...)(always empty now; 03 §6). - Write (§3.3). Deletes children first, then upserts parents first. Every written row gets
"revision" = head + 1. - Log. Insert
rev_opwithrevision = head+1, the applied payload (withbefore, gzip),appliedEqualsSubmitted, the summary, the detail and the outcome.recordIdand the group fields are copied fromcs/arguments. - Head.
UPDATE rev_file SET "headRevision"=@r, stats=@s, "contentCount"=@c, "schemaHash"=@h, "updatedAt"=now() WHERE id=@f(named columns only; CQ-4). - Commit.
- Lock the file.
-
Still inside
state.lock, after the commit.- Set
state.revision = rand add the op to the ring. RevBus.postOp(view, originSessionId: sessionId)withimmediate: true(§5).- Posting before releasing the lock keeps bus messages in revision order per instance.
- Set
-
After the lock.
- Return
RevCommit; the connection sendsRevAckwithappliedwhen!appliedEqualsSubmitted. - Then
RevHooks.kick(fileId)(debounced, §6).
- Return
Errors. RevRejection becomes RevRejected(code). Any other exception is logged with the op
id and becomes RevRejected(internal). Nothing was committed (one transaction), so the client
retries with the same clientOpId.
Budget (measured in R0 and R6): commit p95 < 15 ms for ≤ 100 changes and < 300 ms for 5,000 changes. Per op the statements are:
- 1 head lock;
- 1–3 prefetches per touched table;
- 0–n cascade queries (only when deleting parents);
- 0–1 attribution reads (only with candidates);
- 1–2 writes per table;
- 2 for the op insert and head update.
All statements are generated from RevSchema and run with unsafeQuery/unsafeExecute with
named, explicitly cast or typed parameters (@ids::uuid[], binary arrays @c:_text; inventory
write-infra §6).
-- Prefetch: every table the op touches in one statement, each as one row of typed arrays
-- (UUIDs as text; a column without values as null); by id only, so foreign ids are detected
SELECT * FROM
(SELECT array_agg(t."fileId"), array_agg(t."revision"), array_agg(t.id::text),
CASE WHEN count(t.content) = 0 THEN NULL ELSE array_agg(t.content) END, …
FROM (SELECT * FROM "cell" WHERE id = ANY(@ids0::uuid[]) FOR UPDATE) t) c0,
(SELECT … FROM (SELECT * FROM "sheet_row" WHERE id = ANY(@ids1::uuid[]) FOR UPDATE) t) c1;
-- Cascaded cells of deleted rows / columns that the op didn't list (sheet-scoped, uses cell_row_idx / cell_col_idx)
SELECT id, <all columns> FROM "cell"
WHERE "sheetId" = @s AND "rowId" = ANY(@deletedRows::uuid[]) AND NOT (id = ANY(@explicit::uuid[]));
-- other child tables: WHERE "<parentFk>" = ANY(@deletedParents::uuid[]) (FK columns; small tables)
-- Deletes (children first)
DELETE FROM "cell" WHERE "fileId" = @f AND id = ANY(@ids::uuid[]);
-- Rows that did not exist at the prefetch: plain inserts of the final images (parents first),
-- one binary array per column; a column null in every row is a typed NULL
INSERT INTO "cell" AS t (id, "fileId", "revision", "sheetId", …, content, "note", …)
SELECT u.id, @f, @rev, u.c0, …, u.c3, NULL::text, …
FROM unnest(CAST(@ids:_text AS uuid[]), CAST(@c0:_text AS uuid[]), …, @c3:_text, …) AS u(id, c0, …, c3, …);
-- Rows read (and locked) by the prefetch: updated in place, only the columns the op changed
UPDATE "cell" AS t SET "revision" = @rev, content = u.c3
FROM unnest(CAST(@ids:_text AS uuid[]), @c3:_text) AS u(id, c3)
WHERE t.id = u.id AND t."fileId" = @f;
-- the affected-row count must equal the number of rows; otherwise roll back with invalidChange- Full row images are safe. The file lock serializes every writer of the file's rows, and nothing else writes them after the migration.
- Runs. A run is a sequence of consecutive changes to one table; inserts and updates go in runs of ≤ 5,000 rows, prefetches of ≤ 10,000 ids per table. Binary arrays (
unnest) replacedjsonb_to_recordsetand in-placeUPDATEs replacedINSERT … ON CONFLICTfor existing rows (an update that misses rows, removed by a database cascade earlier in the op, is written again as the upsert) (R6 follow-up: 5,000 updated cells 108 → 55–73 ms, their prefetch 65 → 35 ms). - HOT updates. The cell table has
fillfactor = 85(RevStorage, ensured at server start because models can't declare it), so rewritten cells mostly stay on their page and skip the three index inserts. - File creation (§8.3) answers the prefetch from memory: every row of the new file was written by the creation's earlier ops (
_CreatedRows), and its ops build no view (nobody is connected). - One encoding per op. The applied form is encoded once for
rev_op.payload; an op of inserts only reuses that text and gzip for its view. - Cell deletes rely on the explicit cascade.
cell.rowId/columnIdhave no DB FK (02 §2.1), so deleting asheet_rownever triggers a scan ofcell.
- Per-file throughput is bounded by commit latency. A typing user produces at most one op per round trip (06 §4.2).
- Server changes release the file lock between chunks (§8.1), so a 100k-cell import never blocks typing collaborators for more than one chunk.
Protected sheets and ranges (Sheets 07 §10) were enforced only by the client's write gate. The sequencer now enforces them with the same core code:
ProtectionGuard(wonky_sheets_core/lib/src/data_tools/protection/protection_guard.dart);WriteIntents, moved fromwonky_sheets/lib/features/protection/write_intents.dartintowonky_sheets_core/lib/src/data_tools/protection/write_intents.dart, so the client gate, MCP and the sequencer share it.
When it runs. In SheetsRevBinding.authorize (§3.2 step 9), for every op except
actorKind: system, which covers server-internal writes such as the title mirror and script
builds. It runs only if:
- the file has protections (
stats.tables.protected_range > 0), or - the op inserts, updates or deletes
protected_rangerows.
Otherwise it costs nothing.
What it checks. It runs against models.atPrevious(), the headless model at the head before
this op (§9; ensureAtLeast(head) under the file lock), with the author (authUuid, isOwner = role == owner):
- Derive intents from the applied form, so dropped changes don't count:
WriteIntents.fromApplied(applied, model, deletedIds).-
Cells, row/column default formats, sheet properties, rules, filters, objects and pivots work as today's
WriteIntents.fromChanges. -
Row/column structure is derived from changes, where the client derives it from the operation:
- an inserted
sheet_row/sheet_column→ the "insert between" check with its neighbours by sortKey in the model; - a deleted one →
Structural(delete)at its index; - a
sortKeyupdate →Structural(move); - height, width, hidden and outline groups → no intent (Sheets parity).
- an inserted
-
Boundary repairs are exempt, exactly the changes the client's operation-based derivation ignores:
- an update of a rule table (merge, CF, DV, banding, protection, filter, pivot) that touches only its range columns;
- an update of a cell's
contentfrom a stored formula to a stored formula whose text is identical once reference tokens are removed.
Both qualify only if every reference endpoint that differs between
beforeandafternamed a row or column deleted by this same op.BoundaryRepair.isRepair(change, deletedIds)(core) verifies this. A client therefore can't use the exemption to rewrite protected content, only to repoint references at rows it is legitimately deleting. -
Cells in rows or columns inserted by this op have no index in the model and are skipped; the insert check covers them.
-
- Protection rows themselves. Creating a protection (an insert into
protected_range) is allowed to any editor (Sheets parity). Updating or deleting one needsProtectionGuard.canManage(before, author); otherwise the op is rejected. - Check each intent:
ProtectionGuard(model).check(intent, authUuid, isOwner). Anyblockedresult rejects the whole op withRevRejected(protectedRange).warningsare ignored: warning-only ranges never block on the server, because the client asked for confirmation. - Rejection details:
{"message": GuardResult.blockedMessage, "protections": [{"id", "description", "sheet", "a1"}]}, carried inRevRejected.detailsJson(04 §2.2).
Keeping the model warm.
- A connection to a file with protections takes a
HeadlessFileCachelease at start and holds it while it is open, so the first op doesn't pay the model load inside the lock. - Server changes already hold a lease while planning.
- If budget eviction drops a model anyway, the next check reloads it under the lock (correct, just slow); this is counted in a metric.
- Gate P12 (09 §6): commit p95 < 30 ms on a warm protected 200k-cell file.
Server changes (§8): MCP, AI, scripts, imports into existing sheets, revert/reapply, version restore.
- They are checked the same way, with the attributed actor. Script runs use the run-as user.
SheetsWrites.runkeeps its early guard, which gives friendly MCP errors and warning-onlywarningsin tool results before planning. The sequencer is authoritative: aprotectedRangerejection maps to MCP errorprotected_range.- Version restore pre-checks the whole diff with the guard before writing its first chunk, and refuses with "This version changes protected ranges you can't edit". So it never commits partially because of protection.
Offline edits. Edits made offline before a protection arrived, or by an editor removed from a protection, are now rejected when they sync. The client shows the blocked message and offers Save my changes as a copy for the rejected group (06 §4.6). This replaces the former advisory handling (Sheets 10 §9 "Rejected protected-range edits").
This is lifted from Write's DocumentConnection (inventory write-infra §1.2) with these rules:
| Aspect | Rule |
|---|---|
| Open | static Stream<RevServerMessage> open(Session, RevBinding, UuidValue fileUuid, int knownRevision, String tabId, Stream<RevClientMessage> input): a lazy StreamController that starts on listen and closes on cancel |
| Start | (1) Find rev_file by uuid (missing or purging → notFound) and its DriveItem; AccessService.requireRole(viewer); DriveUserStates.recordOpen. (2) lease = RevFileStates.retain(state); ensureAtLeast(head); for Sheets files with protections and an Editor role, also take a HeadlessFileCache lease (§3.5). (3) Register the bus listener for rev:<fileUuid>, then post hello (roster). (4) Send RevInit. (5) Start the input subscription. After every await, check _closed and undo whatever is already registered (CQ-5) |
RevInit |
ops per 04 §2.2 (knownRevision ≥ prunedBelow, ≤ opsInInit, cut at 1 MiB with more). Otherwise bootstrap: true. knownRevision > head → RevRejected(unknownBase) and close. acceptedHashes, stats, prunedBelow included |
| Effective role | role = min(ACL role, editor-limit role). It is passed into every submit as connectionRole, so a joiner over the limit can't write (review 7) |
| Messages | A serial queue per connection. Each message is handled in its own try/catch (Object), so the queue never dies. A failed RevSubmit replies RevRejected(code) |
RevRequestOps |
Clamped by count and bytes (04 §1) and to what the role may read (§11). Answered with RevRemoteOps |
Bus op |
Forward as RevRemoteOp to every connection except the one whose sessionId committed it (originSessionId; Write's rule). That connection receives RevAck from its own submit |
Bus role |
Re-check the role (AccessService.effectiveRole). No access → RevRejected(forbidden) and close; otherwise RevRoleChanged. Triggered from acl_events.dart, whose spreadsheet/presentation branch posts RevBus.rolesChanged instead of calling SpaceService.mirrorMembership |
| Expiry | The role is also re-checked every 5 min, which closes streams of grants that expired without an ACL event |
| Editor limit | maxEditorsPerFile (100) with Write's roster (hello/here/beat/left, settle 2 s), keyed by connection sessionId |
| Ping | RevPing → RevPong(t, headRevision: state.revision) |
| Close | Idempotent: cancel timers, remove the bus listener, cancel input, release the lease, post left, close output |
| Aspect | Rule |
|---|---|
| Channel | rev:<fileUuid> |
| Message | RevBusMessage (serverOnly: true). Fields: type (op · role · hello · here · beat · left · purged), revision?, opJson? (the RevOpView JSON when its payload is ≤ 64 KiB, else null), originSessionId?, userId?, joinedAt?, headRevision? (on beat) |
| Large ops | An instance whose ring lacks an op fetches it once from rev_op (by revision) and adds it to its ring. Connections on the same instance share it |
ClusterBus.post(…, immediate: true) (new) |
Delivers locally at once, as today. For Redis, it first flushes the outbox's queued messages, so an immediate message never overtakes older queued ones, then publishes. Used for op, role and purged. Write's DocumentBus.post for op switches to it in the same PR (R2) |
| Lost messages | Redis publish is best effort (cluster_bus.dart:156-162). beat messages (every 30 s, only when distributed) and every RevPong carry headRevision. A client or instance that sees a head beyond its revision gap-fills from rev_op |
| Ordering | Not guaranteed across instances. Clients order by revision and fill gaps (06 §4.3), as in Write |
abstract class RevHook {
String get id; // stable; key in rev_hook_state.cursorsJson
bool wants(RevOpView op); // cheap filter (e.g. title hook only if the op touched the root title)
Future<void> run(RevHookContext ctx, List<RevOpRecord> ops); // revision order, ≤ 200
Duration get minInterval => Duration.zero; // throttle (10 s for search text)
}RevHooks.kick(fileId) is debounced per file per instance (hookDebounceMs, 1 s). A drain
works as follows:
- Lease.
UPDATE rev_hook_state SET "leaseOwner"=@me, "leaseUntil"=now()+@lease WHERE "fileId"=@f AND ("leaseUntil" IS NULL OR "leaseUntil" < now() OR "leaseOwner"=@me) RETURNING "cursorsJson". No row returned means another instance is draining, so stop. The lease is renewed while draining (hookLeaseSeconds, 60 s) and cleared at the end, which makes the exclusion real (review 6). - Read the ops with
revision > min(cursors)(≤ 200) from the ring orrev_op, with metadata, summary, detail and the applied payload. - Run each hook in binding order on the ops past its own cursor that it
wants. Hooks with no wanted ops just advance their cursor. - Save the cursors: one
UPDATE rev_hook_state SET "cursorsJson"=@c, attempts=0, "lastError"=NULL. Hooks whose own DB writes must be atomic with their cursor do both in one transaction (ctx.transaction); the cursor JSON is updated in it. - Failure. Catch
Object. Keep that hook's cursor, storeattempts/lastError/nextAttemptAt(backoff 2 s × 2^n up tohookRetryMaxSeconds) and log at error level. The other hooks continue. This is at-least-once, never silently skipped. - Throttling. A hook with
minIntervalthat ran less than the interval ago keeps its cursor, and a timer re-kicks it so it gets a trailing run.
The rev_hook_state row is created with the file, with all cursors at 0. A hook added later
starts at the current head (no backfill). The sweep job (§10) kicks files whose cursors lag or
whose nextAttemptAt is due.
These replace ChangeLogConsumer, DeckChangeLogProcessor with its plugins, and scattered
writes.
| Hook id | Binding | Does | Today |
|---|---|---|---|
activity |
both | ActivityService.logEdit(item, actor, counters) aggregated per actor per batch (Sheets: changes, cells; Slides: changes, slides, slidesDeleted, themes, slideIds) |
consumer duty 2 / plugin activity |
versions |
both | VersionService.recordEdit(item, profileId, revision: op.revision) once per author per batch |
duty 3 / plugin drive |
driveEdit |
both | DriveService.recordEdit(userId, item) once per author per batch |
duty 4 / plugin drive |
title |
both | When the batch wrote the root row's title (seen in the applied form): set DriveItem.name and call DriveEvents.notify. Skips ops whose title came from the drive name itself (the creation-title rule and system rename ops) |
duty 5 (ran on every row today) / plugin drive |
modified |
both | DriveService.touchModified (modifiedAt, modifiedById, search text). Search text uses targeted reads: Sheets reads sheet names plus ≤ 10,000 s:/n: cells, sheet by sheet; Slides reads titles, element text, cells and notes. minInterval 10 s |
duty 6 / plugin drive |
cellHistory |
Sheets | One multi-row INSERT … ON CONFLICT DO NOTHING of the op's detailJson cells into cell_history (revision, author, actorKind, label, at, before, after), then one trim statement (100 per cell) |
duty 7 (2 statements per cell) |
comments |
both | CommentService.orphan/revive from summary.deleted/revived (Sheets rows/cols/sheets; Slides slides/elements) |
duty 8 / plugin comments |
scripts |
Sheets | Compile when kinds ∋ scripts (Editor+); ScriptTriggerService.fireForChange (onEdit/onChange), skipping system/script actors; offline = op.offline |
duty 10 |
contentSize |
both | ContentSizes.onSpaceChanged (renamed onFileChanged). Estimators sum the plain tables by fileId, cells sheet by sheet |
per-batch duty / plugin storage |
Inline, not durable (best effort after the ack): agent cursors (AgentCursors.show for
mcp/ai ops, 5 s TTL presence).
Removed: processSpace, afterRegistration, receipts, pruneOld, SheetsMergedSignal.
- Structure pages walk the non-cell tables in rank order. Their cursor is
{t: tableIdx, after: id}(04 §3), and each page is read in one repeatable-read transaction (one statement per table), so it is one snapshot. A page stops atstructurePageRows(20,000) rows or 4 MiB compressed, whichever comes first. - Tables with more than 50,000 rows for the file (per
stats.tables) are skipped by structure pages and listed inexcludedTables. They are served asrowspages (keyset by id, the same caps); the client may run several in parallel.
-- per table of the page (one repeatable-read transaction = one snapshot):
WITH s AS (SELECT * FROM "sheet_row" WHERE "fileId" = @f AND id > @a::uuid ORDER BY id LIMIT @n + 1),
t AS (SELECT * FROM s ORDER BY id LIMIT @n)
SELECT (SELECT count(*) FROM s) > @n, -- more rows follow
(SELECT id::text FROM t ORDER BY id DESC LIMIT 1), -- the cursor
array_agg(uuid_send(t.id)), -- ids as 16-byte values
CASE WHEN count(t."sheetId") = 0 THEN NULL ELSE array_agg(uuid_send(t."sheetId")) END,
CASE WHEN count(t."sortKey") = 0 THEN NULL ELSE array_agg(t."sortKey") END, …
FROM t;Each table comes back as one row of typed arrays (UUIDs as uuid_send bytes, a column
without values as null), which go into the version-2 body (04 §3.1) through RevPageBuilder
without row maps or JSON; fileId and revision are excluded. A client without format gets
the same rows as version 1. Coverage: one tbl:<table>:<from>:<to> key per table touched by
the page.
SELECT f."headRevision", array_agg(uuid_send(t.id)) FILTER (WHERE t.id IS NOT NULL),
CASE WHEN count(t.content) = 0 THEN NULL ELSE array_agg(t.content) END, …
FROM rev_file f JOIN "sheet" s ON s."fileId" = f.id
LEFT JOIN "cell" t ON t."sheetId" = s.id AND t."rowId" = ANY(@rows::uuid[])
WHERE f.id = @f AND s.id = @s::uuid GROUP BY f."headRevision"; -- no row: not a sheet of @fOne statement checks the sheet, reads the head and the cells. At most pageRows (1,000) sheet
rows per request, and ≤ 4 MiB. Coverage: row:<s>:<rowId> for every requested row. A
13,000-cell page takes ~50 ms on the server (R6: ~190 ms with row_to_json and NDJSON).
- One snapshot per page. Each page is read in one repeatable-read transaction (a cell page in a single statement) and therefore one MVCC snapshot. An op writes its rows and
headRevisionin one transaction, so the snapshot holds exactly the ops ≤revfor every row. - Convergence. Together with the client rule (06 §5.3: apply buffered ops >
Rpto rows covered by the page; keep ops for uncovered rows), the client converges to the head. Property test: 09 §2.5.
static Future<RevServerChangeResult> submitServerChange(Session session, {
required RevBinding binding,
required Actor actor, // user the change is attributed to
required UuidValue fileUuid,
required String actorKind, // mcp | ai | script | system | user (restore)
required String label,
List<RowChange>? changes, // precomputed, or:
Future<List<RowChange>> Function(Object model, int revision)? plan, // planned on the headless model
String? summaryJson, String? detailJson,
UuidValue? recordId, // join an existing record (assistant run); null + record:true → new record
String? onBehalfOfSessionId, String? sessionId,
bool record = false, // revertible (ServerChangeRecord)
Duration lifetime = const Duration(hours: 24),
bool insertsAreRevivals = false,
void Function(int done, int total)? onProgress,
});
// → RevServerChangeResult {changeId?, recordId?, groupId, fromRevision, toRevision, outcome}- Role. Editor+ for the actor, which replaces
SpaceService.ensureWriter.systemchanges (the title mirror, script builds) skip the user check but still require the file to exist and not be trashed or purging. - Planning against a current model. With
plan:- Take a headless lease (§9).
- Under
state.lock, callheadless.ensureAtLeast(state.revision), so the model includes every committed op. - Run the planner once, on the model at revision r.
- Submit the result with
baseRevision = r. - Concurrent client ops resolve by the normal rules (RD-2), as today with LWW.
- Chunking. More than 5,000 changes, or more than
targetOpBytesencoded, are submitted as one group (groupId). The lock is taken and released per chunk, with progress per chunk. - Atomicity. Each op is atomic; a group is not. If chunk k fails, the earlier chunks stay committed. The record (when
record) covers what was committed, so the partial change can be reverted from the UI. This fixes CQ-36. - Signals. Every server change goes through the bus and the hooks like any op. This fixes the writers that never signal today (inventory server §10.1).
- Record:
{changeId, fileUuid, recordId, label, actorKind, userId, cellsAffected, expiresAt, revertedAt?, revertChangeId?}. The ops are those ofrev_opwith thatrecordId. revert(changeId):- Decode the ops' applied forms. With
beforevalues, merged inserts are updates (03 §4.7), so reverting never deletes another user's columns. - Invert them in reverse revision order with
UndoInverter.invertChanges(changes, current: headless model), using the per-column skip rule. Skipped columns becomeRevertResult.skipped, which drives the partial toast as today. - Submit the result with
record: true(a new record). - Set
revertedAt.
- Decode the ops' applied forms. With
reapplyis the same, using the forward changes.- Rule (aligned for MCP and the endpoints; code-quality slt #11): Editor+, and not already reverted.
static Future<RevFile> createFile(Session session, {
required RevBinding binding, required DriveItem item, required Actor actor,
List<RowChange> initialChanges = const [], // empty: initialize:false (client writes the content)
String label = 'Create', String actorKind = 'system',
void Function(int done, int total)? onProgress,
});- Transaction A (the drive item's creation transaction): insert
rev_file(revision 0,initPending = initialChanges.isNotEmpty) and itsrev_hook_staterow. Commit. - Submit
initialChangeswithsubmitServerChange(group). The chunks'clientOpIds are deterministic,UUIDv5(fileUuid, "init:<chunkIndex>"), so a retry is idempotent. - Transaction B:
initPending = false.
A crash between the steps leaves initPending = true. The create* retry (same fileUuid) and
the revlog.initSweep job (§10) resume step 2. Clients that open a file with initPending
connect normally; the initial ops arrive as remote ops.
| Today (inventory server §3b, §3c, §5e) | Becomes |
|---|---|
DriveService.createSpreadsheet → offlineSync.spaces.createFor + SpreadsheetFactory.write (direct ORM inserts, no change log) |
SpreadsheetFactory.changes(...) → createFile |
createSpreadsheetFromSheets (MCP) |
same |
SpaceCopier.copy (templates, Make a copy) |
SpaceCopier.changes(sourceRows, …) (id remap unchanged) → createFile |
SheetsImportService.importNew / import jobs |
createFile with no rows, then chunked submitServerChange(record: true) |
SlidesDecks.create, DeckFactory.write, SlidesTemplateDecks.write, DeckCopies.copy |
…changes() → createFile |
create*(initialize: false) (client-written content) |
createFile(initialChanges: []); the client's first op is revision 1 |
Welcome content (AccountService) |
createFile |
DriveService.rename of a spreadsheet/deck |
submitServerChange(update root title, system) if the root row exists; otherwise only the drive item is renamed, and the creation op picks up the name (04 §5) |
This replaces HeadlessWorkbookProvider and HeadlessDeckCache (CQ-52).
| Aspect | Rule |
|---|---|
| Load | One REPEATABLE READ transaction: read headRevision, then every table by fileId (keyset, 10,000 rows per query) and the perSheetTables sheet by sheet; binding.buildModel |
| Fresh | Subscribed to rev:<fileUuid>. Applied forms from the bus (or from rev_op when large) go through binding.applyToModel in revision order, with gaps filled from rev_op. ensureAtLeast(revision) blocks until applied. No age(xmin) scans, no staleness re-check, no markStale/writeThrough |
| Leases | open() returns a HeadlessLease (model, revision, release()). Eviction disposes a model only when its lease count is 0 (server-domains S2) |
| Budget | One budget for both apps: revlog.headlessBudgetMb (2,048) and revlog.headlessMaxEntries (100), using binding.estimateBytes, re-estimated every 1,000 applied ops |
| Planning lock | RevFileState.lock (§8.1). No separate lock with a timeout that breaks exclusion (server-domains S1) |
| Shadows (AI Suggest) | Copy-on-write overlays over the cached model, counted in the budget (CQ-35) |
Call sites (inventory server §4a/§4b) switch to HeadlessFileCache.sheets/slides.open(session, fileUuid) and must release(). withWriteLock callers become submitServerChange(plan: …).
| Job | Schedule | Does |
|---|---|---|
revlog.pruneOps |
daily | In batches of 10,000 under a 60 s budget, looping until done: (1) for ops older than opRetentionDays that no unexpired ServerChangeRecord references, set payload/summaryJson/detailJson/outcomeJson to null, and advance rev_file.prunedBelow to the first revision still holding a payload; (2) delete slim rows older than opTombstoneDays |
revlog.hookSweep |
every 5 min | Kicks files whose hook cursors lag the head or whose nextAttemptAt is due |
revlog.initSweep |
every 5 min | Resumes createFile for initPending files older than 5 min |
revlog.repairStats |
weekly | Recomputes stats/contentCount for files changed since the last run (sheet by sheet for cells) |
| removed | — | sheets.pruneChangeLog, slides.pruneChangeLog, sheets.evictSpaces |
| changed | — | revlog.purgeServerChanges (ServerChangeRecords.purgeExpired, records of both apps); drive.purgeTrash (§10.1); storage reconcile (estimators by fileId) |
rev_file.item is onDelete=Restrict, so a drive item can't be deleted while its rev_file
exists. DriveTrash.purge calls RevLogService.purgeFile for spreadsheet/presentation nodes
before deleting the items:
UPDATE rev_file SET purging = true. New connects and submits now getnotFound/fileReadOnly.- Post
purgedon the bus. Every instance closes the file's connections withnotFound, evictsRevFileStatesandHeadlessFileCacheentries, and stops hook drains for the file. - Wait for the hook lease to expire or be released.
- Delete in batches: Sheets cells sheet by sheet (50,000 rows per statement), then the other domain tables by
fileId. - Delete
rev_file, which cascadesrev_opandrev_hook_state. - Delete the drive item (the existing purge continues).
Account deletion uses the same path (together with the ownership fix CQ-1).
| Action | Minimum role |
|---|---|
connect, page, info |
Viewer |
ops(from, to) / RevRequestOps |
Viewer for ranges within the last 1,000 revisions at call time; Editor+ for older ranges (CQ-17) |
RevSubmit |
Editor, as the effective role including the editor limit (§4) |
submitServerChange |
Editor for the attributed actor; system exempt (§8.1) |
cellHistory, version history |
Editor (CQ-17) |
| Writing protected cells, structure, rules, objects or sheets | Owner, the protection's creator or its editors (ProtectionGuard.canEdit); warning-only protections: any editor (§3.5) |
Trashed or purging files are read-only. An item that is removed or unshared closes open streams
with forbidden.
- Per op: commit duration, change count, payload bytes, prefetch rows, cascade rows, attribution reads, protection check duration and cold model loads, lock wait. Logged at debug; ops slower than 200 ms at warning (
session.log). - Per connection: open duration, ops sent, gaps filled.
- Settings › Developer (suite 04 §4): ops per minute, p95 commit, hook lag (head minus the minimum cursor) per binding.
Packages:
packages/wonky_revlog: pure Dart. Codec, apply rules, overlay, chunker and composer, and the sync state machineRevSyncCore, which has no I/O.packages/wonky_revlog_flutter: depends onwonky_revlog,wonky_appandwonky_client. Store, transport, I/O wrapper, bootstrap and per-user client.
App bindings: wonky_sheets/lib/features/revlog/ and wonky_slides/lib/features/revlog/. They
replace wonky_sheets/lib/features/sync/, wonky_slides/lib/features/sync/ and
packages/wonky_sync_kit_flutter, which is deleted in R8.
| Class | Package | Role | Replaces |
|---|---|---|---|
RevSyncCore |
wonky_revlog |
Per-file state machine (§4): outbox order, in-flight op, buffer composition, revision queue, gap detection, ack handling, rejection handling. Fed by events; emits commands | engine loop, SpaceWriteQueue, JournalIngestion |
RevConnection |
wonky_revlog_flutter |
Wraps RevSyncCore with a RevTransport, RevLocalStore and ConnectionKeeper |
— |
RevLocalStore |
wonky_revlog_flutter |
SQLite cache per user and app (02 §3) | SyncReplica, SyncWriter, ChangeJournal |
RevTransport |
wonky_revlog_flutter |
Interface over client.sheetsDoc / client.slidesDoc (connect, page, ops, info). FakeRevTransport (in package:wonky_revlog/testing.dart) implements it over FakeRevServer |
SyncRemote |
RevBootstrap |
wonky_revlog_flutter |
Page scheduler and coverage (§5) | SpaceOpening, row streaming in SyncedWorkbookSession |
RevLogClient |
wonky_revlog_flutter |
Per user and app: store, open sessions, background sync of outboxes, availability, Settings provider | AppSyncController, ReplicaInstallation, ReplicaOfflineProvider |
ConnectionKeeper |
wonky_app (moved from wonky_write/lib/features/sync/connection_keeper.dart) |
Reconnect, backoff, connectivity, Simulate offline | — |
LoggedWorkbookSession extends WorkbookSessionBase, not just implements WorkbookSession:
four features downcast to the base for customFunctions, externalData and customLists
(inventory sheets-client §5).
| Member | Contract |
|---|---|
execute / applyChanges |
As today: plan, apply to the model synchronously, persist. Persist = append the changes to the outbox (durable), tagged with the plan's groupId, and hand them to the connection. The LoadingGate checks stay (checkPlan; §5.5 adds the delete gates) |
flush() |
Unchanged semantics: everything applied so far is durable locally, i.e. in the outbox |
whenSynced({Duration? timeout}) (new) |
Completes when this tab's outbox for the file is empty, i.e. every change is acknowledged. Fails with RevOfflineException when offline. Used by script compile (replacing SheetsData.current().syncNow()) and by conflicts |
remoteBatches |
Remote ops and streamed page batches are emitted synchronously, in the same turn as model.applyChanges. Commit hints identify local batches by identity before a Timer.run (inventory sheets-client §7) |
syncStatus |
§9 |
role |
From RevInit, RevRoleChanged and drive item changes (updateRole) |
pendingChanges |
Number of this tab's outbox changes for the file |
history (new) |
RevHistory: recent op metadata from rev_local_op (30 days), plus a broadcast stream of RevOpMeta {revision, authorUuid, sessionId, actorKind, label, groupId, groupLast, recordId, summary, detail, offline, committedAt} for every op this session applies: remote ops, and its own acked ops (summary and detail from the outbox). It replaces the local ChangeLog reads (§7) |
outcomes (new) |
Stream of RevOutcomes (dropped, conflicts, cascaded) of this session's acked ops (§6) |
groupRejected (new) |
Stream of groupIds whose ops were rejected (§4.6); the editor drops the matching undo step |
rowLoading |
As today (RowLoadTracker), fed by RevBootstrap |
LoggedDeckSession extends DeckSessionBase (uses DeckWritePath) gets the same additions. It
runs the 12 DeckMaintenance repairs as SyncedDeckSession does: 1 s after remote batches,
editors only, not undoable, actorKind: user with the summary flag "repair": true (03 §6; 08 §4). Additionally, repairs (Sheets and Slides)
run only when no remote group is half-applied (the last applied remote op had
groupLast == true) and this tab's outbox is empty (review 21).
static Future<RevLocalStore> open({required RevSchema schema, required String userUuid,
required String app /* 'sheets' | 'slides' */, String? directory});
// file: '<app>_cache_<userUuid>.db', via LocalDatabases.open(…, migrations: revCacheMigrations(schema))- PRAGMAs as today's replica (
sync_replica.dart:63-76): 64 MB page cache, in-memory temp store, WAL autocheckpoint 20,000. On the web they are set directly on the single worker connection:sqlite_async's webwithAllConnectionsholds the write lock but passes the unlocked database to its block, so a statement inside it deadlocks and the cache never opens ("storage on this device didn't start"). - Old engine files. On first start of the new build, remove
sheets_<userUuid>.db/slides_<userUuid>.dbwithout reading them. On desktop,LocalDatabases.delete. On the web, whereLocalDatabases.canDeleteis false (local_database.dart:57), open them and drop every table (one transaction), then recordrev_meta.oldReplicaClearedso it runs once.
Every write is one writeTransaction:
| Kind | Writes |
|---|---|
appendOutbox(fileUuid, ChangeSet, tabId) |
one rev_outbox row (inflight = 0, schema_hash) |
markInflight(clientOpId) |
inflight = 1 |
applyServerOp(fileUuid, revision, applied, meta, {ackedClientOpId}) |
Only if revision > rev_local_file.revision (monotonic, RD-22): replay the applied form literally on the domain rows (upsert full rows for inserts; set columns for updates; delete for deletes), set revision = r, insert the rev_local_op row, and delete the outbox row ackedClientOpId when given. A committed op therefore can't be both applied and in the outbox |
applyPage(fileUuid, RevPage, replays) |
For each coverage key whose stored revision is lower than Rp: replace the covered rows (§5.3), apply the replayed buffered changes, upsert the coverage key. Set complete = 1 with the last page |
replaceOutboxRows(fileUuid, tabId, List<ChangeSet>) |
after composing buffered chunks (§4.2) or re-stamping (§2.4) |
Durability before sending. The session sends an op only after the write that contains it has
completed (whenDurable, as Write).
-
loadFile(fileUuid)returns therev_local_filerow plus this tab's outbox and orphaned outbox rows (§2.6). -
loadRows(fileUuid, {skip: {cell}})reads keyset pages of 10,000, as today'sloadSpace. -
loadCells(fileUuid, sheetId, rowIds)reads 500 row ids per query, asSheetsReplica.loadCells. As built:loadScoped(fileUuid, sheetId, rowIds);loadStructure(fileUuid)andloadScopedAt(…)return the rows with the revision of the stored state they belong to (one read transaction), for the viewport-first reopen (As built 24). -
coverage(fileUuid),recentOps(fileUuid, since),filesWithOutbox()andusage().
rev_meta.schemaHash records the RevSchema.hash the domain tables were created with.
- App hash changed after an update:
- Keep
rev_local_file,rev_outboxandrev_local_op. - Drop and recreate the domain tables and
rev_coverage. - Set every file to
complete = 0andrevision = 0. - Re-encode every outbox row with the new schema and set its
schema_hash. Changes to tables or columns that no longer exist are dropped and reported once in a toast. This keeps a stale hash from blocking the outbox (review 11). - The next open re-bootstraps and overlays the outbox.
- Keep
- Server ahead of the app: the client continues if its hash is in
RevInit.acceptedHashes; decoders skip unknown tables and columns. Otherwise it shows "Update required" (02 §4.3).
- Kept complete: files opened in the last 14 days (max 50 per app) or pinned (Make available offline).
- Never evicted: files with outbox rows.
- Evicted: everything else is deleted from the cache (domain rows by
file_uuid,rev_local_op,rev_coverage,rev_local_file) byRevLogClient.maintain(), after sign-in and then daily. This replaces the weekly purge and the cell-cap eviction. - Without OPFS (
offlineStorageLimited()): files beyond 200,000 cells in total (Sheets) or 50,000 elements (Slides) are not kept offline, least recently opened first; pinned files win. A warning shows as today.
- Tab id. Every app instance has a
tabId. It sends its own outbox rows and only those. - Liveness. Each tab holds a Web Lock
wonky.revlog.tab.<tabId>for its lifetime; on desktop, a process lock file. - Adoption.
RevLogClient.syncOutboxes(§7.3) adopts outbox rows whose tab no longer holds its lock: it rewritestab_idto itself, then sends them. Adoption runs inside a Web Lockwonky.revlog.<app>.<fileUuid>.adopt, so two tabs never adopt the same rows. - Monotonic store. Two tabs with the same file open each run their own connection and model. Store writes are monotonic (§2.2), so the tab that is behind never moves the cache backwards. A tab that sees
rev_local_file.revisionbeyond its own just skips the store write; its model still applies the op.
- The store holds server state at
revision. - The model holds
applyAll(outbox, server state)(03 §7). - Loading: read the rows from the store (viewport-first, §5.1), then apply the outbox ops to the model with the pure rules. The batch has origin
loadfor Slides and is a load batch for Sheets.
onRemoteOp(op): // strictly in revision order (§4.3)
if op.clientOpId ∈ thisTabOutbox: // our own op seen through catch-up (ack lost)
onAck(op.clientOpId, op.revision, applied: op.payload, outcome: op.outcomeJson); return
applied = decode(op.payload) // applied form, no before
patch = Overlay.build(model, serverRows, applied, outbox) // 03 §7
batch = model.applyChanges(patch) // origin remote
remoteBatches.add(batch) // same turn
store.applyServerOp(fileUuid, op.revision, applied, meta(op)) // async, ordered, monotonic
history.add(meta(op))
if bootstrapping: bootstrap.buffer(op) // §5.3
rev_local_op keeps 30 days or 20,000 ops per file, whichever is smaller, pruned on open and
daily. It holds metadata, summary and detail only.
This is ported from Write's DocumentSyncController (inventory write-infra §7.1) without the OT
parts.
submitLocal(changes, {label, actorKind, groupId, summary, detail}):
cs = ChangeSet(clientOpId: v7, baseRevision: serverRevision, changes, …, offline: !connected)
for chunk in RevChunker.split(cs): // ≤ 5,000 changes and ≤ targetOpBytes encoded (one group)
if inflight == null and buffer empty and connected: inflight = chunk
else buffer.addOrCompose(chunk) // §4.2
store.appendOutbox/replaceOutboxRows → whenDurable → sendInflight()
- Only one op is in flight at a time.
sendInflightsendsRevSubmitonly once the write is durable. It sends at most once per connection and resends after a reconnect with the sameclientOpId.
A new change set is composed into the last buffered chunk if all of these hold:
- both belong to the same group, or both are plain edits (no
groupId); - the last chunk started less than 2 s ago;
- the composed result stays within the limits.
Otherwise it starts a new chunk. Composition follows 03 §2: composeRowChanges, merged summary,
capped detail, OR-ed offline. Undo entries hold their own RowChanges and never reference ops,
so composing never changes what undo does.
- Revision-carrying server messages (
RevRemoteOp,RevAck) go into aSplayTreeMap<int, …>queue. They are processed strictly asrevision == serverRevision + 1. - Gap: send
RevRequestOps(serverRevision + 1, firstQueued − 1), re-requested after 3 s. A gap also opens whenRevPong.headRevision > serverRevisionand the queue is empty (lost bus message, 05 §5). - Duplicates (
revision ≤ serverRevision) are ignored.
onAck(clientOpId, revision, applied?, outcome?): // in revision order
op = outbox.take(clientOpId)
appliedForm = applied ?? op.changes // RD-20: never re-derived
store.applyServerOp(fileUuid, revision, appliedForm, meta(op), ackedClientOpId: clientOpId)
if applied != null: // the server's result differs (drops, merges, cascades, creation title)
patch = Overlay.build(model, serverRows, appliedForm, remainingOutbox) // as for a remote op
remoteBatches.add(model.applyChanges(patch))
serverRevision = revision
history.add(meta(op)); if outcome: outcomes.add(outcome)
if buffer not empty: inflight = buffer.removeFirst(); sendInflight()
connect(knownRevision: serverRevision, tabId).- Handle
RevInit:ops: process them in order through the queue, then gap-fill the rest whenmore. An op carrying this tab'sclientOpIdis handled as an ack (§3.2).bootstrap: true: run §5 again from the current model (pages replace covered rows; the outbox overlay stays).
- Resend the in-flight op (same
clientOpId), then the buffer. - Pruned duplicate. If the server answers a resend with
duplicate: trueandapplied == null(payload pruned), the op committed more than 30 days ago. Re-bootstrap the file. - Outbox age. The server never refuses an old base. Outbox rows older than 180 days are not resent; they are offered as a copy (§8).
On invalidChange, tooLarge, limitExceeded and protectedRange for an op of group g:
- Remove that op and every other outbox op of group g (the rest of the plan).
- Rebuild the model rows they touched as
applyAll(remaining outbox, server state), emitted as one remote batch. - Emit
groupRejected(g). The editor's undo service removes the undo step recorded for that plan;RowsUndoEntrygets an optionalgroupId(05 §8.2 analogue on the client). If the step is buried under later steps, it is marked inert: its undo becomes a no-op with the toast "This change was not saved". - Show one toast for the group with the server message, and log the op id and code (error level).
protectedRange.- The toast is the blocked-cell message (
GuardResult.blockedMessage) naming the protections fromdetailsJson. - For ops with
offline: true, the toast adds Save my changes as a copy (§8). The rejected group's changes are kept for 7 days for that purpose. - A rejected repair op (
"repair": true) shows nothing and is not retried for that violation for 24 h.
- The toast is the blocked-cell message (
The client write gate is unchanged. SheetEditorController.apply → WriteIntents.of →
ProtectionGuard, now imported from wonky_sheets_core. It still shows the blocked dialog and the
warning-only "Heads up!" confirm before anything is sent. Server enforcement only catches what the
client couldn't know: a protection that arrived later, a changed editor list, or a stale or
modified client. Property test: for the same model, client-gate "allowed" implies server "allowed"
(09 §2.8).
Sheets is shown; Slides is the same with whole-deck loading.
open(fileUuid):
local = store.loadFile(fileUuid)
if local?.createdLocally: // queued create not replayed yet — FIRST
open from the outbox (the creation op) → WorkbookReady; connect once the create has replayed
else if local != null and (local.complete or local.revision > 0):
load from the store viewport-first (today's _loadAll: all non-cell tables, frozen rows +
top−20…top+200 of the ViewportHint; files ≤ 2,000 rows fully) → overlay outbox → WorkbookReady
connect(knownRevision: local.revision) // RevInit ops or bootstrap
if !local.complete: continue §5.2 for what is missing
else if online:
connect(0) → RevInit(R0, bootstrap, stats)
if R0 == 0 (created elsewhere, not initialized yet):
WorkbookUnavailable("This spreadsheet is still being created on another device")
— keep the stream open; the first remote ops create it → re-open
else: structure pages → first cell page → WorkbookReady; continue §5.2
else:
WorkbookUnavailable(offline: true) // re-opens when back online (as today)
- Structure. Request
structurepages untilnextCursor == null. In parallel, requestrowspages for every table inexcludedTables, two at a time (largesheet_row/sheet_column). - Viewport. While structure loads, request a
cellspage for the frozen rows plustop−20 … top+200of the active sheet. Apply it after the structure, because the model needs the row ids. - Ready.
WorkbookReadyonce steps 1–2 are applied. Files with ≤ 2,000 rows load all their cells first, as today. - Streaming. The remaining cells come in blocks of 500 rows:
- first the grid's
prioritizerequests (newest of the last 4, ±50 rows); - then the active sheet;
- then the other sheets in tab order.
At most
RevFileSync.maxPagesInFlight(3) cell pages are in flight; they are fetched and decoded concurrently and applied in request order. A block is split over the free slots (≥ 100 rows per page), so even a plan that hands out one block at a time gets parallel pages; rows the plan hands out again while their page is in flight are not requested twice (the loop waits for that page instead). A plan that skips rows already handed out (not yet loaded) keeps every slot busy.
- first the grid's
- Decoding (
RevPageDecoder,wonky_revlog_flutter/lib/src/transport/) runs off the UI thread: on desktop and mobile in a short-lived isolate per page (Isolate.run; the rows come back without a copy), on the web in slices of ~8 ms that yield to the event loop. Bodies under 32 KiB are decoded inline. - Progress.
rowLoading.progress= loaded rows / total rows, with the total taken fromRevInit.stats.sheets.
During bootstrap the client keeps:
serverRevision: the last op processed in order. It starts atR0.- The buffer: every op processed since
R0, as(revision, applied form), in revision order. This includes this tab's own acked ops (their applied form), not only remote ops (review 16). - Coverage: the keys from
rev_coveragewith their page revisions.
A change's coverage key comes from the schema (RevTable.coverage):
- structure tables:
tbl:<table>plus the id, matched against the stored ranges. cell:row:<sheetId>:<rowId>. Inserts carrysheetId/rowId. For updates and deletes, the key is found from the row in the store or in the page being applied; if neither has it, the cell is not covered yet. Cell key columns are immutable (03 §4.3), so the key never changes.
| Event | Action |
|---|---|
Page (Rp, coverage keys C, rows) |
(1) For each key in C not already covered at a revision ≥ Rp: rows present in the page replace the store/model rows; rows of that key absent from the page are deleted. (2) Replay, in order, the buffered changes with revision > Rp whose coverage key is in C. (3) Store C → Rp |
Op r processed |
For each change in its applied form: if its key is covered at Rp < r, apply it now; if covered at Rp ≥ r, skip it (already contained); if not covered, it waits in the buffer. Then append the op to the buffer |
| Reconnect during bootstrap | connect(knownRevision: serverRevision). RevInit.ops are processed as ops, as above. bootstrap: true restarts the remaining page requests; pages already applied stay valid, because their coverage revisions still decide what is newer |
| Bootstrap complete | Drop the buffer; complete = 1 if the file qualifies for offline (§2.5); collapse row: coverage keys into sheet: keys |
The model is updated through the overlay at every step, so outbox changes stay on top. Property test: 09 §2.5.
- Reopen. Pages are always written to the store, so reopening a partly loaded file is instant for the loaded part.
rev_coveragerecords what is present and at which revision. - Reconnect.
RevInit.opsupdate the covered rows normally; withbootstrap: true, missing and stale parts are requested again.
- Kept as today:
LoadingGate(write and range-read refusal over unloaded rows);#LOADINGformulas (LoadingFormulaSource);- the
LoadingLayershimmer; ViewportHint(fromLocalViewStateor?gid=).- The
_touchedprotection now follows from §5.3, because newer ops win over older pages; keep its test.
- Added (RD-3, review 8):
DeleteSheetandDeleteDimension(columns) require the whole sheet to be loaded. The planners list descendants from the model, so unloaded cells would be cascaded instead of listed, and undo couldn't restore them.DeleteDimension(rows) requires every deleted row to be loaded.- The refusal message is today's
still_loading("Still loading this part of the sheet…").
The RevOutcomes of acked ops drive the conflict UI. They replace today's heuristics, which
compare the model after a sync cycle and look up the winner in the local ChangeLog (inventory
sheets-client §6, slides-client §5).
| Outcome | Sheets UI | Slides UI |
|---|---|---|
conflicts (overwritten: last arrival wins, Q1) |
Toast "Your offline changes replaced {n} newer edits by {name} — Review". Dialog: cell, your value, their previous value, author; Restore theirs (undoable, one step) and Dismiss | "You replaced {name}'s newer text in {n} boxes", with Restore theirs |
dropped (target or parent gone) |
Entries "Row deleted by {name}" / "Column…" / "Sheet…" (no apply). The deleter comes from rev_local_op or ops() (the op that deleted the row; byRev when known) |
"Slide deleted by {name}" |
cascaded |
Info toast "Also removed {n} cells added by {name}" (P2; logged in P1) | — |
- When shown. Only for
offline: trueops (the online case is normal collaboration), and for Slides edits made withforcedsoft-lock claims (as today;sum.forcedBodies). - Storage. Kept in
rev_outcomeuntil reviewed or 7 days old. The user-store keyssheets.offlineEdits.*,sheets.conflicts.*,slides.offlineEdits.*andslides.conflicts.*are removed. - Sorts. A
sheet_row.sortKeyconflict on at least half the rows of an offline sort gives the sort toast ("Your sort was replaced").
| Today (inventory sheets-client §4–§5) | With RevLog |
|---|---|
SheetsData.install/uninstall/current/ready (sheets_app_config.dart:125,136) |
SheetsRevLog.install/uninstall/current/ready: same lifecycle (per user, follows currentUser, pauses when hidden) |
openWorkbookSession(services, itemUuid) |
Same signature; fileUuid = item.contentRef ?? itemUuid; role item.myRole; subscribes driveChanges → updateRole |
SheetsData.createSpreadsheet (reserved space + registerSpreadsheet) |
Online: drive.createSpreadsheet(fileUuid: v7, initialize: true, locale, timeZone) (templates go to the server as today), then open. Offline: queue PendingDriveOp(createSpreadsheet {fileUuid, parentUuid, name, initialize: false}), create rev_local_file(created_locally = 1), put the creation op (CreateSpreadsheet plan with SheetsUserDefaults) in the outbox, and open at once (§5.1). NoReservedSpaceException disappears |
Import as new (import_pipeline.dart:369-413, second headless session) |
Create with initialize: false (online: direct call; offline: queued), then openWorkbookSession for the new file, CreateSpreadsheet + chunked applyChanges, flush, dispose. The outbox chunks it into groups |
Version copy (version_copy.dart:26-57, writes into a reserved space) |
SpaceCopier.changes(...) → create with initialize: false (online direct or queued). The changes become the first op group. sheetsVersion.copyExtras unchanged (online only) |
Script compile (script_service.dart:1048-1050: flush() + syncNow()) |
await session.whenSynced(timeout: 10 s), then backend.compile |
Script user properties (user_properties.dart, personal space) |
Online via sheets.get/setUserProperties (02 §2.4), cached per file in the user store (sheets.userProps.<fileUuid>). Offline writes are queued in the user store and flushed on reconnect (last write wins per key) |
External agent highlights (external_agent_writes.dart, local ChangeLog) |
session.history stream filtered to actorKind ∈ {mcp, ai} and sessionId starting with server:; ranges from summary.ranges |
"Row deleted by" toast (sheets_collab.dart:248-275) |
session.history (last hour), summary.deleted |
Cell history (cell_history.dart:310-346) |
Local: session.history.recentOps (30 days), detail.cells. Server: sheets.cellHistory (unchanged API; changeLogId → revision). Deduplicate by revision |
Conflicts (local_conflicts.dart) |
§6. SheetsConflicts keeps its dialog and commands; detection and tracking come from outcomes |
Settings › Offline & storage (ReplicaOfflineProvider) |
RevOfflineProvider implements OfflineDataProvider. isAvailableOffline(item) = complete. usage = files, pinned, pending changes (outbox rows), bytes, warnings. clear() empties the cache after exporting non-empty outboxes (§8) |
| Simulate offline | Unchanged (ConnectivityService.forceOffline); ConnectionKeeper drops the streams |
| Recovery export of a broken replica | Outbox export (§8) |
WorkbookLoading(progress) "Downloading spreadsheet… N%" |
Removed. WorkbookLoading() shows "Opening spreadsheet…". Delete the downloadingSpreadsheet string. Wire loadingRowsStatus to rowLoading.progress (the string exists but is unused) |
createPresentation:- Built-in templates and blank decks are created by the client with
initialize: false(online: direct call; offline: queued), with the template's rows (template.build(...).inserts) as the creation op group. - Non-built-in templates are created on the server (
initialize: true+templateUuid). NoReservedSpaceExceptionand its toast disappear.
- Built-in templates and blank decks are created by the client with
DeckOpenHistoryfeeds availability (§2.5).- Agent flash (
mcp/agent_writes.dart) and the overwritten-text review (local_conflicts.dart) usesession.history/session.outcomes. - AI "wait for the slide to arrive" (
ai_commands.dart:500-516) waits insession.historyfor an op with therecordIdreturned by the AI endpoint (RevServerChangeResult.recordId, added to the responses). That replaces polling the model 40 × 250 ms.
RevLogClient.syncOutboxes() runs on reconnect and on drive changes:
- For every file with outbox rows that no open session of this tab owns, including adopted rows (§2.6), open a headless
RevSyncCoreconnection (no model), keyed'<app>.offline:<fileUuid>', as Write'sWriteOffline.syncOutboxes. RevInit.ops: apply them to the store (applyServerOp).RevInit(bootstrap: true): send the outbox only. Apply acks to the outbox (delete rows) without domain writes, setcomplete = 0, clear the file's coverage, and setrevision = 0. The next open re-bootstraps (review 34).- Stop after the outbox is empty and 5 s of idle, or after 2 minutes.
Files with created_locally = 1 wait until the queued create has replayed
(CachedDriveRepository.isPendingCreate). If the create fails permanently (replay dropped it
as non-retryable: quota, permission), the file is marked orphaned (§8).
| Situation | Behavior |
|---|---|
Access lost (forbidden; or RevRoleChanged(viewer) / forbiddenForRole with a non-empty outbox) |
Read-only. Toast "You can no longer edit this spreadsheet". The outbox is kept for 7 days and offered as Save my changes as a copy, which creates a new file with initialize: false from server state + outbox (as Write's copy). Then it is discarded |
| Orphaned local file (queued create failed permanently) | Toast "This spreadsheet couldn't be created on the server", with Save as new spreadsheet: a new fileUuid, the same changes, queued again |
| Outbox older than 180 days | Not resent; offered as a copy |
schemaMismatch |
Stop sending; "Update required"; after the reload, §2.4 |
unknownBase |
Re-bootstrap; offer the outbox as a copy |
| Cache can't be opened | Read rev_outbox with plain SQLite (desktop) and offer Download unsynced changes (a ZIP with one JSON per file of the change sets). On the web the cache is emptied. Then start with a fresh cache |
| Rejected op | §4.6 |
| Offline edits rejected by a protection | Save my changes as a copy creates a new spreadsheet (initialize: false) from server state plus the rejected group, so nothing typed offline is lost (§4.6 step 5) |
SyncStatus (wonky_ui) states keep their meaning, with "saved" now meaning acknowledged:
| State | Condition |
|---|---|
saving (+ pendingChanges, progress for groups) |
this tab's outbox for the file is not empty, and connected |
saved |
outbox empty: every change is acknowledged by the server. Today "saved" means "in local SQLite" |
offline ("Offline — changes saved on this device") |
not connected; outbox durable |
connecting / reconnecting (+ reconnectAt) |
stream opening or backing off |
error (+ retry) |
a rejection waits for the user, or a store write failed |
- Removed: the
cellandrangehints (RD-18), the hint overlay's wait for synced rows (_dropSynced) andCellRenderCache.hinted. - Kept:
typing(Sheets), and the text, transform and structure hints (Slides), unchanged.pending(the shimmer for groups over 500 cells) stays and is now dropped when the group's last op arrives (groupLast).
The Slides text-hint removal rule (the synced text equals the hint) keeps working, because remote batches still arrive as remote batches.
Measured in R6 on a Windows desktop (debug JIT, embedded Postgres, in-process FakeRevHub for
the clients); details, setup and the fixes in
plans/sheets-sync-migration/baseline.md, the
misses in TODO.md ("RevLog sync — open items"). The client gates (P6–P9) were measured again
in R7 after the client work of As built 24 (same machine, three runs). The server and wire gates (P3, P4, P6, P7,
P10, P11) were measured again in R7 after As built 25.
| # | Metric | Budget | Measured by | Measured (R6) |
|---|---|---|---|---|
| P1 | Local op → visible in another client's model, one instance | p95 < 250 ms | e2e + revlog_perf_test (two connections, 500 single-cell ops) |
server 4.9 ms, client apply 5.6 ms (1M cells) — pass |
| P2 | Same, two instances (Cluster.debugDistributed with Redis) |
p95 < 400 ms | multi-instance perf test | not measured (no Redis) |
| P3 | Server commit, ≤ 100 changes | p95 < 15 ms | revlog_perf_test |
1 change 6.5 ms, 100 changes 13.5 ms → R7 6.7 / 11.4 ms — pass |
| P4 | Server commit, 5,000 changes | p95 < 300 ms | same | 288–306 ms → R7 250 ms (282 ms in a run while other suites ran) — pass |
| P5 | Idle server cost per connected client | no DB queries while idle except the 5-minute role check | query counter in the perf test | 0 — pass |
| P6 | Cold open (not on the device) of a 1M-cell sheet → WorkbookReady |
< 1.5 s desktop, < 3 s web (OPFS) | client perf test against a local server; web via release build | server part 1.1 s → R7 0.45–0.54 s; client 1.44–2.0 s → R7 0.89–1.12 s — pass; decks 0.70 → 0.38 s; web not measured |
| P7 | Complete download of a 1M-cell sheet | < 15 s desktop, < 30 s web | same | server part 31 s → R7 7.1–7.3 s (pages 2.6 s, decode 0.8–0.9 s, model 3.4 s); client 98 s → R7 12.6–14.5 s (2.6–2.9 s of it the in-process page serving) — pass |
| P8 | Reopen from cache (1M cells) → ready | ≤ today (0.55 s for 200k; scale linearly) | client perf test | 28.6–47 s → R7 0.28–0.43 s (viewport-first; every cell in 8.5–10.8 s in the background) — pass; decks 0.29 → 0.22 s |
| P9 | 1M-cell local import (applyChanges into the outbox) |
< 30 s | client perf test | 30–32 s → R7 17.3–18.6 s (planner + model 26 → 13.3–14.2 s, outbox 3.9–4.4 s) — pass |
| P10 | Server import of 1M cells (server change, 200 ops) | < 60 s | revlog_perf_test |
57.8 s → R7 41–42 s — pass; as a new file in one transaction 63–69 s → R7 37.6–38.0 s — pass |
| P11 | Storage per non-empty cell (Postgres, incl. indexes) | ≤ 250 bytes average for typical content | pg_total_relation_size / cells |
210 B → R7 238 B (fillfactor = 85) — pass |
| P12 | Server commit on a warm 200k-cell file with 20 protections, ≤ 100 changes | p95 < 30 ms; no cold model load inside the lock while a connection is open | revlog_perf_test |
5.6 / 14.2 ms, 0 cold loads — pass |
The suites: wonky_server/test/integration/sheets/revlog/revlog_perf_test.dart,
wonky_sheets/test/revlog/revlog_perf_test.dart and wonky_slides/test/revlog/revlog_perf_test.dart
(tag slow). The gates were to be measured in R0 (baseline of the old engine; not recorded) and
R6 (above). A regression > 20 % in the nightly job fails it (code-quality CQ-38).
{ "v": 1, // codec version "s": "<revSchemaHash>", // 02 §4.3; the server rejects a mismatch (schemaMismatch) "t": ["cell", "sheet_row"], // table dictionary for this op "c": [["content", "bold"], ["sortKey"]], // column dictionary per table (same order as "t") "ch": [ // [op, tableIdx, rowId(base62, 22 chars), after?, before?] // op: 0 = insert, 1 = update, 2 = delete // after/before: arrays aligned with the table's column dictionary; a column that is not // part of the change is encoded as the sentinel {"_":0} ("absent") [1, 0, "4fG…", ["n:42", {"_":0}], ["s:old", {"_":0}]], [0, 1, "7Hb…", ["a0V"]], [2, 0, "9zA…", null, ["n:1", true]] ] }