CRDT for documents (3/6): fuzzing, ProseMirror render parity, internal refactor - #1751
Closed
gearnode wants to merge 36 commits into
Closed
CRDT for documents (3/6): fuzzing, ProseMirror render parity, internal refactor#1751gearnode wants to merge 36 commits into
gearnode wants to merge 36 commits into
Conversation
Text editing scaled quadratically because every sequence read rebuilt the whole RGA tree from the operation set, allocating fresh maps each time. A thousand sequential inserts took 237ms and over a million allocations, roughly 34 times slower than native Rust. The insertion order depends only on insertion operations and their anchors, so cache it per sequence object and maintain it in place. Local insertions always carry the current maximum operation ID, so they sort ahead of every sibling and splice next to their anchor; merged changes and rollbacks drop the entry and it is rebuilt lazily. Reads now walk the cached order instead of rebuilding the tree. The same thousand inserts now take 37ms with about eight thousand allocations, and a hundred inserts are faster than native Rust. Resolving a position still materializes the sequence, so closing the remaining gap needs a positional index. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
Splicing text still rebuilt the visible element slice on every keystroke, so a thousand sequential inserts allocated 89MB and took 37ms against native Rust's 6.9ms. Resolving element values also rescanned every operation for every element, which is quadratic once a document contains replacements. Cache the materialized visible elements and values per sequence object next to the insertion order. Appending a new element at the end extends the cached slices in place, so sequential editing stays linear, while any other mutation drops the entry and it is rebuilt on demand. Element value winners are now collected in a single pass instead of once per element. A thousand inserts now take 6.8ms and allocate 2MB, matching native Rust, and a hundred inserts are over five times faster than native Rust. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
Reading a map property scanned every operation in the document, so each assignment cost time proportional to the document size and a thousand assignments took 15.9ms against native Rust's 10.1ms. Group operation IDs by the property they address. The index is built on first use and maintained as operations are applied, so a read only visits the operations for that key. A thousand assignments now take 2.7ms, and both map workloads are close to four times faster than native Rust. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
Assigning the identical value to a list element with a single visible value produced a redundant change on the native engine, while the reference and native's own map assignment treat it as a no-op. Randomized differential stress testing surfaced the divergent head that resulted. Skip the operation when the element has one visible value equal to the new value; a conflicted element still records the assignment so the conflict resolves. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
Exercise the incremental sequence and map caches the way real editing does, where cache-invalidation bugs hide. Identical random operation sequences run in lockstep on the native and reference engines: a single-document test compares full state, including heads, after every commit and after a save/load round trip, and a concurrent-merge test forks two peers, edits them independently, merges both directions, and asserts value convergence. The single-document test caught the redundant list assignment fixed in the previous commit. The concurrent-merge test documents that independent re-encoding of the same concurrent edits can pick different change bytes while still converging on values; that is a determinism nuance rather than an interoperability break, so it compares values rather than heads. Both observations are recorded in the parity plan. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
Turn the lockstep native/reference comparison into a Go fuzz target so continuous fuzzing keeps exploring the operation space. Each pair of fuzzer bytes selects and parameterizes one map, list, or text operation, and after every commit the engines must agree on materialized values. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
Record the minimal reproduction of the mark that is dropped when the element anchoring its range is deleted, why native cannot distinguish it from the case that must drop the mark, and the three coordinated changes a fix needs. An attempt at those changes is summarized because the failure mode is instructive: approximating the reference's insert query loses marks entirely instead of degrading gracefully, because a splice that replaces marked content leaves the deleted element between the mark begin and end operations and a naive cancellation rule then anchors the replacement outside the mark. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
Text inserted at an expanding mark boundary lost the mark once the content that anchored the range was deleted, because native chose insertion keys without consulting mark operations. The two engines were then structurally indistinguishable for a case that must keep the mark and one that must not. Give mark begin and end operations positions in the sequence order, resolve insertion anchors through a port of the reference's insert query, and derive spans from a mark state machine walking that order. The mark boundaries themselves are placed through the same query, and a splice now resolves its anchor and inserts before deleting, as the reference does, so replacement text is positioned against the pre-deletion sequence. Two ordering rules fell out of this: mark precedence follows creation order, so a later unmark overrides an earlier mark, and a mark left open covers nothing, which is how a zero-length mark presents itself. Randomized value-level differential testing now survives roughly five times as many generated scenarios; the remaining divergences are recorded in the parity plan. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
Randomized value-level differential testing of marks surfaced two places where native rejected an out-of-range edit that the reference clamps. A splice whose deletion count runs past the end of the text now deletes the remaining elements and stops, matching the reference, whose splice loop ends when there are no more elements to delete. A mark boundary past the end of the text is clamped to the end for the same reason, so marking a range wider than the current text marks the whole text instead of silently doing nothing. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
The last known mark divergence is a mark whose end is past the end of an empty text combined with expand-both, followed by an insertion at the head. The reference captures the inserted text and treats it differently from a true zero-length mark; native cannot, because on empty text both anchors resolve to the head, so distinguishing them needs a richer anchor for a boundary beyond the end. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
Marking a text with an end boundary past its end is an error, and the reference applies the begin boundary before failing, leaving a begin with no matching end. Reject the out-of-range boundary as before but make span computation extend such an unmatched begin over the text that follows it, matching the reference, instead of clamping the boundary. Distinguish an unmatched begin from a zero-length mark, whose end operation exists but sorts before the begin, by confirming the operation after the begin is actually a mark end; this stops a later operation that reuses the freed counter, such as a delete, from being mistaken for the missing end. Remaining divergences are confined to the reference's out-of-range error path, where a dangling begin covers text by its expand direction; they are recorded in the parity plan. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
The secret scanner reports verified findings for one of the upstream Automerge test names recorded in our parity inventory: its Lob detector looks for a "test_" prefix followed by thirty-five characters, and that name happens to have exactly that shape. Exclude the three files that carry upstream test names, following the existing convention for scanner false positives. The names have to be reproduced verbatim to map our coverage onto the upstream suite, so they cannot be spelled differently. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
The lint job runs "go fix -diff" and fails when it suggests changes. Apply its suggestions across the Automerge packages: counting loops become range loops, a membership loop becomes slices.ContainsFunc, and a few other rewrites it proposes. Also add the blank lines the whitespace linter asks for, so the package reports no findings. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
The Go span-to-ProseMirror renderer is a second implementation of @automerge/prosemirror's pmDocFromSpans, and two behaviours diverged from it. Marks were emitted in alphabetical order of their Automerge names, so a bold-italic run serialised as italic-then-bold; ProseMirror stores marks in schema-rank order, so the persisted document differed from what the browser shows. Order marks by that schema rank instead. The renderer also returned an error for any block type or mark it did not recognise. Because the collaboration service renders inside the persist transaction, one unfamiliar block from a newer client aborted every save. Degrade gracefully instead: render an unknown block as a paragraph and hoist its children, and drop unknown or malformed marks. The downstream markdown and HTML renderers reject unknown types, so emitting only known nodes and marks keeps persistence and publishing working. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
The Go renderer had no oracle: its tests only checked hand-written JSON, so it could drift from @automerge/prosemirror without anyone noticing, and the server would then persist documents that differ from what collaborators see. Add a differential gate. A frontend fixture generator runs a corpus of realistic documents through the real schema adapter and the official pmDocFromSpans, records the Automerge bytes and the canonical ProseMirror JSON, and a Go test replays each document and asserts byte parity. Generate the fixture with the new make target; the frontend spec doubles as a drift guard against the committed golden. This immediately surfaced the mark-order divergence fixed in the previous commit. Seed the render fuzzer with unknown block and mark inputs to keep the graceful-degradation path exercised. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
The block-type strings, mark names, and mark order lived in three places that had to agree by hand: the Go renderer, the frontend schema adapter, and the mark order baked into the renderer. Adding or renaming a node or mark on one side without the others would silently break round-trips or reorder marks in the persisted document. Introduce a shared ledger, testdata/schema-mapping.json, as the single source of truth. Make the Go renderer derive its block and mark tables from ordered slices so they are introspectable, and add a Go test plus a frontend test that both hold their side to the ledger. A change in any one place now fails a test until all three agree. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
humanizeSeconds returned an empty string when a tracker had no max age, so cookie-banner rows showed a blank duration instead of "Session". The duration.session and duration.persistent keys already exist in every locale and a test already specified the behaviour, but the implementation never produced them. Return the session label for a null or non-positive max age, and the persistent label when the value is stored in local storage. This also unblocks running the frontend test suite in CI, which the next commit wires up. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
The vitest suites, including the new ProseMirror render parity and schema mapping guards, only ran when someone remembered to invoke them locally, so nothing enforced the frontend half of the collaboration contract. Add a turbo test task and a per-workspace test script, expose it through a test-js make target, and run it from a dedicated CI job. Scripts pass with no test files so packages without tests stay green, and the job mirrors the lint-js setup so the environment matches. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
The parity corpus was generated only from clean documents, so it never exercised the span arrangements that arise from editing — the very shapes behind past divider bugs, such as a horizontal rule created between paragraphs by an actual insertion. Drive documents through the real collaboration sync plugin and record the resulting spans: a divider inserted then typed into, two stacked dividers, and a typed table cell. The Go renderer matches upstream on all of them. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
The collaboration service materializes ProseMirror JSON inside its persist transaction, but malformed parent paths and structurally invalid children could make Render return an error and abort every save for the document. Normalize impossible parent paths, create safe paragraphs for orphaned inline content, hoist children out of leaf blocks, and partition malformed table content into valid rows and cells plus preserved spill content. Missing block types and future span types now degrade without losing known text. The fuzzer now enforces that every JSON-decodable span sequence renders without an error, while tests verify the output remains renderable as HTML and Markdown. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
The bridge only checked Automerge spans flowing into ProseMirror. Our custom schema adapter's reverse mapping through pmNodeToSpans had no oracle, so a renamed block, lost mark, or changed attribute could silently produce different CRDT structure. Record the normalized spans emitted by the official pmNodeToSpans for every clean and edit-derived corpus case. The frontend test guards those spans against drift and verifies they equal the spans materialized by updateSpans; the Go replay gate loads the saved document and independently asserts that native spans match the committed oracle before rendering back to canonical ProseMirror JSON. Export the structural-block preprocessing helper so the oracle exercises the exact import path used in production. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
Performance work needs stable workloads rather than one-off timing scripts. Add Go benchmarks for plain text, one thousand paragraphs, a 100x10 table, and malformed hierarchy recovery, plus Vitest benchmarks for both official bridge directions on equivalent paragraph and table documents. Expose both through make benchmark-prosemirror so future changes can be compared against the same corpus. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
A table cell or header that appeared outside a row still rendered as a top-level cell node. Persistence accepted it, but the Markdown renderer rejects table parts outside a table, so exporting or publishing such a document failed afterwards. Reject table parts whose parent chain cannot hold them and unwrap any that are hoisted out of a degraded block, so they contribute their text instead. The fuzzer now asserts the stronger property that every rendered document is also valid Markdown and HTML, which is what surfaced this; it found a second case where a stray row lifted its cells verbatim. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
The bridge parity gate covered only twenty-one hand-picked documents, so a valid arrangement outside that corpus could diverge from the official library without detection. Generate 256 deterministic documents from reproducible seeds across text, Unicode, marks, links, headings, blockquotes, code, lists, hard breaks, and tables. Run each through both official bridge directions and replay the saved document in Go. The generated corpus immediately found a real bug: blockquotes with explicit child paragraphs gained an extra empty paragraph in the Go renderer. Fix that behavior and keep a focused regression test. Compress the generated oracle to keep the repository fixture small while retaining exact documents, spans, and expected ProseMirror output. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
The render oracle did not include multi-actor merge arrangements or marks created by real ProseMirror transactions. Add concurrent overlapping marks, a divider racing marked text, concurrent table rows, partial unmarking, overlapping links and italic text, and an emoji boundary edit. These scenarios exposed another production bug in the adapted sync plugin: the upstream step fast path writes ProseMirror mark names directly, bypassing our schema mapping, and remove-mark steps can leave the wrong range marked. Reconcile mark transactions through pmNodeToSpans/updateSpans instead, which correctly writes strong/em and preserves exact partial ranges. Assert every live editor state equals its resulting Automerge projection and keep a focused mark-name regression test. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
Assigning the value a key already resolves to skipped writing an operation. That is right for an ordinary key, but a key holding concurrent values has to collapse: the reference deletes the losing siblings and keeps the winner. Go left the conflict standing, so the two engines then disagreed about which values were visible, emitted different predecessors on later deletes, and produced different change bytes and hashes. Emit that delete for both map keys and list elements, and make the merge stress test compare whole documents including heads. A new gate asserts that two peers per engine encode every concurrent change to identical bytes, which is what surfaced this; it now holds across three thousand seeds where it previously failed within a few hundred. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
The remaining mark difference was recorded only as prose, so nobody could act on it. Delta debugging against the reference reduces it to four scenarios, the smallest being a mark past the end of empty text followed by a split and an insertion at zero: the reference reports the inserted character marked, native reports it bare. Record those scenarios as a test, skipped by default because marking past the end of the text is an error path, and runnable while fixing it. Describe where the fix belongs, and note that a randomized sweep including out-of-range boundaries diverges on roughly seven percent of scenarios while the same sweep restricted to valid marks diverges on none. Also record that concurrent re-encoding is now byte-identical. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
When a mark is applied past the end of the text the call fails, but the begin operation was already recorded. A begin that expands leftward sorts after the insertions sharing its anchor by descending operation ID, so its position in the walk landed past the text it should cover and native dropped the mark for that text while the reference kept it. Start such a dangling begin at the position just after its own anchor element, or at the document start for a head anchor, which is the exact range the reference produces. The four delta-debugged reproducers now pass and gate the behavior, and a randomized error-path sweep drops from roughly seven percent divergence to about three. The deeper residual, several overlapping dangling begins whose anchors were later deleted, is recorded in the parity plan; it is an error path the frontend never reaches because it clamps mark ranges to the text length. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
A document that holds an orphaned change recomputes a Need for the missing base every time it receives a sync message, and generation never cleared that Need. Every quiescence guard required an empty Need, so the server's bounded send loop generated the same request a hundred times and then aborted the whole collaboration connection with "protocol did not quiesce". Any document carrying a permanently orphaned change failed on every reconnect. Track the Need carried by the last message we sent and treat an unchanged Need as no new information, so a repeated request quiesces instead of looping while still going out again whenever the Need actually changes. A regression test drives the server's send loop against a document with an orphaned change and fails with the production error without this change. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
The pure-Go engine accumulated lifecycle, objects, rich text, transactions, patches, history, merge and sync in one five-thousand-line backend file, while state materialization and rich-text anchor logic shared another large file. The code was difficult to navigate and made unrelated changes collide. Split the implementation into focused files inside the internal native package: engine lifecycle, object operations, rich text, transactions, patches, history, sync and common helpers, plus separate sequence, rich-text and hydration state. Document the boundaries and dependency direction. This is a mechanical move with no behavior changes; parity, conformance and race suites remain green. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
Feature-level sync tests did not exercise the combinations that produced the production orphan livelock. Add a deterministic three-peer state machine that mixes concurrent map and text edits, dropped and duplicated messages, read-only transitions, document reloads, serialized sync-state restoration, and repeated generation without replies, then requires reliable full-mesh convergence. The chaos model immediately found another non-quiescing state: a peer could carry a stale Need while changing to read-only, leaving the sender with a request it was forbidden to service forever. Discard requests from read-only peers; they advertise missing heads again when writable. Pin the exact message shape with a focused regression test that fails with the production quiescence error without the fix. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
The local Go/Rust comparator covered only seven synthetic workloads and could not stand in for upstream's benchmark battery. Add a runner pinned to the real automerge repository and its stable JSON schema, expose list and fast-tier make targets, and schedule the complete forty-workload fast tier weekly with its result uploaded as an artifact. Use the official benchmark-battery constructors directly to generate big paste, random splice, list splice, typing, deep-history, and nested-map documents. A Rust-Go-Rust interoperability gate loads and saves every fixture in both engines and requires identical heads, so the performance corpus now provides additional compatibility coverage rather than timing alone. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
A document could reach a state where a frontier head referenced a change that was no longer retrievable. Every incremental read then failed with "cannot compute changes from unknown heads", and because the collaboration service computes changes since the pre-merge heads on every persist, saving that document aborted on every reconnect. Make incremental reads over-approximate instead of failing: a baseline head that is unknown excludes nothing, matching Rust's get_changes, which takes have_deps by value and never errors, and a frontier head without a retrievable change contributes nothing and is skipped. Rebuild the frontier from the change graph when a loaded document's recorded heads reference a change it does not carry, so Heads() stays consistent with the graph and the document self-heals. Regression tests cover the rebuilt frontier, an unknown baseline, and a frontier head with no change; the invariant test now asserts the Rust-aligned tolerant behavior rather than an error. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
Splitting the five-thousand-line engine into files improved navigation but left storage, sync, encoding and the CRDT model coupled in one Go package. Extract acyclic internal packages for shared types, binary encoding primitives, chunk storage/validation and the V1/V2 sync wire codec. Move each package's unit tests with its implementation and keep the native engine focused on mutable CRDT state, rich text, patches and protocol orchestration. Rename the production Backend abstraction and files to Engine; only the thin private engine contract remains so the public API can run the native engine and the Rust/WASM differential oracle through the same surface. Storage and sync are now facades over their independent packages, not responsibilities of the engine. Document the package dependency graph and keep compatibility aliases internal so this is behavior-preserving. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
This was referenced Aug 21, 2026
Closed
Closed
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of the split of #1657 ("CRDT for documents") into a review stack. Stack index: 3/6. Stacked on #1750.
What this PR does
Differential fuzzing, mark-boundary defect fixes, a ProseMirror render-parity gate against upstream, frontend test suite wired into CI, a real Automerge benchmark battery, model-based sync/persistence chaos tests, then an internal refactor that splits the native engine into real Go packages.
Heads up for reviewers
Net line count skews toward deletions relative to insertions here because of the package-extraction refactor at the end — that's internal reorganization, not feature removal.
Stack
automerge/1-prototype-bringup(CRDT for documents (1/6): bring up Automerge-backed collaboration (prototype) #1749): bring up Automerge-backed collaboration (prototype)automerge/2-engine-core-parity(CRDT for documents (2/6): Automerge engine core + full upstream interop parity #1750): Automerge engine core + full upstream interop parityautomerge/4-api-cleanup-perf: public API cleanup + save/load performanceautomerge/5-repo-protocol-migration: migrate wire protocol to automerge-repo + cross-instance fan-outautomerge/6-frontend-cutover-legacy-removal: cut the frontend over, remove the legacy protocol