CRDT for documents (4/6): public API cleanup + save/load performance

pkg/automerge/PARITY_PLAN.md

@@ -67,7 +67,8 @@ patches, and native reproduces that reference behavior exactly.
 
 DEFLATE compression on save is now covered: the native save compresses change
 chunks whose body reaches the reference DEFLATE_MIN_SIZE threshold (small changes
-stay byte-identical), and SaveNoCompress mirrors AutoCommit::save_nocompress.
+stay byte-identical), and `Save(ctx, NoCompress())` mirrors
+AutoCommit::save_nocompress.
 
 Transaction isolation is now covered: `isolate` pins reads and writes to a
 historical frontier using derived concurrency actors (matching Rust's
@@ -79,11 +80,46 @@ remains, because it asserts the exact incremental patch stream produced by Rust'
 patch log across isolate/integrate, which native's state-comparison diff does not
 reproduce.
 
-Orphan retention across save/load is now covered without a full document-chunk
-encoder: the native save appends retained orphan changes and the native load
-falls back to a dependency-tolerant path that applies every change whose
-dependencies are satisfiable and queues the rest (still failing a load that can
-apply nothing, so a bare orphan without a base is rejected as before).
+Orphan retention across save/load is covered: the native save appends retained
+orphan changes and the native load falls back to a dependency-tolerant path that
+applies every change whose dependencies are satisfiable and queues the rest
+(still failing a load that can apply nothing, so a bare orphan without a base is
+rejected as before).
+
+Snapshot writing is implemented and is now what `Save` produces, matching the
+Rust and JavaScript `save()` semantics. It compacts the whole history into one
+document chunk, followed by any retained orphan changes as trailing change
+chunks, and DEFLATEs individual columns above a size threshold. It is gated on
+byte identity with the reference for the same history across linear text, map
+puts and deletes, counters, marks and unmarks, and text deletion; re-encoding a
+reference-written snapshot reproduces that file exactly for the official fixture
+and for reference histories covering nested objects, lists and a merged
+multi-actor graph. Compaction matters for size as well as parity: a 200-commit
+typing history is 21816 bytes as the old change stream and 400 bytes compacted.
+
+`Save` falls back to the faithful change stream (the loaded base plus each change
+chunk since) when a history cannot be compacted: while isolated, or when the
+change graph is not internally consistent. That stream is also what preserves
+loaded bytes verbatim, so a document is never rewritten in a lossy way when it
+cannot be safely compacted. `SaveIncremental` is unaffected: `Save` leaves the
+incremental cursor at the end exactly as the stream save did.
+
+Compressed columns are not byte-identical to the reference because the DEFLATE
+implementations differ, so the byte-identity gate holds only for histories small
+enough that no column crosses the threshold; above it both files are valid and
+load equally, and compression is a size optimization the decoder reverses on the
+way back in.
+
+Two smaller differences remain. The reference `am_save_no_orphans` shim sets
+`deflate: false`, while Rust's own `SaveOptions::default()` compresses, so
+`Save(ctx, DiscardOrphans())` diverges from the shim rather than from Rust;
+correcting it means rebuilding the WASM oracle. Unknown columns survive a normal
+load
+because the loaded bytes are kept verbatim, and `EncodeDocument` writes them back
+to the table they came from when a document is re-encoded unmodified, but they
+cannot be carried across a compaction of a mutated history because their rows no
+longer line up with the recomputed columns; this matches Rust, which also drops
+them across re-serialization.
 
 The V2 sync internals are now covered: the empty-message codec round-trips to
 the reference wire bytes, and Bloom false-positive recovery is verified on both
@@ -101,9 +137,9 @@ JavaScript binding-type helpers (ImmutableString/RawString, legacy Text-as-array
 proxy/change-callback) are likewise recorded as api-convenience or
 language-specific rather than interop-required.
 
-## Known native defects (found by parity reproduction, fix pending)
+## Native defects found by parity reproduction (resolved)
 
-**Mark boundaries: fixed for the reported case, deeper cases still diverge.**
+**Mark boundaries: matches the reference, including the error paths.**
 The originally reported defect is fixed. Mark begin and end operations now hold
 positions in the sequence order, insertions (including the mark boundaries
 themselves) resolve their anchors through a port of the reference's insert
@@ -139,24 +175,29 @@ checks span consolidation and text) drove a series of fixes this pass:
   operation reusing that counter is not mistaken for it.
 
 These closed the common cases. The dangling begin the reference leaves behind
-when a mark is applied with an out-of-range end boundary is now largely handled:
-the mark call fails, but the begin was already recorded, and it then covers text
+when a mark is applied with an out-of-range end boundary is now handled in full.
+The mark call fails, but the begin was already recorded, and it then covers text
 according to its expand direction. A leftward-expanding begin (expand "before" or
 "both") sorts after same-anchor insertions by descending operation ID, so its
-walk index lands past text it should cover; `richTextMarks` now starts such a
+walk index lands past text it should cover; `richTextMarks` starts such a
 dangling begin at the position just after the begin's own anchor element (or the
-document start for a head anchor) rather than at its walk index. This is the exact
-range the reference produces: for `mark(0,3,before)` on empty text, split at 0,
-insert `"w"` at 0, both engines now report `"w"` bold (`bold 0..2`).
-
-`TestRustText_DanglingMarkBoundaries` gates four delta-debugged reproducers that
-previously diverged. A randomized value-level sweep including out-of-range
-boundaries and all four expand modes dropped from roughly 7% divergence to about
-3%; the same sweep restricted to in-range marks diverges on none. What remains are
-deeper interactions — several overlapping dangling begins whose anchor elements
-have since been deleted — where native over-extends one of the marks. These are
-strictly an error path (no valid caller marks past the end of the text) and never
-arise from the frontend, which clamps mark ranges to the text length.
+document start for a head anchor) rather than at its walk index.
+
+The last remaining divergences were not in span computation but in authoring: a
+split block did not resolve its insertion anchor against neighbouring mark
+boundaries the way a text insertion does. A block inserted next to a dangling
+begin therefore landed on the wrong side of it, and every insertion anchored
+after that block inherited the mistake, so the marks the following text carried
+diverged from the reference in both directions (a mark dropped, or a mark leaking
+past a block). `SplitBlock` now resolves its anchor through the same insert query
+as `Splice`.
+
+`TestRustText_DanglingMarkBoundaries` gates eleven delta-debugged reproducers,
+and `TestRustText_MarkValuesMatchReferenceUnderErrors` compares marked spans run
+for run against the reference across two thousand randomized scenarios that
+include out-of-range boundaries and every expand mode. A wider sweep of
+twenty-four thousand scenarios across six seeds, up to seventeen steps each, found
+no divergence.
 
 **Concurrent re-encoding is now byte-identical.** Assigning the value a key
 already resolves to used to skip writing an operation. That is correct for an
@@ -191,6 +232,7 @@ These do not block Go/Rust engine parity but remain tracked:
 Implement and verify:
 
 - complete document, change, compressed-change, and bundle parsing;
+- canonical document-chunk encoding, what Save now writes (done);
 - canonical encoding for every operation and scalar column;
 - expanded/compressed change byte and hash stability;
 - 64-bit object IDs and actor tables referenced only by deletes;