Support resumable large URL site imports

[chubes4]

Summary

• discovers same-origin routes from sitemap indexes and URL sets into canonical website artifacts
• imports large sites through deterministic, bounded, resumable page batches
• retains artifact ownership, response and negative caches, adaptive splits, compact quality evidence, and atomic checkpoints across failures
• preserves optional failed assets externally while rejecting incomplete required HTML collection
• canonicalizes encoded and colliding source routes without losing sitemap entries
• adds generic artifact-run lifecycle primitives and complete standalone/WordPress regression coverage

Tracks #771.

Scale proof

The implementation completed a real Studio import from https://www.richlynngroup.com/:

• 1,145/1,145 routes completed
• 124/124 batches completed
• zero failed or pending batches at completion
• 1,148 unique WordPress page URIs with no duplicate page paths
• 5,364 network requests avoided by the retained response cache
• recovery proven across interrupted compilation, encoded Unicode paths, transient HTML classification, and canonical route collisions
• measured retained-batch compiler CPU improved by about 31% with the dependent Blocks Engine changes

This proves the collector and resumable execution path can handle the target scale.

Quality boundary

The generated Richlynn site is evidence, not an accepted generated-site result:

• 1,142 pages contain core/html
• 13,055 core/html blocks remain
• aggregate batch evidence reports 1,235 fallbacks
• 396 unavailable assets remain external
• exact visual/editor parity has not been proven

Those are Blocks Engine conversion-quality gaps, not route-scale or resumability failures. The site must not be marked accepted/built under the zero-fallback parity gate.

Validation

• npm test
• strict test manifest: 42 passed, 0 failed
• focused URL collector, URL batch, retained ownership, and WordPress materialization contracts
• dependent Blocks Engine suite: 258 parity fixtures plus canonical/unit/package tests

Dependencies

• Blocks Engine: Harden bounded site artifact compilation blocks-engine#776
• Studio orchestration: Route studio create --from URLs through Data Liberation studio#3952

AI assistance

• Model: openai/gpt-5.6-sol
• Tool: OpenCode
• Used for: architecture exploration, implementation, performance profiling, failure diagnosis, merge-conflict resolution, regression coverage, 1,145-route verification, and PR drafting. Chris Huber remains responsible for the submitted changes.