perf: phased codecpipeline#3885

Open

d-v-b wants to merge 85 commits intozarr-developers:mainfrom

d-v-b:perf/prepared-write-v2

Commits on Apr 7, 2026

feat: define `PreparedWrite` and `SupportsChunkPacking` data structures
d-v-b
committed
Merge branch 'main' of https://github.com/zarr-developers/zarr-python into perf/prepared-write-v2
d-v-b
committed
feat: new codec pipeline that uses sync path
d-v-b
committed

Commits on Apr 14, 2026

docs: add design spec for PhasedCodecPipeline performance fix

d-v-b
and
claude
committed
docs: update spec — global thread pool for compute, not inline or to_thread

d-v-b
and
claude
committed
docs: rework spec around streaming pipeline architecture

d-v-b
and
claude
committed
docs: add partial shard write support to spec

d-v-b
and
claude
committed
docs: add implementation plan for PhasedCodecPipeline performance fix

d-v-b
and
claude
committed
fix: remove duplicate @staticmethod decorator on _scatter
d-v-b
committed
feat: add global thread pool for codec compute

d-v-b
and
claude
committed
perf: add SimpleChunkLayout fast path in _transform_read

d-v-b
and
claude
committed
perf: rewrite PhasedCodecPipeline read/write as streaming pipelines

d-v-b
and
claude
committed
refactor: simplify sync read/write with SimpleChunkLayout fast path

d-v-b
and
claude
committed
refactor: remove dead code from old phase-based read/write

d-v-b
and
claude
committed
feat: add ByteRangeSetter protocol and implement in memory/local stores

d-v-b
and
claude
committed
feat: add partial shard write support for fixed-size inner codecs

d-v-b
and
claude
committed
perf: inline decode for non-sharded reads — skip executor overhead

d-v-b
and
claude
committed
perf: sync fast path in async read/write — bypass event loop for sync stores

d-v-b
and
claude
committed
perf: optimize read_sync — inline layout lookup, direct store.get_sync

d-v-b
and
claude
committed
perf: cache default ArraySpec for regular chunk grids

d-v-b
and
claude
committed
perf: optimize shard decode — eliminate BasicIndexer, cache prototype

d-v-b
and
claude
committed
feat: add partial shard write support for fixed-size inner codecs

d-v-b
and
claude
committed
perf: vectorized shard decode for fixed-size inner codecs

d-v-b
and
claude
committed

Commits on Apr 15, 2026

Commits on Apr 17, 2026