[do not merge] benchmarks + tests for phased codecpipeline

[d-v-b]

This PR should not be merged. It contains changes necessary to make the codec pipeline developed in #3885 the default, which allows us to run our full test suite + benchmarks against that codec pipeline class.

[codecov]

Codecov Report

❌ Patch coverage is 86.62207% with 80 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.10%. Comparing base (0ea15fd) to head (efce610).

Files with missing lines	Patch %	Lines
src/zarr/core/codec_pipeline.py	86.18%	63 Missing ⚠️
src/zarr/codecs/numcodecs/_codecs.py	66.66%	8 Missing ⚠️
src/zarr/codecs/sharding.py	92.30%	7 Missing ⚠️
src/zarr/codecs/_v2.py	80.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3891      +/-   ##
==========================================
- Coverage   93.07%   92.10%   -0.98%     
==========================================
  Files          85       87       +2     
  Lines       11228    11801     +573     
==========================================
+ Hits        10451    10869     +418     
- Misses        777      932     +155     

Files with missing lines	Coverage Δ
src/zarr/abc/codec.py	98.86% <100.00%> (+0.09%)	⬆️
src/zarr/core/array.py	97.82% <100.00%> (+0.01%)	⬆️
src/zarr/core/config.py	100.00% <ø> (ø)
src/zarr/codecs/_v2.py	90.19% <80.00%> (-3.43%)	⬇️
src/zarr/codecs/sharding.py	75.42% <92.30%> (-13.99%)	⬇️
src/zarr/codecs/numcodecs/_codecs.py	93.18% <66.66%> (-3.21%)	⬇️
src/zarr/core/codec_pipeline.py	87.62% <86.18%> (-6.57%)	⬇️

... and 6 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[codspeed-hq]

Merging this PR will degrade performance by 99.6%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 32 improved benchmarks
❌ 29 regressed benchmarks
✅ 5 untouched benchmarks
⏩ 6 skipped benchmarks¹

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
⚡	WallTime	test_read_array[memory-Layout(shape=(1000000,), chunks=(100,), shards=(1000000,))-None]	2.9 s	1.1 s	×2.5
⚡	WallTime	test_write_array[memory-Layout(shape=(1000000,), chunks=(1000,), shards=(1000,))-None]	1,607.9 ms	845 ms	+90.28%
⚡	WallTime	test_write_array[memory-Layout(shape=(1000000,), chunks=(1000,), shards=None)-gzip]	1,015.3 ms	676.7 ms	+50.04%
⚡	WallTime	test_read_array[memory-Layout(shape=(1000000,), chunks=(100,), shards=(1000000,))-gzip]	5.9 s	2 s	×3
⚡	WallTime	test_write_array[memory-Layout(shape=(1000000,), chunks=(100,), shards=(1000000,))-gzip]	9.5 s	2.7 s	×3.5
⚡	WallTime	test_slice_indexing[None-(0, 0, 0)-memory]	848.4 µs	679.5 µs	+24.85%
⚡	WallTime	test_slice_indexing[None-(slice(None, 10, None), slice(None, 10, None), slice(None, 10, None))-memory]	875.1 µs	676.4 µs	+29.37%
⚡	WallTime	test_write_array[memory-Layout(shape=(1000000,), chunks=(1000,), shards=(1000,))-gzip]	2.1 s	1 s	×2.1
⚡	WallTime	test_read_array[local-Layout(shape=(1000000,), chunks=(1000,), shards=(1000,))-gzip]	1.8 s	1.4 s	+25.32%
❌	WallTime	test_read_array[local-Layout(shape=(1000000,), chunks=(100,), shards=(1000000,))-None]	2.9 s	4.6 s	-38.5%
⚡	WallTime	test_slice_indexing[(50, 50, 50)-(slice(0, None, 4), slice(0, None, 4), slice(0, None, 4))-memory_get_latency]	426.4 ms	261.6 ms	+62.99%
⚡	WallTime	test_write_array[local-Layout(shape=(1000000,), chunks=(1000,), shards=None)-gzip]	1.6 s	1.2 s	+36.57%
⚡	WallTime	test_read_array[local-Layout(shape=(1000000,), chunks=(1000,), shards=None)-gzip]	938.5 ms	776 ms	+20.94%
⚡	WallTime	test_write_array[local-Layout(shape=(1000000,), chunks=(1000,), shards=(1000,))-None]	2.8 s	1.4 s	×2
⚡	WallTime	test_write_array[memory-Layout(shape=(1000000,), chunks=(100,), shards=(1000000,))-None]	5.3 s	1.5 s	×3.5
⚡	WallTime	test_write_array[local-Layout(shape=(1000000,), chunks=(1000,), shards=None)-None]	1,181.3 ms	991.7 ms	+19.12%
❌	WallTime	test_slice_indexing[None-(slice(0, None, 4), slice(0, None, 4), slice(0, None, 4))-memory_get_latency]	425.1 ms	576.4 ms	-26.24%
⚡	WallTime	test_write_array[local-Layout(shape=(1000000,), chunks=(1000,), shards=(1000,))-gzip]	3.2 s	1.5 s	×2.1
❌	WallTime	test_slice_indexing[None-(slice(None, None, None), slice(None, None, None), slice(None, None, None))-memory]	376.5 ms	519.9 ms	-27.59%
⚡	WallTime	test_write_array[local-Layout(shape=(1000000,), chunks=(100,), shards=(1000000,))-None]	5.3 s	1.5 s	×3.5
...	...	...	...	...	...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

---

_{Comparing d-v-b:perf/prepared-write-v2-bench (8330cde) with main (7c78574)²}

Footnotes

1. 6 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

2. No successful run was found on main (2fbd30a) during the generation of this report, so 7c78574 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩