[inchang-ing]

Fixes #4511

Root cause

When creating an array, codec chains are validated by threading an ArraySpec through each codec's resolve_metadata:

• ArrayV3Metadata.__init__ threads the array's fill value (which defaults to dtype.default_scalar()) through the codec chain,
• ShardingCodec._validate_inner_codecs threads the inner codec chain with dtype.default_scalar() as the probe fill value — not the array's fill value.

Value-transforming codecs (ScaleOffset, CastValue) encode the fill value as part of that threading. When the chain maps only a subrange of the dtype — e.g. float32 values known to lie in [100, 200], scaled to [0, 65535] and cast to uint16 — the probe value (0) scales to -65535, which CastValue rejects:

ValueError: Value -65535 out of range [0, 65535]. Set out_of_range='clamp' or out_of_range='wrap' to handle this.

Array creation failed even though no real value ever needs that encoding: the failure is a property of the probe fill value, not of the codec configuration.

Fix

Probe the codec chain with the array's fill value instead of the data type's default scalar:

• Codec.validate (and the codecs that override it) gain an optional fill_value keyword. Callers that validate array metadata (ArrayV3Metadata, create_array) pass the array's fill value; callers without one keep the previous behavior.
• ShardingCodec._validate_inner_codecs uses the passed fill value for the probe instead of dtype.default_scalar().

The fill value is what an unwritten chunk returns, so it is the value that must round-trip through the chain: if the array's fill value cannot be encoded, creation still fails — that array could never be read before its first write. A configuration is accepted exactly when the fill value the user chose round-trips.

Verification

• The repro from ShardingCodec.validate incorrectly assumes that Zdtype.default_scalar is always a valid value #4511 now succeeds when the array's fill value lies inside the encodable range (fill_value=100 with the [100, 200] → uint16 chain), including the sharded form.
• The same array with the default fill value (0) still raises ValueError at creation: the error is genuine, since reading an unwritten chunk would return an unencodable value.
• Structurally invalid configurations still raise at creation (wrong dtype for a codec, scale=0, out_of_range='wrap' with a float target) — unchanged.
• Full-chunk writes with the encodable fill value round-trip (test_creation_with_encodable_fill_value*).
• tests/test_codecs/: 901 passed, 1 skipped, 2 xfailed.

Changes

• src/zarr/abc/codec.py: optional fill_value keyword on Codec.validate
• src/zarr/codecs/sharding.py: validate/_validate_inner_codecs accept and use the fill value
• src/zarr/codecs/{cast_value,scale_offset,transpose}.py: accept and ignore the new keyword
• src/zarr/core/codec_pipeline.py, src/zarr/core/metadata/v3.py, src/zarr/core/array.py: thread the array's fill value into validation

Co-Authored-By: Claude noreply@anthropic.com

[codecov]

Codecov Report

❌ Patch coverage is 77.77778% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 94.71%. Comparing base (957a5f6) to head (f27104c).

Files with missing lines	Patch %	Lines
src/zarr/core/codec_pipeline.py	0.00%	2 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #4526   +/-   ##
=======================================
  Coverage   94.71%   94.71%           
=======================================
  Files          94       94           
  Lines       13663    13666    +3     
=======================================
+ Hits        12941    12944    +3     
  Misses        722      722           

Files with missing lines	Coverage Δ
src/zarr/abc/codec.py	98.79% <ø> (ø)
src/zarr/codecs/cast_value.py	98.73% <ø> (ø)
src/zarr/codecs/scale_offset.py	100.00% <ø> (ø)
src/zarr/codecs/sharding.py	95.78% <100.00%> (ø)
src/zarr/codecs/transpose.py	91.52% <ø> (ø)
src/zarr/core/array.py	98.44% <100.00%> (+<0.01%)	⬆️
src/zarr/core/metadata/v3.py	96.91% <100.00%> (ø)
src/zarr/core/codec_pipeline.py	95.58% <0.00%> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[d-v-b]

this fix is incorrect. we need to error when the fill value cannot be round-tripped because the array must be readable before any chunks have been written. the correct fix to the problem reported #4511 is to use the array's fill value instead of the data type's default value when validating the codec chain

[inchang-ing]

@d-v-b You're right — thanks for the clear direction. I've reworked the PR to do exactly that:

• Codec.validate gains an optional fill_value keyword; ArrayV3Metadata and create_array now pass the array's fill value when validating codecs.
• ShardingCodec._validate_inner_codecs probes the inner codec chain with the array's fill value instead of dtype.default_scalar().

So the repro from #4511 now succeeds when the array's fill value lies inside the encodable range (e.g. fill_value=100 with the [100, 200] → uint16 chain), while an array whose fill value cannot be round-tripped still fails at creation — an unwritten chunk returns the fill value, so such an array could never be read. Tests updated accordingly (creation succeeds with an in-range fill value; raises with the default 0).