Merge the upgrade-to-repair rename from #4334

Conflicts resolved by keeping this branch's lines and applying the same rename to
them; this branch's own text now says repair as well.

Assisted-by: ClaudeCode:claude-fable-5-1
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Author: d-v-b

changes/4334.bugfix.md

@@ -4,4 +4,4 @@ A chunk edge length is now always at least 1, while an array extent may be 0. Ev
 
 Stored metadata with a regular chunk size of 0 or JSON `false` (Zarr format 2 `chunks`, Zarr format 3 `regular` `chunk_shape`) is now read as chunk size 1 (the inner chunk size for sharded arrays, which previously failed to open), however long the axis is: that is the chunk size `chunks=-1` gives on a zero-length axis, and it does not depend on how far the axis has grown since. zarr-python wrote such sizes for arrays created with a zero-length axis until 3.4, and, from 2.18.7 to 3.2.1, for an explicit chunk size of 0 or `False` on an axis of any length, as in `chunks=(0,)` (those arrays could store no data). On a zero-length axis this opens silently, as before, and appending to it stores chunks of size 1. On an axis of positive length it opens with a `ZarrUserWarning`, which says that the array holds only its fill value, so that recreating it with the wanted chunk shape loses nothing (`zarr.from_array(array.store, name=array.path, data=array, chunks=..., overwrite=True, write_data=False)` keeps its data type, fill value, attributes and codecs), and how to keep it by storing valid metadata instead: `array.update_attributes({})`, then `zarr.consolidate_metadata` if the metadata is consolidated. JSON `true`, as zarr-python 3.0 and 3.2 wrote for a chunk size of `True`, is read as 1, silently, in the chunk grid (regular, and the explicit edges and run-length encoded sizes of a rectilinear one) and in the inner chunk shape of every sharding codec, nested or not. A stored rectilinear chunk grid whose edge lengths are integral JSON floats (`[[4.0, 2]]`), as zarr-python 3.2 wrote for float edges, is read with those edges as integers, silently; a float anywhere no release wrote one (a regular chunk shape, the inner chunk shape of a sharding codec, a run-length repeat count, an edge below 1) is rejected.
 
-Writing data to an array whose stored metadata holds a regular chunk size of 0 or `false` (other than an empty selection) first stores the upgrade of the metadata the store then holds, unless another writer has stored valid metadata since, so that other readers find the chunks written (a `true` chunk size or an integral float edge is read as the value it equals, so a write stores no metadata for it); in a `ZipStore` this adds a second entry for the metadata document, as every metadata update does. If the metadata the store then holds lays out chunks differently (another writer stored a different chunk size since, say), the write raises a `ValueError` asking to reopen the array, and stores nothing. Apart from the array's own metadata writes (`update_attributes`, `resize`), which store the upgrade as before, this is the only operation that stores it: a group's consolidated metadata keeps such an array's metadata as it was stored (`zarr.consolidate_metadata` copies it as the array's own document holds it) until the array stores its upgrade through the group's handle, and changing a group reads and writes no metadata of its members.
+Writing data to an array whose stored metadata holds a regular chunk size of 0 or `false` (other than an empty selection) first stores the repair of the metadata the store then holds, unless another writer has stored valid metadata since, so that other readers find the chunks written (a `true` chunk size or an integral float edge is read as the value it equals, so a write stores no metadata for it); in a `ZipStore` this adds a second entry for the metadata document, as every metadata update does. If the metadata the store then holds lays out chunks differently (another writer stored a different chunk size since, say), the write raises a `ValueError` asking to reopen the array, and stores nothing. Apart from the array's own metadata writes (`update_attributes`, `resize`), which store the repair as before, this is the only operation that stores it: a group's consolidated metadata keeps such an array's metadata as it was stored (`zarr.consolidate_metadata` copies it as the array's own document holds it) until the array stores its repair through the group's handle, and changing a group reads and writes no metadata of its members.

changes/4431.removal.md

@@ -1,6 +1,6 @@
 Metadata built in code is strict about chunk edge lengths: `ArrayV2Metadata(chunks=...)`, `RegularChunkGridMetadata`, `RectilinearChunkGridMetadata` and `ShardingCodec(chunk_shape=...)` take them as Python `int`s of at least 1. A chunk size of 0 now raises a `ValueError`, as in `ArrayV2Metadata(chunks=(0,))` for an empty array. Code that passes an array's shape as its chunk shape, such as `ArrayV2Metadata(chunks=arr.shape)` or `RegularChunkGridMetadata(chunk_shape=arr.shape)`, fails for an array with a zero-length axis: use `tuple(max(1, s) for s in arr.shape)`. A `bool` or a NumPy integer, including a scalar `chunks=np.int64(5)`, raises a `TypeError`, as a float already did, and so does a chunk shape that is not a list or tuple. This reverses what 3.4.1 said of these classes: `ArrayV2Metadata(chunks=(0,))` is no longer accepted, and a `bool` chunk edge length is no longer read as the `int` it equals. (This entry assumes 3.4.1 is released before this release.) The array creation functions still accept NumPy integers in `chunks=` and `shards=`.
 
-This applies to stored metadata too wherever the stored-document upgrades do not read it: a stored chunk shape that is a bare integer instead of a list, as in a Zarr format 2 `"chunks": 4` or a sharding codec's `"chunk_shape": 2`, which earlier releases read as a one-dimensional chunk shape, now raises a `TypeError` when the array is opened. A group whose consolidated metadata includes such an array does not open from its consolidated metadata at all, and the error names that member: open the group with `use_consolidated=False` to reach its other members. A stored `true` as a rectilinear chunk grid's bare step (`"chunk_shapes": [true]`) or as the count of a run-length pair (`[[4, true]]`), which no known writer stored, raises a `TypeError` too. The chunk sizes the upgrades read open as before: 0 or `false` as a regular chunk size (in Zarr format 2 `chunks` or a `regular` grid's `chunk_shape`); `true` as a regular chunk size, as an edge in a rectilinear edge list, as the size of a run-length pair, or as a sharding codec's inner chunk size; and integral float edges in a rectilinear chunk grid (`[[4.0, 2]]`).
+This applies to stored metadata too wherever the stored-document repairs do not read it: a stored chunk shape that is a bare integer instead of a list, as in a Zarr format 2 `"chunks": 4` or a sharding codec's `"chunk_shape": 2`, which earlier releases read as a one-dimensional chunk shape, now raises a `TypeError` when the array is opened. A group whose consolidated metadata includes such an array does not open from its consolidated metadata at all, and the error names that member: open the group with `use_consolidated=False` to reach its other members. A stored `true` as a rectilinear chunk grid's bare step (`"chunk_shapes": [true]`) or as the count of a run-length pair (`[[4, true]]`), which no known writer stored, raises a `TypeError` too. The chunk sizes the repairs read open as before: 0 or `false` as a regular chunk size (in Zarr format 2 `chunks` or a `regular` grid's `chunk_shape`); `true` as a regular chunk size, as an edge in a rectilinear edge list, as the size of a run-length pair, or as a sharding codec's inner chunk size; and integral float edges in a rectilinear chunk grid (`[[4.0, 2]]`).
 
 Other than `chunks=False` or `shards=False` as the whole specification (one chunk spanning every axis), a boolean anywhere in a chunk specification given to the array creation functions (`chunks=True`, `np.True_`, `np.False_`, a boolean array, `chunks=(True, 5)`, `chunks=(False, 5)`, or a boolean edge in a list) raises one `TypeError`, instead of being read as a chunk size of 1 or 0 or raising a different error for each spelling. `chunks=True` and `create_array(chunks=None)` raised a `ValueError` before; both errors say how to ask for automatic chunking. The legacy `zarr.create(..., zarr_format=2)` no longer reads a falsy `chunks` as not given: `chunks=0`, `np.int64(0)` and `chunks=[]` raise a `ValueError`, and `chunks=False` means one chunk spanning the array, as it does for Zarr format 3, instead of automatic chunking; `chunks=None` still chunks automatically.
 

src/zarr/core/array.py

@@ -1597,28 +1597,28 @@ async def _save_metadata(self, metadata: ArrayMetadata, ensure_parents: bool = F
 
     def _stored_document_replaced(self) -> None:
         """Record that the store no longer holds a document of this array that needs an
-        upgrade: it holds the upgrade, a valid document, or none. The metadata this handle
+        repair: it holds the repair, a valid document, or none. The metadata this handle
         holds, which a consolidated group handle may share, then stops standing for the
-        document it was read from (see `mark_upgraded`), so no later write through either
+        document it was read from (see `mark_repaired`), so no later write through either
         handle stores that document again."""
         object.__setattr__(self.metadata, "_stored_document", None)
 
-    async def _store_upgraded_document(self) -> None:
-        """Store the upgrade of this array's current stored document, if it needs one,
+    async def _store_repaired_document(self) -> None:
+        """Store the repair of this array's current stored document, if it needs one,
         before chunks are written under this handle's metadata.
 
-        Only for metadata read from a document whose upgrade moves chunks (see
-        `zarr.core.metadata.upgrades`). The document is read again, because the store may
+        Only for metadata read from a document whose repair moves chunks (see
+        `zarr.core.metadata.repair`). The document is read again, because the store may
         hold a newer one than this handle's metadata. If that one lays out chunks
         differently (another writer stored a different chunk size since), this handle
         would write chunks no reader finds, so it raises and stores nothing. If it needs
-        no upgrade (the array was re-saved since, possibly by another implementation), it
-        is left as written; if there is none, there is nothing to upgrade.
+        no repair (the array was re-saved since, possibly by another implementation), it
+        is left as written; if there is none, there is nothing to repair.
 
-        Storing the same upgrade twice is harmless, so handles that write chunks
+        Storing the same repair twice is harmless, so handles that write chunks
         concurrently need no coordination. The read and the store are not one atomic
         step, though: a metadata write by another handle between them (a resize, an
-        attribute update) is replaced by the upgrade of the document read before it, as
+        attribute update) is replaced by the repair of the document read before it, as
         with any two metadata writes that race.
         """
         if self.metadata._stored_document is None:
@@ -1648,9 +1648,9 @@ async def _set_selection(
         fields: Fields | None = None,
     ) -> None:
         if product(indexer.shape) > 0:
-            # Chunks are about to be stored under the upgraded metadata, so store it
+            # Chunks are about to be stored under the repaired metadata, so store it
             # first: every reader of the store then agrees with them.
-            await self._store_upgraded_document()
+            await self._store_repaired_document()
         return await _set_selection(
             self.store_path,
             self.metadata,

src/zarr/core/common.py

@@ -293,7 +293,7 @@ def _subject(name: str, axis: int | None) -> str:
 def _parse_positive_int(value: object, name: str, axis: int | None) -> int:
     """`value` as an `int` of at least 1. A `bool`, a NumPy integer or a float (even an
     integral one: stored documents with integral floats are read by
-    `zarr.core.metadata.upgrades`) is rejected."""
+    `zarr.core.metadata.repair`) is rejected."""
     subject = _subject(name, axis)
     if isinstance(value, bool) or not isinstance(value, int):
         raise TypeError(f"{subject} must be an int, got {value!r}")

src/zarr/core/group.py

@@ -155,8 +155,8 @@ class ConsolidatedMetadata:
 
     def to_dict(self) -> dict[str, JSON]:
         """The consolidated metadata document. An array read from a stored document that
-        had to be upgraded is written as that document was stored, so every reader of
-        the consolidated metadata reads it as upgraded again (see `mark_upgraded`)."""
+        had to be repaired is written as that document was stored, so every reader of
+        the consolidated metadata reads it as repaired again (see `mark_repaired`)."""
         return {
             "kind": self.kind,
             "must_understand": self.must_understand,
@@ -394,7 +394,7 @@ class GroupMetadata(Metadata):
     def to_buffer_dict(self, prototype: BufferPrototype) -> dict[str, Buffer]:
         if self.consolidated_metadata is not None:
             for path, member in self.consolidated_metadata.flattened_metadata.items():
-                # A member read from a document that had to be upgraded is stored as it
+                # A member read from a document that had to be repaired is stored as it
                 # was stored (see `ConsolidatedMetadata.to_dict`).
                 if isinstance(member, ArrayV3Metadata) and member._stored_document is None:
                     try:

src/zarr/core/metadata/io.py

@@ -10,7 +10,7 @@
 from zarr.core.buffer.core import default_buffer_prototype
 from zarr.core.buffer.cpu import buffer_prototype as cpu_buffer_prototype
 from zarr.core.common import ZARR_JSON, ZARRAY_JSON, ZATTRS_JSON
-from zarr.core.metadata.upgrades import mark_upgraded, upgrade_array_document
+from zarr.core.metadata.repair import mark_repaired, repair_array_document
 from zarr.errors import ArrayNotFoundError, ContainsArrayError
 from zarr.storage._common import StorePath, ensure_no_existing_node
 
@@ -138,9 +138,9 @@ def parse_stored_array(
     documents: Mapping[str, Buffer], zarr_format: ZarrFormat, path: str | None = None
 ) -> ArrayMetadata:
     """The metadata of an array from its documents (by store key, see `ARRAY_DOCUMENTS`),
-    read with the upgrades but without their warnings (whoever asks has warned, or reads
-    metadata built in code), and marked (see `mark_upgraded`) if they had to be
-    upgraded. Raises `ArrayNotFoundError` if there is no array document among them.
+    read with the repairs but without their warnings (whoever asks has warned, or reads
+    metadata built in code), and marked (see `mark_repaired`) if they had to be
+    repaired. Raises `ArrayNotFoundError` if there is no array document among them.
 
     Only operations that store metadata read documents this way, so documents read as a
     rectilinear chunk grid require the rectilinear chunks flag, as storing it does; the
@@ -157,9 +157,9 @@ def parse_stored_array(
         stored = _array_metadata_dict_v3(documents[ZARR_JSON])
     else:
         raise ArrayNotFoundError(f"No Zarr format {zarr_format} array metadata document.")
-    upgraded, readings = upgrade_array_document(stored, zarr_format)
-    metadata = parse_array_metadata(dict(upgraded), path)
-    return mark_upgraded(metadata, stored, readings, None, warn=False)
+    repaired, readings = repair_array_document(stored, zarr_format)
+    metadata = parse_array_metadata(dict(repaired), path)
+    return mark_repaired(metadata, stored, readings, None, warn=False)
 
 
 def _build_parents(store_path: StorePath, zarr_format: ZarrFormat) -> dict[str, GroupMetadata]: