You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
we now have a few subpackages that spin out core functionality into self-contained python packages:
zarr-indexing handles lazy, chunked array indexing. It's a drop-in replacement for indexing.py in zarr, and it supports lazy indexing.
zarr-metadata handles the type-safe representation of zarr v2 and v3 metadata documents. it can add a missing layer to our metadata handling and ultimately replace our ArrayMetadataV3 and ArrayMetadataV2 classes.
zarr-chunk-key-encodings is unreleased but proposed in this PR. It would be a drop-in replacement for our chunk_key_encodings.py
There are more subpackages I want to make, namely something for stores, codecs and data types. But before I start those, we should sketch out how we bring any subpackage into zarr.
here's my current plan:
we first depend on the subpackage in our test suite
we then bring drop-in replacements, if any, that package provides to runtime
new features (like lazy indexing) are gated behind a feature flag
with the perspective from 1-3 we can plan out major changes (like sunsetting ArrayV3Metadata).
@zarr-developers/python-core-devs let me know if this direction makes sense. I'm also happy to explain why we are doing this. If nobody objects, I will open PRs soon with step 1 for the existing subpackages.
Activity