Skip to content

Latest commit

 

History

History
25 lines (23 loc) · 1.54 KB

File metadata and controls

25 lines (23 loc) · 1.54 KB

Added core support for URL pipelines (https://github.com/jbms/url-pipeline): |-chained URLs that address zarr data through nested storage layers, e.g. s3://bucket/data.zip|zip:|zarr3:. This PR adds the parser, the single-method zarr.abc.url_pipeline.URLPipelineAdapter interface, and the zarr.url_adapters entry-point group through which third-party packages (e.g. Icechunk) register adapters for their own schemes. Adapters for a scheme are loaded lazily and individually. Builtin adapters (zip:, zarr2:/zarr3:) follow in separate pull requests.

Behavior notes:

  • The | character is now reserved as the pipeline delimiter in every string store specification, and no percent-escape is decoded; pass a pathlib.Path to address a local file whose name contains |. URLs without a | (and without a registered root adapter scheme) are handled exactly as before — registered adapters cannot intercept zarr's native file:/memory: routing, and fsspec chained URLs (zip::s3://...) keep flowing to fsspec.
  • Inside a pipeline, memory: and file: roots follow the URL pipeline spec's semantics (spelling equivalences; file: must be absolute, with at most a localhost authority).
  • Mode "a" (open-or-create, the zarr.open default) on a read-only store now serves the "open" half instead of raising upfront, for all stores; unambiguous write modes ("w", "w-", "r+") still raise.
  • For root-adapter URLs (e.g. gh://org/repo), storage_options are handed to the adapter and are not validated as used by make_store.