[bjyberg]

Target branch: develop

Summary

I was testing this library and found it was unable to read the consolidated metadata from zarr v3 written by xarray which place it inside zarr.json instead of .zmetadata where this function looks.

pizzarr::zarr_open("https://pub-45a1d62ac8d94c4c89f4dc63681a98ed.r2.dev/precipitation.zarr")
#> .zmetadata not found for this http store. Can't listdir
#> <ZarrGroup> /
#>   Store type  : HttpStore
#>   Zarr format : 3
#>   Read-only   : TRUE
#>   No. members : 0

This isn't in the core zarr v3 spec, but it is implemented by zarr-python and xarray by default, so I thought I would submit a PR to see if you are interested.

Alongside this, I found a related issue where contains_item returns FALSE for any chunk key on a consolidated store, since the index only lists metadata files:

z <- HttpStore$new("https://raw.githubusercontent.com/DOI-USGS/rnz/main/inst/extdata/bcsd.zarr")
length(z$get_item("pr/0.0.0"))
z$contains_item("pr/0.0.0")      
#> [1] 82010
#> [1] FALSE

I've kept these as 2 separate commits.

All of the changes are in R/stores.R + the added tests.

The main changes:

• When .zmetadata is missing, try to read consolidated_metadata from the root zarr.json. If it exists, it is reshaped to the same structure as .zmetadata to limit the number of downstream functions needing modified. If neither exist, if fails with the current behavior
• listdir() now skips any key without a / instead of keys starting with a . The old filter removed .zgroup but would have listed the root zarr.json as a member
• contains_item() was returning FALSE for any chunk key on a consolidated store, since the index only lists metadata files. It now only takes that shortcut for metadata keys and checks chunk keys over HTTP

Test plan

• New test in test-http-store.R against the xarray tutorial store, which has nested groups: root and nested listing, member count, chunk shape, attributes, get_item() byte round-trip, and contains_item() on a present and a missing chunk.
• Test suite passes locally with 0 failures
• Installed and verified against a 2 V3 stores with consolidated metadata

pizzarr::zarr_open("https://pub-45a1d62ac8d94c4c89f4dc63681a98ed.r2.dev/precipitation.zarr")
#> <ZarrGroup> /
#>   Store type  : HttpStore
#>   Zarr format : 3
#>   Read-only   : TRUE
#>   No. members : 3

pizzarr::zarr_open("https://huggingface.co/datasets/aaguilar90/chirps-v3-daily-rnl/resolve/main/chirps-v3-daily-rnl.zarr")
#> <ZarrGroup> /
#>   Store type  : HttpStore
#>   Zarr format : 3
#>   Read-only   : TRUE
#>   No. members : 5

# plus the other fix

z <- HttpStore$new("https://raw.githubusercontent.com/DOI-USGS/rnz/main/inst/extdata/bcsd.zarr")
length(z$get_item("pr/0.0.0"))
z$contains_item("pr/0.0.0")
#> [1] 82010
#> [1] TRUE