Skip to content

Commit b028ea6

Browse files
dblodgett-usgsdblodgett-usgs
authored
Merge pull request #187 from zarr-developers/build
add s3, gcs, and blosc to default RUST build - fixes #185
2 parents 3c8963a + dcb8724 commit b028ea6

3 files changed

Lines changed: 34 additions & 35 deletions

File tree

‎README.Rmd‎

Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -52,9 +52,9 @@ pre-compiled binaries for Windows and macOS — no Rust toolchain needed.
5252

5353
The r-universe binaries compile the zarrs backend with local filesystem I/O,
5454
synchronous HTTP reads, gzip/zstd/blosc codecs, sharding, and S3/GCS cloud
55-
store support via `object_store`. The build system enables these features
56-
automatically when `NOT_CRAN` is set (which r-universe does). To compile with
57-
a different feature set, set `PIZZARR_FEATURES` explicitly (see below).
55+
store support via `object_store`. Any build that compiles Rust enables these
56+
features by default. To compile with a different feature set, set
57+
`PIZZARR_FEATURES` explicitly (see below).
5858

5959
`pizzarr_compiled_features()` lists what the zarrs backend provides, and
6060
`pizzarr_upgrade()` prints the install command when zarrs is not compiled in.
@@ -80,10 +80,10 @@ The build system uses two environment variables to control compilation:
8080
skips link-time optimization and does not strip symbols. Use this during
8181
active development on the Rust code.
8282
- **`PIZZARR_FEATURES`** — a comma-separated list of extra Cargo features.
83-
When `NOT_CRAN` is set and `PIZZARR_FEATURES` is empty, the build defaults
84-
to `s3,gcs` (S3 and GCS cloud store support via `object_store` and `tokio`).
85-
Set this explicitly to override — e.g., `PIZZARR_FEATURES=none` for default
86-
features only, or `PIZZARR_FEATURES=s3` for S3 without GCS.
83+
When unset, the build defaults to `s3,gcs` (S3 and GCS cloud store support
84+
via `object_store` and `tokio`, which also pulls in blosc and sharding).
85+
Set this explicitly to override — e.g., `PIZZARR_FEATURES=none` for Cargo's
86+
default features only, or `PIZZARR_FEATURES=s3` for S3 without GCS.
8787

8888
A release build (no `DEBUG`) uses LTO, single codegen unit, and symbol
8989
stripping — fast at runtime but takes several minutes to compile from scratch.
@@ -93,15 +93,15 @@ stripping — fast at runtime but takes several minutes to compile from scratch.
9393
DEBUG=1 R CMD INSTALL .
9494

9595
# Production: slow first build, fast runtime, S3/GCS included
96-
NOT_CRAN=true R CMD INSTALL .
97-
98-
# Production with default features only (no cloud stores)
9996
R CMD INSTALL .
97+
98+
# Production with Cargo default features only (no cloud stores)
99+
PIZZARR_FEATURES=none R CMD INSTALL .
100100
```
101101

102-
`NOT_CRAN` is set automatically when `DEBUG` is present. For builds without
103-
vendored crates (the normal case for local development), `NOT_CRAN` prevents
104-
the build system from attempting offline compilation.
102+
`NOT_CRAN=true` opts builds out of the offline-vendored-crates path used for
103+
CRAN; it's set automatically when `DEBUG` is present and only matters when
104+
`src/rust/vendor.tar.xz` is also present.
105105

106106
Development happens on the `develop` branch. See
107107
[CONTRIBUTING.md](https://github.com/zarr-developers/pizzarr/blob/main/CONTRIBUTING.md)

‎README.md‎

Lines changed: 13 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -41,10 +41,9 @@ toolchain needed.
4141

4242
The r-universe binaries compile the zarrs backend with local
4343
filesystem I/O, synchronous HTTP reads, gzip/zstd/blosc codecs,
44-
sharding, and S3/GCS cloud store support via `object_store`. The
45-
build system enables these features automatically when `NOT_CRAN` is
46-
set (which r-universe does). To compile with a different feature set,
47-
set `PIZZARR_FEATURES` explicitly (see below).
44+
sharding, and S3/GCS cloud store support via `object_store`. Any build
45+
that compiles Rust enables these features by default. To compile with
46+
a different feature set, set `PIZZARR_FEATURES` explicitly (see below).
4847

4948
`pizzarr_compiled_features()` lists what the zarrs backend provides,
5049
and `pizzarr_upgrade()` prints the install command when zarrs is not
@@ -71,10 +70,10 @@ The build system uses two environment variables to control compilation:
7170
runtime because it skips link-time optimization and does not strip
7271
symbols. Use this during active development on the Rust code.
7372
- **`PIZZARR_FEATURES`** — a comma-separated list of extra Cargo
74-
features. When `NOT_CRAN` is set and `PIZZARR_FEATURES` is empty,
75-
the build defaults to `s3,gcs` (S3 and GCS cloud store support via
76-
`object_store` and `tokio`). Set this explicitly to override — e.g.,
77-
`PIZZARR_FEATURES=none` for default features only, or
73+
features. When unset, the build defaults to `s3,gcs` (S3 and GCS
74+
cloud store support via `object_store` and `tokio`, which also pulls
75+
in blosc and sharding). Set this explicitly to override — e.g.,
76+
`PIZZARR_FEATURES=none` for Cargo's default features only, or
7877
`PIZZARR_FEATURES=s3` for S3 without GCS.
7978

8079
A release build (no `DEBUG`) uses LTO, single codegen unit, and symbol
@@ -86,16 +85,15 @@ scratch.
8685
DEBUG=1 R CMD INSTALL .
8786

8887
# Production: slow first build, fast runtime, S3/GCS included
89-
NOT_CRAN=true R CMD INSTALL .
90-
91-
# Production with default features only (no cloud stores)
9288
R CMD INSTALL .
89+
90+
# Production with Cargo default features only (no cloud stores)
91+
PIZZARR_FEATURES=none R CMD INSTALL .
9392
```
9493

95-
`NOT_CRAN` is set automatically when `DEBUG` is present. For builds
96-
without vendored crates (the normal case for local development),
97-
`NOT_CRAN` prevents the build system from attempting offline
98-
compilation.
94+
`NOT_CRAN=true` opts builds out of the offline-vendored-crates path
95+
used for CRAN; it's set automatically when `DEBUG` is present and only
96+
matters when `src/rust/vendor.tar.xz` is also present.
9997

10098
Development happens on the `develop` branch. See
10199
[CONTRIBUTING.md](https://github.com/zarr-developers/pizzarr/blob/main/CONTRIBUTING.md)

‎tools/config.R‎

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -71,16 +71,17 @@ if (!is_not_cran) {
7171
# when DEBUG env var is present we use `--debug` build
7272
.profile <- ifelse(is_debug, "", "--release")
7373

74-
# PIZZARR_FEATURES env var enables extra Cargo features (e.g., "s3,gcs").
75-
# When NOT_CRAN is set (r-universe, DEBUG) and PIZZARR_FEATURES is empty,
76-
# default to "s3,gcs" so r-universe binaries ship with cloud store support.
77-
if (!nzchar(env_features) && is_not_cran) {
74+
# PIZZARR_FEATURES env var enables extra Cargo features. When unset, default
75+
# to "s3,gcs" so every Rust build ships with cloud store + blosc + sharding
76+
# support. The CRAN tier is the pure-R tarball produced by tools/cran-build.sh
77+
# (src/ stripped), so this default never reaches CRAN — it only affects builds
78+
# that actually compile Rust (r-universe binaries, local source installs).
79+
if (!nzchar(env_features)) {
7880
env_features <- "s3,gcs"
79-
message("Defaulting PIZZARR_FEATURES to '", env_features, "' (NOT_CRAN build).")
81+
message("Defaulting PIZZARR_FEATURES to '", env_features, "'.")
8082
}
8183

82-
# "none" is an escape hatch: suppresses the NOT_CRAN default without
83-
# passing any extra features to Cargo.
84+
# "none" is an escape hatch: build with Cargo's default features only.
8485
if (identical(env_features, "none")) {
8586
env_features <- ""
8687
message("PIZZARR_FEATURES='none': using default Cargo features only.")

0 commit comments

Comments
 (0)