Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
2df6038
docs(storage): add gaxios migration guide
thiyaguk09 Oct 5, 2026
ab86096
chore: remove airlock-proxy from lockfile, pin public npm registry, a…
thiyaguk09 Oct 6, 2026
daa05b9
Merge branch 'storage-gaxios-migration' into docs/gaxios-migration-guide
thiyaguk09 Oct 6, 2026
7381ca5
Merge branch 'storage-gaxios-migration' into docs/gaxios-migration-guide
thiyaguk09 Oct 6, 2026
d2a4a29
Merge branch 'storage-gaxios-migration' into docs/gaxios-migration-guide
thiyaguk09 Oct 6, 2026
25948f6
Merge branch 'storage-gaxios-migration' into docs/gaxios-migration-guide
thiyaguk09 Oct 8, 2026
69b68ea
Merge branch 'storage-gaxios-migration' into docs/gaxios-migration-guide
thiyaguk09 Oct 9, 2026
808521e
Merge branch 'storage-gaxios-migration' into docs/gaxios-migration-guide
thiyaguk09 Oct 12, 2026
2cf5e6f
Merge branch 'storage-gaxios-migration' into docs/gaxios-migration-guide
thiyaguk09 Oct 13, 2026
b741cff
Merge branch 'storage-gaxios-migration' into docs/gaxios-migration-guide
thiyaguk09 Oct 13, 2026
52e183c
Merge branch 'storage-gaxios-migration' into docs/gaxios-migration-guide
thiyaguk09 Oct 14, 2026
2f50807
Merge branch 'storage-gaxios-migration' into docs/gaxios-migration-guide
thiyaguk09 Oct 15, 2026
5b33396
Merge branch 'storage-gaxios-migration' into docs/gaxios-migration-guide
thiyaguk09 Oct 19, 2026
8b80157
Merge branch 'storage-gaxios-migration' into docs/gaxios-migration-guide
thiyaguk09 Oct 20, 2026
04486f2
Merge branch 'storage-gaxios-migration' into docs/gaxios-migration-guide
thiyaguk09 Oct 22, 2026
6a619e5
Merge branch 'storage-gaxios-migration' into docs/gaxios-migration-guide
thiyaguk09 Oct 23, 2026
f1076fb
Merge branch 'storage-gaxios-migration' into docs/gaxios-migration-guide
thiyaguk09 Oct 26, 2026
4cebaeb
Merge branch 'storage-gaxios-migration' into docs/gaxios-migration-guide
thiyaguk09 Oct 30, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .npmrc
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
link-workspace-packages=false
enable-pre-post-scripts=true
registry=https://registry.npmjs.org/
239 changes: 239 additions & 0 deletions handwritten/storage/MIGRATION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,239 @@
# Storage Migration Guide - Gaxios (`StorageTransport`) Update

This guide helps you migrate your `@google-cloud/storage` application through the HTTP transport update—replacing `teeny-request`, `retry-request`, `form-data`, and `node-fetch` with **`gaxios` v7 (`StorageTransport`)** and native Node.js Web APIs (`Headers`, `URL`, `FormData`, `Blob`, `fetch`).

---

## Key Breaking Changes for Storage Users

### 1. Response Headers are now `Headers` objects

When you receive a full API response from Storage methods (e.g., via callbacks or promise resolutions that include the response object), the `headers` property of the response (`GaxiosResponse['headers']`) is now a standard Web `Headers` instance rather than a plain JavaScript object.

**Before (`teeny-request`):**

```js
const [retrievedFile, apiResponse] = await file.get();
const contentType = apiResponse.headers['content-type'];
```

**After (`Gaxios` / `StorageTransport`):**

```js
const [retrievedFile, apiResponse] = await file.get();
// Accessing headers requires the .get() method
const contentType = apiResponse.headers.get('content-type');
```

---

### 2. Passing and Reading Request Headers in Options

If you pass custom headers in options to Storage methods (which extend `GaxiosOptions`), you can still pass plain objects, as the Storage library will convert them to standard `Headers` internally for the request. However, if you read them back from the prepared options or response (`apiResponse.config.headers`), they will be `Headers` objects.

**Before (`teeny-request`):**

```js
// Reading request headers back from response metadata returned a plain object
const customHeader = apiResponse.config.headers['x-custom-header'];
```

**After (`Gaxios` / `StorageTransport`):**

```js
// Reading request headers back from response metadata requires .get()
const customHeader = apiResponse.config.headers.get('x-custom-header');
```

> [!WARNING]
> **Header Value Stringification:** Plain JavaScript objects allow passing non-string values (such as arrays or numbers) which are implicitly processed. However, the native `Headers` constructor strictly converts all values to standard string representations. For example, passing an array of values (e.g., `['val1', 'val2']`) will result in a single comma-separated string (e.g., `'val1, val2'`). Ensure you pre-format or verify your header values before passing them to custom options.

---

### 3. URL Resolution (`baseURL` and Native `URL` Objects)

If you are using custom `baseURL` options or passing relative URLs to methods that accept them, resolution now strictly follows the standard native `URL` constructor spec (`new URL(url, baseURL)`). Additionally, `config.url` on prepared request configurations (`GaxiosOptionsPrepared`) is now a native `URL` instance instead of a string.

> [!IMPORTANT]
> **Relative Path Normalization in `StorageTransport`:** `StorageTransport.makeRequest()` automatically normalizes non-absolute URLs to begin with a leading `/` before calling `new URL(normalizedUrl, this.baseUrl)`. Because a leading slash resolves relative to the host root, any relative URL passed to `makeRequest()` against a `baseUrl` containing `/storage/v1` must explicitly include the `/storage/v1` prefix (e.g., `url: '/storage/v1/b/my-bucket'`).

**Before (`teeny-request` / `Service`):**

Using custom path-joining resolution:

- `baseURL`: `https://storage.googleapis.com/storage/v1`
- `url`: `/b/my-bucket` (or `b/my-bucket`)
- Resolved URL: `https://storage.googleapis.com/storage/v1/b/my-bucket`

**After (`Gaxios` / `StorageTransport`):**

Strictly resolved via the standard native `URL` constructor rules (where a leading slash resolves relative to the root of the host):

- `baseURL`: `https://storage.googleapis.com/storage/v1`
- `url`: `/b/my-bucket` (or `b/my-bucket`, which is normalized to `/b/my-bucket`)
- Resolved URL: `https://storage.googleapis.com/b/my-bucket` (resolves relative to host root, stripping `storage/v1`)
- To preserve `/storage/v1`, pass `url`: `/storage/v1/b/my-bucket`

---

### 4. Request Interceptors (`interceptors_` / `interceptors`)

The legacy `teeny-request` interceptor format (`{ request: (reqOpts) => reqOpts }`) has been replaced by Gaxios request interceptors (`GaxiosInterceptor<GaxiosOptionsPrepared>`).

**Before (`teeny-request`):**

```js
const storage = new Storage({
interceptors_: [
{
request: reqOpts => {
reqOpts.headers['x-custom'] = 'value';
return reqOpts;
},
},
],
});
```

**After (`Gaxios` / `StorageTransport`):**

```js
const storage = new Storage({
interceptors_: [
{
resolved: async config => {
config.headers.set('x-custom', 'value');
return config;
},
},
],
});
```

---

### 5. Error Objects & Custom Retry Functions (`retryOptions.retryableErrorFn`)

The legacy `retry-request` library and `ApiError` class have been replaced by Gaxios's retry mechanism and `GaxiosError`.

- **Error structure in `retryableErrorFn` vs. final rejected errors**:
- **Inside `retryOptions.retryableErrorFn(err)`**: `err` is the raw `GaxiosError` evaluated by Gaxios's `shouldRetry` hook *before* `StorageTransport` decorates it. HTTP status is at `err.response?.status` (or `err.status` / `err.code`), the parsed error payload is at `err.response?.data` (`err.response?.data?.error?.errors`), and the request configuration is at `err.config`. Top-level `err.errors` is not yet populated at this stage.
- **Final rejected / callback errors**: Once retries are exhausted, `StorageTransport` decorates the `GaxiosError` for backward compatibility by copying `err.code` (`err.response?.status || err.status || err.code`), `err.message`, and `err.errors` (`err.response?.data?.error?.errors`) onto the top-level error instance.
- **Idempotency requirement in default retry function**: `RETRYABLE_ERR_FN_DEFAULT(err)` now inspects `err.config` (`isRequestIdempotent(err.config) && isTransientError(err)`). Calling `RETRYABLE_ERR_FN_DEFAULT(err)` without an `err.config` object will return `false`.
- **Retry configuration mapping**: `StorageOptions.retryOptions` (`maxRetries`, `retryDelayMultiplier`, `maxRetryDelay`, `totalTimeout`, `retryableErrorFn`) maps directly to Gaxios `retryConfig` (`retry`, `noResponseRetries`, `retryDelayMultiplier`, `maxRetryDelay`, `totalTimeout`, `shouldRetry`).

**Before (`retry-request` / `ApiError`):**

```js
const storage = new Storage({
retryOptions: {
retryableErrorFn: err => {
// Legacy ApiError used err.code and err.errors
return err.code === 502 || err.errors?.some(e => e.reason === 'rateLimitExceeded');
},
},
});
```

**After (`Gaxios` / `GaxiosError`):**

```js
const storage = new Storage({
retryOptions: {
retryableErrorFn: err => {
// Raw GaxiosError in shouldRetry provides err.response?.status, err.response?.data, and err.config
const status = err.response?.status;
const errors = err.response?.data?.error?.errors || [];
return status === 502 || errors.some(e => e.reason === 'rateLimitExceeded');
},
},
});
```

---

### 6. Multipart Uploads & `FormData` (`form-data` Dependency Removed)

The external `form-data` and `node-fetch` dependencies have been removed in favor of Node.js native Web APIs (`FormData`, `Blob`, `fetch`) and Gaxios multipart options (`GaxiosMultipartOptions`).

1. **V4 Signed POST Policy Uploads (`generateSignedPostPolicyV4`)**:
Use global `FormData` and `Blob` with native `fetch` instead of the `form-data` npm package.

**Before (`form-data` / `node-fetch`):**

```js
const FormData = require('form-data');
const fetch = require('node-fetch');

const [policy] = await file.generateSignedPostPolicyV4(options);
const form = new FormData();
for (const [k, v] of Object.entries(policy.fields)) form.append(k, v);
form.append('file', Buffer.from('my-content'));
await fetch(policy.url, {method: 'POST', body: form, headers: form.getHeaders()});
```

**After (Native Web `FormData` / `fetch`):**

```js
const [policy] = await file.generateSignedPostPolicyV4(options);
const form = new FormData();
for (const [k, v] of Object.entries(policy.fields)) form.append(k, v);
form.append('file', new Blob(['my-content']), file.name);
await fetch(policy.url, {method: 'POST', body: form});
```

2. **Low-Level `multipart` Request Option (`StorageRequestOptions.multipart`)**:
If calling `storageTransport.makeRequest` directly with `multipart`, each part must use `{ headers, content }` (`GaxiosMultipartOptions`) instead of `{ 'Content-Type', body }`. (While `StorageTransport` accepts either a `Headers` instance or a plain object for `part.headers` at runtime, using `new Headers(...)` is required for TypeScript compatibility with `GaxiosMultipartOptions`.)

**Before (`teeny-request`):**

```js
multipart: [
{'Content-Type': 'application/json', body: JSON.stringify(metadata)},
{'Content-Type': 'text/plain', body: contentStream},
]
```

**After (`Gaxios` / `StorageTransport`):**

```js
multipart: [
{headers: new Headers({'Content-Type': 'application/json'}), content: JSON.stringify(metadata)},
{headers: new Headers({'Content-Type': 'text/plain'}), content: contentStream},
]
```

---

### 7. Custom HTTP Agents (`http.Agent` / `https.Agent`) & Proxies

Legacy `teeny-request` connection pool options (`forever: true`, `pool: { maxSockets }`) are no longer used. Custom HTTP/HTTPS agents and proxies are now configured through Gaxios options:

- **Environment variables**: `HTTPS_PROXY`, `HTTP_PROXY`, and `NO_PROXY` are automatically respected by Gaxios.
- **Per-request or custom transport**: Pass `agent` (`http.Agent` / `https.Agent`), `proxy` (`string | URL`), or `noProxy` (`(string | URL | RegExp)[]`) in `StorageRequestOptions`, or provide a pre-configured `gaxiosInstance` (`new Gaxios({ agent, proxy, noProxy })`) to `StorageTransport`.

---

### 8. Low-Level Request Option Renames (`Service` / `ServiceObject` -> `StorageTransport`)

The internal `Service` class (`src/nodejs-common/service.ts`) has been removed and replaced by `StorageTransport`. If your integration calls low-level `request()` or `storageTransport.makeRequest()` directly:

- `uri` is now `url`.
- `qs` is now `queryParameters` (or `params`).
- `json` request body objects should now be passed via `body` (serialized JSON string) or `data`.

---

## Upgrade Instructions

Update your `@google-cloud/storage` dependency to the latest release:

```sh
npm install @google-cloud/storage@latest
```

## Troubleshooting

- **`TypeError: apiResponse.headers.get is not a function` or `undefined` header values**: Ensure you use `apiResponse.headers.get('header-name')` instead of bracket notation (`apiResponse.headers['header-name']`).
- **Custom `retryableErrorFn` not triggering**: Check `err.response?.status` instead of `err.code` for HTTP status codes, and ensure `err.config` is preserved when delegating to `RETRYABLE_ERR_FN_DEFAULT(err)`.
- **Signed POST Policy (`FormData`) upload errors**: Ensure you pass a native `FormData` instance without manually overriding the `Content-Type` header so `fetch` / `Gaxios` can append the multipart boundary automatically.
Loading