Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 23 additions & 10 deletions core/packages/gaxios/src/gaxios.ts
Original file line number Diff line number Diff line change
Expand Up @@ -728,6 +728,8 @@ export class Gaxios implements FetchCompliance {
proxy?: string;
cert?: string;
key?: string;
size?: number;
maxContentLength?: number;
Comment on lines +731 to +732

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Generated] Declare size and maxContentLength on rawInit in #createBunFetch() so the Bun fetch wrapper can read the configured byte limit.

tls?: {cert?: string; key?: string};
})
: undefined;
Expand Down Expand Up @@ -807,6 +809,7 @@ export class Gaxios implements FetchCompliance {
}
throw err;
}
const maxSize = rawInit?.size ?? rawInit?.maxContentLength ?? 0;

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Generated] Compute maxSize from rawInit.size or rawInit.maxContentLength (defaulting to 0 when unset) to determine whether response stream byte-limiting is needed.

if (
res?.body &&
typeof Readable.fromWeb === 'function' &&
Expand All @@ -820,13 +823,23 @@ export class Gaxios implements FetchCompliance {
const origArrayBuffer = res.arrayBuffer.bind(res);
const origBlob = res.blob.bind(res);
const readBuffer = async () => {
const chunks: Buffer[] = [];
for await (const chunk of nodeStream!) {
chunks.push(
Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk as Uint8Array),
);
nodeStream ||= Readable.fromWeb(rawBody);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Generated] Lazily initialize nodeStream from Readable.fromWeb(rawBody) inside readBuffer() so buffered readers (text(), json(), arrayBuffer(), blob()) can stream and measure the body even when res.body has not been accessed yet.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

out of date

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like a genuine bug. I notice that nodeStream wasn't even assigned before!

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re: your comment on why nodeStream wasn't assigned in readBuffer() previously: in main, nodeStream was only initialized inside the res.body getter because res.text(), res.json(), res.arrayBuffer(), and res.blob() all started with if (!nodeStream) return origText(). That meant readBuffer() was previously only reachable if the caller had already accessed res.body (which locked rawBody via Readable.fromWeb(rawBody)).

Now that those guards check if (!nodeStream && maxSize <= 0), readBuffer() runs whenever maxSize > 0 even if res.body was never touched, so lazily initializing nodeStream ||= Readable.fromWeb(rawBody) here is required.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is good to know. Thank you!

const chunks: Uint8Array[] = [];
let totalBytes = 0;
for await (const chunk of nodeStream) {
const bytes = chunk as Uint8Array;
totalBytes += bytes.byteLength;
if (maxSize > 0 && totalBytes > maxSize) {
throw Object.assign(
new Error(
`content size at ${String((input as {url?: unknown})?.url ?? input)} over limit: ${maxSize}`,
),
{type: 'max-size'},
);
}
chunks.push(bytes);
}
return Buffer.concat(chunks);
return Buffer.concat(chunks, totalBytes);
};
Object.defineProperty(res, 'body', {
get() {
Expand All @@ -837,23 +850,23 @@ export class Gaxios implements FetchCompliance {
enumerable: true,
});
res.text = async () => {
if (!nodeStream) return origText();
if (!nodeStream && maxSize <= 0) return origText();

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Generated] Route res.text() through readBuffer() whenever maxSize > 0 so headerless/chunked text responses enforce maxContentLength instead of bypassing the stream limiter via origText().

return (await readBuffer()).toString('utf8');
};
res.json = async () => {
if (!nodeStream) return origJson();
if (!nodeStream && maxSize <= 0) return origJson();

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Generated] Route res.json() through res.text() / readBuffer() whenever maxSize > 0 so headerless/chunked JSON responses enforce maxContentLength.

return JSON.parse(await res.text());
};
res.arrayBuffer = async () => {
if (!nodeStream) return origArrayBuffer();
if (!nodeStream && maxSize <= 0) return origArrayBuffer();

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Generated] Route res.arrayBuffer() through readBuffer() whenever maxSize > 0 so headerless/chunked binary responses enforce maxContentLength.

const buf = await readBuffer();
return buf.buffer.slice(
buf.byteOffset,
buf.byteOffset + buf.byteLength,
);
};
res.blob = async () => {
if (!nodeStream) return origBlob();
if (!nodeStream && maxSize <= 0) return origBlob();

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Generated] Route res.blob() through readBuffer() whenever maxSize > 0 so headerless/chunked Blob responses enforce maxContentLength.

const buf = await readBuffer();
const contentType = res.headers?.get?.('content-type') ?? '';
return new Blob([buf], contentType ? {type: contentType} : undefined);
Expand Down
28 changes: 28 additions & 0 deletions core/packages/gaxios/test/test.getch.ts
Original file line number Diff line number Diff line change
Expand Up @@ -343,6 +343,28 @@ describe('🥁 configuration options', () => {
scope.done();
});

it('should enforce maxContentLength on responses without a Content-Length header', async () => {
const body = {hello: 'world'};
for (const responseType of [
'json',
'text',
'arraybuffer',
'blob',
] as const) {
// nock's .reply(200, body) omits Content-Length unless .replyContentLength()
// or an explicit 'content-length' header is specified, so this exercises
// the stream byte-counting limit ('over limit: 5') rather than the upfront
// Content-Length header check ("Response's `Content-Length` is over the limit.").
const scope = nock(url).get('/').reply(200, body);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

By default, nock automatically calculates and includes a Content-Length header when replying with a body. This means the test might be passing due to the pre-existing Content-Length validation rather than actually exercising the new stream-limiting logic for headerless responses.

To ensure the response is truly headerless and the stream limiter is exercised, explicitly set the transfer-encoding header to chunked in the nock reply.

      const scope = nock(url)
        .get('/')
        .reply(200, body, {'transfer-encoding': 'chunked'});

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note on the gemini-code-assist comment here: nock does not add a Content-Length header unless .replyContentLength() or an explicit 'content-length' header is specified (by default .reply(200, body) only sets {'content-type': 'application/json'}), so this test already exercises the headerless response path as written.

@danieljbruce danieljbruce Oct 30, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Generated] Thank you for pointing that out! To make it unambiguous in the test itself that the headerless stream byte-counting path is exercised rather than the upfront Content-Length header check ("Response's Content-Length is over the limit."), I tightened the assertion in eb5273f to match /over limit: 5/.test(err.message) and added a brief comment noting nock's default .reply(200, body) behavior.

await assert.rejects(
request({url, maxContentLength: 5, responseType}),
(err: Error) =>
err instanceof GaxiosError && /over limit: 5/.test(err.message),
);
scope.done();
}
});

Comment on lines +346 to +367

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Generated] Add unit test verifying that maxContentLength rejects responses without a Content-Length header across json, text, arraybuffer, and blob response types.

it('should support redirects by default', async () => {
const body = {hello: '🌎'};
const scopes = [
Expand Down Expand Up @@ -658,6 +680,12 @@ describe('🥁 configuration options', () => {
const jsonRes = await request({url});
assert.deepStrictEqual(jsonRes.data, responseBody);

await assert.rejects(
request({url, maxContentLength: 2}),
(err: Error) =>
err instanceof GaxiosError && /over limit: 2/.test(err.message),
);

Comment on lines +683 to +688

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Generated] Add unit assertions in the Bun fetch test verifying that maxContentLength rejects oversized responses with over limit: 2 for both buffered JSON and responseType: 'stream'.

globalThis.fetch = (async () => {
throw {code: 'ECONNRESET', message: 'socket hang up'};
}) as typeof fetch;
Expand Down
Loading