Skip to content

feat(storage): expose highWaterMark option for File stream operations - #9705

Open
thiyaguk09 wants to merge 3 commits into
googleapis:mainfrom
thiyaguk09:feat/file-upload-download-highwatermark
Open

thiyaguk09 wants to merge 3 commits into
googleapis:mainfrom
thiyaguk09:feat/file-upload-download-highwatermark

Conversation

@thiyaguk09

Copy link
Copy Markdown
Contributor

Description

Expose the highWaterMark option for File upload and download streams.

Impact

Users can tune stream buffer sizes to optimize throughput performance.

Changes

  • Add optional highWaterMark?: number with TSDoc/JSDoc documentation to CreateReadStreamOptions and CreateWriteStreamOptions in src/file.ts, as well as MakeWritableStreamOptions in src/nodejs-common/util.ts.
  • Extract highWaterMark in File#createReadStream and pass it to the underlying PassThroughShim stream returned to callers.
  • Extract highWaterMark in File#createWriteStream and propagate it to the outer Writable, PassThroughShim, and duplexify streams, as well as to both resumable (resumableUpload.upload) and simple (File#startSimpleUpload_ / util.makeWritableStream) upload streams.

Testing

  • Add unit test in test/file.ts verifying File#createReadStream sets readableHighWaterMark when highWaterMark is provided.
  • Add unit tests in test/file.ts verifying File#createWriteStream sets writableHighWaterMark and propagates highWaterMark to both resumable and simple upload streams.
  • Add unit tests in test/file.ts and test/nodejs-common/util.ts verifying highWaterMark propagation in startResumableUpload_, startSimpleUpload_, and util.makeWritableStream.

@product-auto-label product-auto-label Bot added the api: storage Issues related to the Cloud Storage API. label Oct 23, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for configuring the stream highWaterMark (in bytes) for both readable and writable streams. It adds the highWaterMark option to CreateWriteStreamOptions, CreateReadStreamOptions, and MakeWritableStreamOptions, ensuring it is correctly propagated to underlying streams, including PassThroughShim, Writable, duplexify, and ProgressStream, as well as resumable and simple uploads. Comprehensive unit tests have been added to verify this propagation. There are no review comments, and I have no feedback to provide.

@thiyaguk09
thiyaguk09 marked this pull request as ready for review October 23, 2026 09:54
@thiyaguk09
thiyaguk09 requested review from a team as code owners October 23, 2026 09:54

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: storage Issues related to the Cloud Storage API.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant