Skip to content

fix(cli): resume a cloud download part when its connection drops - #5873

Merged
miguel-heygen merged 1 commit into
mainfrom
hfcloud/download-resume-range
Oct 19, 2026
Merged

miguel-heygen merged 1 commit into
mainfrom
hfcloud/download-resume-range

Conversation

@miguel-heygen

Copy link
Copy Markdown
Collaborator

What

hyperframes cloud render / cloud get downloads the output in parallel ranged parts. Until now one part losing its connection failed the whole download. Now:

  • A ranged part whose connection drops re-requests only the bytes it is still missing (bytes=<next>-<end>), up to 3 retries with 0.25 / 0.5 / 1 s backoff. After that it fails with Failed to download <url>: bytes <start>-<end> dropped 4 times (the last error is the cause). The previous output is kept and no staged file is left behind, as before.
  • A server that ignores Range (200, no Content-Range) still fails fast. Without ranges there is nothing to resume from, and retrying from zero would redownload the whole file.
  • A user cancel or a sibling part's failure still stops everything at once: retries and backoff waits listen to the same abort signal.

Why the test failed on Windows main

splits a large file into parallel ranged requests failed on the Windows lane with TypeError: terminated, caused by SocketError: other side closed, after 16.8 s. The socket info in the error shows one request written (205 bytes) and one complete response read (7,789,629 bytes = headers + the 7,789,435-byte last part). So the server closed a connection after sending everything, while the client was still writing that body to disk.

Inference (not reproduced on macOS or Linux): the Node test server's keep-alive idle timer (5 s by default) closed the socket. Windows completes a large loopback write at once, so the idle clock starts right away, while the test took 16.8 s under the parallel suite. On macOS and Linux the write only finishes as the reader drains, so the timer never wins.

Fix at the cause: the test server sets keepAliveTimeout = 0, so it never closes a socket during a test (server.close() still closes idle sockets at teardown). The client fix means a real server closing a connection early is now survived as well.

Tests

  • New: resumes a range whose connection drops mid-body from the bytes it already has. The server cuts the range covering one byte twice; the download completes byte-identical and the resume request starts at the cut byte.
  • Changed: keeps the previous output and leaves no staged file when one part keeps dropping. The range always drops, the error says dropped 4 times, and the old output survives.
  • Against main's download.ts both fail with the same terminated / other side closed as the Windows failure. With the fix, the download suite passes 17/17, 3 runs in a row (Linux).

A ranged part that loses its connection now re-requests only the bytes it is missing, up to 3 retries with backoff, then fails naming the byte range. A server that ignores Range still fails fast: there is nothing to resume from.

The test server no longer closes idle keep-alive sockets, which closed a socket under a slow reader on Windows.

@jrusso1020 jrusso1020 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving at 8559dd09.

  • Resume is byte-exact. pump writes each chunk at part.at and then advances it, so a retry asks for bytes=<part.at>-<end> and never rewrites or double-counts bytes it already has (bytes/onProgress stay right). A body that ends short throws connection closed at byte N and resumes from there.
  • Cancel and sibling failure still stop everything. if (signal.aborted) throw error and sleep(..., { signal }) both use the combined failFast + user signal. The catch on downloadToFile still rethrows the user's own abort reason.
  • No-Range fallback keeps the old fail-fast truncation check unchanged.
  • The test fix matches the evidence. The error recorded one complete 7,789,629-byte response before the socket closed, which fits the 5 s keep-alive idle timer. keepAliveTimeout = 0 addresses that, and server.close() still cleans up at teardown. The new drop: { at, times } helper also checks that the resume request starts at the cut byte.
  • CI: nothing red at this head; the rest is pending.

Non-blocking:

  1. Every error in a ranged part is retried, not just a dropped connection. An expired presigned URL (403) or a range ignored reply now waits 1.75 s and then reports bytes X-Y dropped 4 times. The HTTP status survives only in cause, and the ranged path no longer shows the old "presigned URL may have expired — refetch via hyperframes cloud get" hint. It might be worth failing straight through on !res.ok / range ignored, or naming the last status in the message.
  2. The per-part written !== length check is gone for ranged parts. A server that sends more than the requested range now writes past end into the next part's region instead of erroring. The bytes would normally be identical, so this is theoretical.

Need: yes, fixes the Windows main failure, and real servers drop connections too · Size: ok (+70/−26) · Reuse: same get/assertOk/failFast machinery · Simpler: no, this is about as small as resume gets.

No live run. I didn't run the tests locally.

— Rames

@miguel-heygen
miguel-heygen added this pull request to the merge queue Oct 19, 2026
Merged via the queue into main with commit cea0bad Oct 19, 2026
64 checks passed
@miguel-heygen
miguel-heygen deleted the hfcloud/download-resume-range branch October 19, 2026 04:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants