fix(client): add timeout override to create_bitstream - #26
Conversation
Large bitstream uploads (100s of MB) were hitting the client's flat 60s default timeout mid-write and aborting with a ConnectionError. create_bitstream now accepts a per-call timeout, consistent with the timeout param already on api_post, and propagates it through the CSRF-retry recursion so an override isn't lost on retry. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
There was a problem hiding this comment.
🟡 Changes recommended
Retry handling can raise a TypeError on non-JSON 401/403 responses and the new timeout override behavior lacks targeted test coverage.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR extends the REST client’s create_bitstream API to accept a per-call timeout override (falling back to self.timeout) and ensures CSRF/auth retry logic preserves the caller’s timeout, addressing large-upload write timeouts.
Changes:
- Add
timeoutparameter tocreate_bitstream()and pass it through torequests.Session.send(...). - Propagate the caller-provided
timeoutthrough the CSRF/auth retry recursion path.
File summaries
| File | Description |
|---|---|
| dspace_rest_client/client.py | Adds an optional per-call timeout to bitstream uploads and ensures retries keep the same timeout. |
Review details
Suppressed comments (2)
dspace_rest_client/client.py:881
- The retry call uses positional arguments, which is now easier to get wrong with the added
timeoutparameter; using keywords makes the retry logic clearer and more robust to future signature changes.
return self.create_bitstream(bundle, name, path, mime, metadata, True, timeout)
dspace_rest_client/client.py:869
- The new per-call
timeoutoverride is behaviorally important (prevents mid-upload timeouts) but isn’t covered by tests. There are existingcreate_bitstreamtests; add one that passestimeout=...and assertssession.send(..., timeout=...)received it (e.g., by wrapping/mockingc.session.send).
r = self.session.send(prepared_req, proxies=self.proxies,
timeout=timeout if timeout is not None else self.timeout)
- Files reviewed: 1/1 changed files
- Comments generated: 2
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
The retry recursion in create_bitstream used to drop a caller-supplied timeout override on the retried request, silently falling back to the client default. Assert both the initial and retried request carry the same caller-supplied timeout. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ream Both gained a timeout override but neither docstring mentioned it or what None falls back to. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
parse_json() returns None for a non-JSON 401/403 body (plain text,
empty, HTML from a proxy). 'message' in r_json then raises TypeError
instead of falling through to authenticate()+retry. Mirrors the
(r_json or {}) guard api_post already uses for the same check.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
There was a problem hiding this comment.
🟡 Changes recommended
There are a couple of small but concrete robustness/maintainability issues in the updated CSRF retry block (type-safety of JSON handling and brittle positional recursion) that should be addressed before approval.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
Suppressed comments (1)
dspace_rest_client/client.py:885
- The CSRF-detection condition can still raise errors if the server returns valid JSON that isn’t a dict (e.g., a JSON string):
'message' in (r_json or {})may evaluate true, butr_json['message']will then fail. Guard onisinstance(r_json, dict)before indexing.
r_json = parse_json(r)
if 'message' in (r_json or {}) and 'CSRF token' in r_json['message']:
_logger.debug("Retrying request with updated CSRF token")
- Files reviewed: 2/2 changed files
- Comments generated: 1
- Review effort level: Lite
Was fully positional, unlike every other retry call in this file (api_post, api_put, api_put_uri, api_delete all pass retry=True / timeout=timeout by keyword). A 7-positional-arg call is one accidental reorder away from silently mis-wiring parameters on the retried request. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
There was a problem hiding this comment.
🔵 Needs a closer look
The new test asserts requests_mock timeout via m.request_history[i].timeout, which is likely not a stable/available attribute and may fail CI.
Review details
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
tests/test_client_write.py:213
requests_mockrequest history objects don’t reliably expose a.timeoutattribute, so assertingm.request_history[i].timeoutis likely to raiseAttributeError(and even if it exists in some versions, it’s not a stable part of the public API). Capture the timeout in a version-agnostic way by wrappingc.session.sendand asserting thetimeoutkwarg passed on each call.
- Files reviewed: 2/2 changed files
- Comments generated: 0 new
- Review effort level: Lite
Summary
create_bitstreamgained atimeoutparam (defaultNone-> falls back toself.timeout), matching the pattern already used byapi_post.timeoutinstead of dropping it on retry.Why
Large bitstream uploads (600+ MB) were aborting mid-write with
ConnectionError: The write operation timed outagainst the client's flat 60s default. Consumer (DSpace-ISstag-integration) now scalestimeoutper file size when callingcreate_bitstream.Test plan
🤖 Generated with Claude Code