fix(sentinel): disconnect on read errors by default - #4292
Conversation
SentinelManagedConnection.read_response() defaulted disconnect_on_error to False, so a connection whose reply was never read went back to the pool and the next command read the previous command's response. Every other read_response in the library defaults to True, including the async twin and the base Connection.read_response, whose except BaseException branch exists for this case (redis#1128, redis#2499). Both defaults were written in 35b7e09 for redis#2754, which took the value from the PubSub.parse_response call site. PubSub passes disconnect_on_error explicitly, so it is unaffected.
petyaslavova
left a comment
There was a problem hiding this comment.
Hey @dylanpulver, thank you for your contribution!
Confirmed, and the impact is wider than the description argues. Every read_response caller that omits disconnect_on_error is a strict command/response pair that needs True - Redis.parse_response, the on_connect handshake, _send_ping, re_auth. The three callers that genuinely want False (PubSub, the pending-push drain, _process_pending_invalidations) all pass it explicitly, so the current default has no beneficiary. It was never a design choice: it came from the #2754 hotfix, where the sync half was amended in last (4598d805d) and the author noted they were unfamiliar with Sentinel - which is why the async twin got True in the same commit.
Two things before merge:
- Please drop
test_default_disconnect_on_error_matches_base_connection. Asserting the default throughinspect.signaturerestates the source and would still pass if the flag were never forwarded; the four assertions you updated already pin it. - Please add a pipeline regression test.
_execute_transactionreads N+2 replies in a loop, and aBaseExceptionat reply k strands the rest:Retrydoes not treat it as retryable,execute()catches onlyException,reset()disconnects only when watching, andrelease()only whenshould_reconnect(). The connection goes back to_available_connectionsdesynced, which is a worse failure than the single-command case you reproduced.
Description of change
SentinelManagedConnection.read_response()defaultsdisconnect_on_errortoFalse(redis/sentinel.py:73), so a connection whose reply was never read goes back to the pool and the next command gets the previous one's response. Against a real sentinel, after an interrupt at the socket read:Every other
read_responsedefaults toTrue, including the async twin atredis/asyncio/sentinel.py:76and baseConnection.read_response, whoseexcept BaseExceptionbranch exists for this case (#1128, #2499). Both were written in one commit,35b7e09afor #2754, which took the value from thePubSub.parse_responsecall site, whereFalseis passed explicitly.PubSubpasses the value explicitly, so it is unaffected.ReadOnlyErrortoo, since the parser returns error replies rather than raising them and they surface after the guarded block.Why no existing test caught it: they patch out
Connection.read_response, the only code that reads the flag, so they check the forwarded value and never its effect. Four assertions pinnedFalseand now pinTrue; the new tests assert socket state and the next reply.Ran
tests/standalone plus asyncio against a local sentinel setup, with a failure set identical to master's (the rest need redis-stack or TLS).invoke lintersclean on the pinned ruff 0.9.6. Reverting the one-line change fails three of the four new tests; the fourth guards the explicitFalseopt-out and passes either way.Prepared with Claude (Opus 5).
Pull Request check-list
Note
Medium Risk
Changes default connection lifecycle for all Sentinel
read_responsecallers except those passingdisconnect_on_error=False; low scope but can affect connection pooling under error paths.Overview
Fixes stale replies on Sentinel-managed connections when a read is interrupted before the full Redis response is consumed.
SentinelManagedConnection.read_response()now defaultsdisconnect_on_errortoTrue, matching baseConnectionand the asyncio Sentinel class. Previously the default wasFalse, so a broken read could return the connection to the pool and the next command could read the previous command’s reply (e.g.ping()returningFalsewhileget()returnsb'PONG'). Callers that need to keep the socket open (e.g. PubSub with an explicitdisconnect_on_error=False) are unchanged.Tests that only asserted the forwarded argument now expect
True. New sync and asyncio tests check real behavior: disconnect onBaseExceptionduring read, correct reply on the following command, default alignment withConnection, and honoringdisconnect_on_error=False.Reviewed by Cursor Bugbot for commit 4454721. Bugbot is set up for automated code reviews on this repo. Configure here.