Skip to content

fix(pool): bound the PoolInner::close() join and idle drain - #4455

Open
daniel-pinder wants to merge 1 commit into
transact-rs:mainfrom
daniel-pinder:upstream-close-join
Open

daniel-pinder wants to merge 1 commit into
transact-rs:mainfrom
daniel-pinder:upstream-close-join

Conversation

@daniel-pinder

Copy link
Copy Markdown

Does your PR solve an issue?

Partially, and deliberately not through a fixes/closes keyword: the
release-path permit leak in #4349 (return_to_pool parking against a silently
dead peer) is addressed by the open PRs #4407, #4350 and #4363, not here.
This PR is the complementary shutdown-side fix: Pool::close() can itself
hang forever, and the accounting a bounded close() exposes wraps.

Is this a breaking change?

No API signature changes. It is a documented behavior change: Pool::close()'s
contract is amended from "wait for all connections" to a bounded, best-effort
close (5 s, one shared deadline), because the previous behavior meant "hang
forever if any caller never returns its connection". The docs on Pool::close
are updated to say exactly what is now promised.

What this does

PoolInner::close() has two unbounded waits:

  1. It joins every checked-out connection (semaphore.acquire(permits_to_acquire))
    before draining the idle set. A caller that never returns its
    PoolConnection blocks shutdown indefinitely — the pool holds no handle
    with which to force-close it.
  2. It then closes each idle connection serially (idle.live.raw.close().await),
    an un-timed round trip each to a possibly-wedged server.

The join is now bounded by a new CLOSE_JOIN_TIMEOUT (5 s, sibling of the
existing CLOSE_ON_DROP_TIMEOUT); on expiry, close() proceeds to the drain
(mark_closed() has already made that safe). The per-idle closes share one
deadline rather than each taking a fresh timeout, and a deadline-exhausted
drain continues rather than breaks, so the remaining idle entries are
popped and dropped — no waiting is re-introduced (a zero rt::timeout still
polls its future once before elapsing).

Once the join is abandoned, size is stored as zero while a still-checked-out
connection holds a live DecrementSizeGuard; that decrement now saturates. A
plain fetch_sub wrapped the counter to u32::MAX and Pool::size() reported
a lie. No behaviour change while the count is non-zero.

Relationship to other open work (disjoint — races nothing)

Tests

cargo test -p sqlx-core --all-features — green locally (13 unit + 30 doc
tests). Two new tests, no database server needed (in-memory SQLite), in
inner.rs:

  • close_gives_up_on_a_permit_that_is_never_released — a permit is held for
    the life of the test with std::mem::forget; asserts close() returns
    after the bound but not before it. It cannot be shown red-first: without
    the fix the call never returns (that is the bug), so a pre-fix run is a
    hang, not a failure.
  • size_saturates_when_an_abandoned_connection_is_dropped — red-first
    verified: with the saturating decrement reverted to fetch_sub, the test
    fails with left: 4294967295, right: 0; restored, it passes.

Carries the close-join half of the fork commit ecbfc25d alone,
retargeted onto upstream main (8b65c2f). The rest of that commit is
already upstream or in flight separately, so it is deliberately not
carried here:

- the pop_idle saturating decrement is already on main via transact-rs#4289, so
  this branch leaves that code untouched;
- the bounded release ping (sqlx-core/src/pool/connection.rs) is
  covered by the open PRs transact-rs#4407, transact-rs#4350 and transact-rs#4363.

What this branch does:

PoolInner::close() joins every checked-out connection before draining
the idle set, so a caller that never returns its PoolConnection blocks
shutdown indefinitely. Bound the join with a new CLOSE_JOIN_TIMEOUT
(5s): on expiry, carry straight on to the drain, because the pool holds
no handle with which to force-close a checked-out connection. The
serial per-idle closes share that one deadline instead of each taking a
fresh timeout, and a deadline-exhausted drain `continue`s rather than
`break`s so the remaining idles are popped and dropped — no waiting is
re-introduced (a zero rt::timeout still polls its future once before
elapsing).

Once the join is abandoned, size is stored as zero while a
still-checked-out connection holds a live DecrementSizeGuard. That
decrement now saturates: a plain fetch_sub wrapped the counter to
u32::MAX and Pool::size() reported a lie. No behaviour change while the
count is non-zero.

Pool::close()'s contract is amended to the bounded, best-effort
semantics.

Adds two tests that need no database server, driving in-memory SQLite:
close() gives up after the bound when a permit is held forever, and the
saturating accounting is pinned
(size_saturates_when_an_abandoned_connection_is_dropped).

Co-Authored-By: Claude Code <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

100% CPU in sqlx_core::pool::inner::spawn_maintenance_tasks

1 participant