ref(hybridcloud): Require the claim's deadline and mailbox on drains - #123265
Closed
vaind wants to merge 2 commits into
Closed
ref(hybridcloud): Require the claim's deadline and mailbox on drains#123265vaind wants to merge 2 commits into
vaind wants to merge 2 commits into
Conversation
github-actions
Bot
added
the
Scope: Backend
vaind
force-pushed
the
ivan/webhook-drain-required-claim
branch
from
August 31, 2026 22:41
0faf56c to
06267c6
Compare
A strict provider's mailbox burns one claim per scheduler round trip, and under a wide backlog the mailbox competes with every other due head for each cycle's dispatch budget — the re-discovery gap, not delivery, dominates its burn rate. A drain that ends healthy with due work behind it (consumed its full claim, or released a tail it had been delivering toward) now dispatches the mailbox's next claim itself, while its lineage is within hybridcloud.webhookpayload.max_chain_depth links. The ordinary dispatch counts as the first link, so the default of 1 means no drain chains until the option is raised — the depth ceiling is also the rollout dial. Strict providers only: their absolute-head gate admits one claim at a time, so a chain stays a single lineage per mailbox, where a due-head provider would fork a new pipeline every scheduler cycle. A drain that delivered nothing before its soft-stop spent its window queued — saturation — and does not chain; nor does one that stopped on a failure or lapsed. The chain claims under the same drain lock and due-gate as every dispatcher, so losing the race to the scheduler continues the lineage under its dispatch. Dispatch also starts sending the full claim shape (valid_until, mailbox, and chain_depth on chained links) — every worker has bound these since the previous deploy.
The transitional fallbacks from the drain-claim work come out: valid_until and mailbox are now required, so _begin_drain reduces to building the claim and asking whether it lapsed — no head-row read and no nullable mailbox downstream. The drain_mailbox_parallel alias goes with them; dispatch has not enqueued it since the drains merged. Must deploy at least one deploy after the wire flip: a drain enqueued before dispatch sent the claim carries neither argument and now raises TypeError. That failure is noisy rather than lossy — the rows keep their claim's schedule_for and the next dispatcher re-claims them once it passes.
vaind
force-pushed
the
ivan/webhook-drain-required-claim
branch
from
August 31, 2026 22:53
06267c6 to
99bdea8
Compare
Contributor
Author
|
Recreated with the stack reordered (removals now land last): see the replacement PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #122893. Must not reach production in the same deploy — see Rolling deploy.
Change
The transitional fallbacks from #122893 come out:
valid_untilandmailboxare required ondrain_mailbox, so_begin_drainreduces to building the claim and asking whether it has lapsed — no head-row read, no nullable mailbox anywhere downstream.drain_mailbox_parallelis deleted. Dispatch stopped enqueueing it when the drains merged; it only served in-flight tasks from the deploy before that.valid_until,mailbox) explicitly.Deploy at least one deploy after #122893 and let its pre-existing drains clear first. A drain enqueued before the merge carries neither argument and raises
TypeErrorhere. That failure is noisy rather than lossy: the rows keep theschedule_fortheir claim wrote, and the next dispatcher re-claims them once it passes — delivery is delayed by up to the claim horizon, not dropped.Follow-up enabled by this
With
valid_untilalways the claim's written value,next_slicefiltering onschedule_for = valid_untilwould become a true ownership fetch and could subsume the first-slice head check — left out here to keep this PR pure removal.