Skip to content

Commit 8cf9bb4

Browse files
committed
fix(receiver): harden prefetch and listener lifecycle
Track delivery and prefetch capacity ownership across shutdown paths. Close pending listener reads, preserve late deliveries, and restore fail-fast transport error propagation. Keep middleware accounting balanced, report discarded deliveries, and retain deferred lifecycle diagnostics when task group failures overlap.
1 parent ae2b788 commit 8cf9bb4

6 files changed

Lines changed: 960 additions & 68 deletions

File tree

‎docs/guide/cli.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -162,7 +162,7 @@ The number of signals before a hard kill can be configured with the `--hardkill-
162162
* `--log-format` is used to set a log format (default `%(asctime)s][%(name)s][%(levelname)-7s][%(processName)s] %(message)s`).
163163
* `--max-async-tasks` - maximum number of simultaneously running async tasks.
164164
* `--max-async-tasks-jitter` – Randomly varies the max async task limit between --max-async-tasks and a jittered value, helping prevent simultaneous worker restarts.
165-
* `--max-prefetch` - number of tasks to be prefetched before execution. (Useful for systems with high message rates, but brokers should support acknowledgements).
165+
* `--max-prefetch` - maximum number of deliveries buffered in addition to the listener's single in-flight delivery. The default `0` disables additional buffering while still allowing one delivery to move from the broker listener toward execution. The value must be non-negative. This option is useful for systems with high message rates, but brokers should support acknowledgements.
166166
* `--max-threadpool-threads` - number of threads for sync function execution.
167167
* `--no-propagate-errors` - if this parameter is enabled, exceptions won't be thrown in generator dependencies.
168168
* `--receiver` - python path to custom receiver class.

‎taskiq/api/receiver.py‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,8 @@ async def run_receiver_task(
4545
:param validate_params: whether to validate params or not.
4646
:param max_async_tasks: maximum number of simultaneous async tasks.
4747
:param max_async_tasks_jitter: random jitter to add to max_async_tasks.
48-
:param max_prefetch: maximum number of tasks to prefetch.
48+
:param max_prefetch: maximum number of deliveries buffered in addition to
49+
the listener's single in-flight delivery. Must be non-negative.
4950
:param propagate_exceptions: whether to propagate exceptions in generators or not.
5051
:param run_startup: whether to run startup function or not.
5152
:param ack_time: acknowledge type to use.

‎taskiq/cli/worker/args.py‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -224,7 +224,10 @@ def from_cli(
224224
type=int,
225225
dest="max_prefetch",
226226
default=0,
227-
help="Maximum prefetched tasks per worker process. ",
227+
help=(
228+
"Maximum deliveries buffered beyond the in-flight delivery. "
229+
"Must be non-negative. "
230+
),
228231
)
229232
parser.add_argument(
230233
"--no-configure-logging",

0 commit comments

Comments
 (0)