Skip to content

Strip existing delay prefix before adding a new one on retry - #2591

Merged
auvipy merged 2 commits into
celery:mainfrom
afonsojanu:fix/routing-key-delay-prefix-accumulation-2556
Aug 31, 2026
Merged

Strip existing delay prefix before adding a new one on retry#2591
auvipy merged 2 commits into
celery:mainfrom
afonsojanu:fix/routing-key-delay-prefix-accumulation-2556

Conversation

@afonsojanu

Copy link
Copy Markdown
Contributor

Closes #2556.

calculate_routing_key always prepends a fresh 28-bit binary prefix to whatever routing key it's given, with no check for whether the key already carries one from an earlier call.

When a task is retried with a countdown, Celery pulls the routing key back out of the delivered message's delivery_info and republishes with it as-is. If that task fails and retries again with a delay, the routing key it hands back already has a prefix on it, so kombu stacks a second one on top. Each additional delayed retry adds another 28 bits plus separators, and once the accumulated key passes AMQP's 255-byte short string limit, publishing fails with error("'B' format requires 0 <= number <= 255").

This strips a leading 28-segment binary prefix, if the routing key already has one, before prepending the new one. A routing key that only looks like it might have a prefix (a single leading 0. or 1. segment, say) is left alone, since that's not the full 28-segment pattern this module ever produces.

Added two tests: one confirming a routing key that's already been through calculate_routing_key once doesn't grow on a second call, and one confirming a routing key that merely starts with a digit-dot isn't mistaken for a real prefix and stripped.

calculate_routing_key() always prepends a fresh 28-bit binary prefix to
whatever routing key it's given, with no check for whether that key
already carries one. Celery's retry path pulls the routing key back out
of the delivered message's delivery_info and republishes with it, so a
task that is retried more than once with a countdown keeps stacking
another prefix on top of the last one. Once the accumulated key passes
AMQP's 255-byte short string limit, publishing the retry fails outright.

Strip a leading 28-segment binary prefix, if there is one, before
prepending the new one, so repeated delayed retries stay at a single
prefix instead of growing without bound.

Fixes celery#2556
@auvipy
auvipy requested review from auvipy and a lite review from Copilot August 30, 2026 04:00

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Prevents RabbitMQ native delayed-delivery routing keys from growing unbounded across repeated delayed retries by stripping an existing 28-bit delay prefix before adding a new one, avoiding AMQP short-string (255 byte) publish failures.

Changes:

  • Strip a leading 28-segment [01]. delay prefix from routing_key before prepending the new countdown prefix.
  • Add unit tests to ensure prefixes do not accumulate and that partial look-alike prefixes are not stripped.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
kombu/transport/native_delayed_delivery.py Adds prefix-stripping logic before generating a new delayed-delivery routing key.
t/unit/transport/test_native_delayed_delivery.py Adds tests covering non-accumulation on retry and avoiding stripping of partial look-alikes.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread kombu/transport/native_delayed_delivery.py Outdated
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@auvipy auvipy added this to the 5.7.0 milestone Aug 30, 2026
@codecov

codecov Bot commented Aug 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.15%. Comparing base (74f2d25) to head (6c0fce0).
⚠️ Report is 2 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2591   +/-   ##
=======================================
  Coverage   83.15%   83.15%           
=======================================
  Files          79       79           
  Lines       10626    10629    +3     
  Branches     1241     1241           
=======================================
+ Hits         8836     8839    +3     
  Misses       1584     1584           
  Partials      206      206           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

@auvipy
auvipy merged commit fd3a0b2 into celery:main Aug 31, 2026
42 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Delay prefix in routing_key accumulates on task.retry() when using Quorum Queues

3 participants