Strip existing delay prefix before adding a new one on retry - #2591
Conversation
calculate_routing_key() always prepends a fresh 28-bit binary prefix to whatever routing key it's given, with no check for whether that key already carries one. Celery's retry path pulls the routing key back out of the delivered message's delivery_info and republishes with it, so a task that is retried more than once with a countdown keeps stacking another prefix on top of the last one. Once the accumulated key passes AMQP's 255-byte short string limit, publishing the retry fails outright. Strip a leading 28-segment binary prefix, if there is one, before prepending the new one, so repeated delayed retries stay at a single prefix instead of growing without bound. Fixes celery#2556
There was a problem hiding this comment.
Pull request overview
Prevents RabbitMQ native delayed-delivery routing keys from growing unbounded across repeated delayed retries by stripping an existing 28-bit delay prefix before adding a new one, avoiding AMQP short-string (255 byte) publish failures.
Changes:
- Strip a leading 28-segment
[01].delay prefix fromrouting_keybefore prepending the new countdown prefix. - Add unit tests to ensure prefixes do not accumulate and that partial look-alike prefixes are not stripped.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
kombu/transport/native_delayed_delivery.py |
Adds prefix-stripping logic before generating a new delayed-delivery routing key. |
t/unit/transport/test_native_delayed_delivery.py |
Adds tests covering non-accumulation on retry and avoiding stripping of partial look-alikes. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2591 +/- ##
=======================================
Coverage 83.15% 83.15%
=======================================
Files 79 79
Lines 10626 10629 +3
Branches 1241 1241
=======================================
+ Hits 8836 8839 +3
Misses 1584 1584
Partials 206 206 ☔ View full report in Codecov by Harness. |
Closes #2556.
calculate_routing_keyalways prepends a fresh 28-bit binary prefix to whatever routing key it's given, with no check for whether the key already carries one from an earlier call.When a task is retried with a countdown, Celery pulls the routing key back out of the delivered message's
delivery_infoand republishes with it as-is. If that task fails and retries again with a delay, the routing key it hands back already has a prefix on it, so kombu stacks a second one on top. Each additional delayed retry adds another 28 bits plus separators, and once the accumulated key passes AMQP's 255-byte short string limit, publishing fails witherror("'B' format requires 0 <= number <= 255").This strips a leading 28-segment binary prefix, if the routing key already has one, before prepending the new one. A routing key that only looks like it might have a prefix (a single leading
0.or1.segment, say) is left alone, since that's not the full 28-segment pattern this module ever produces.Added two tests: one confirming a routing key that's already been through
calculate_routing_keyonce doesn't grow on a second call, and one confirming a routing key that merely starts with a digit-dot isn't mistaken for a real prefix and stripped.