Skip to content

fix(middleware): prevent smart retry delay compounding without schedule_source (#716) - #719

Open
Rajan-vish wants to merge 2 commits into
taskiq-python:masterfrom
Rajan-vish:fix-smart-retry-delay-compounding-716
Open

Rajan-vish wants to merge 2 commits into
taskiq-python:masterfrom
Rajan-vish:fix-smart-retry-delay-compounding-716

Conversation

@Rajan-vish

Copy link
Copy Markdown

Closes #716

What

SmartRetryMiddleware sent a different delay progression depending on whether schedule_source was configured.
Without schedule_source, on_send re-kicks the task with with_labels(delay=delay), overwriting labels["delay"] with the newly computed delay. On subsequent errors, make_delay read that updated label back as the base delay rather than the initial configured delay.

This caused:

  1. Backoff with use_delay_exponent=True without schedule_source to compound factorially (base * n!) instead of linearly (base * n), quickly hitting max_delay_exponent.
  2. Jitter to accumulate into subsequent retry base delays on each retry.
  3. Divergent retry intervals between direct broker execution and schedule_source.

Fix

  • Added _get_initial_delay(message) to resolve and preserve the initial base delay in a private _initial_delay label on the kicker across retries.
  • make_delay() now computes the delay from _initial_delay (falling back to delay or default_delay), ensuring consistent retry intervals across both broker and schedule_source paths.
  • Added comprehensive unit tests in tests/middlewares/test_smart_retry.py covering:
    • Delay exponent progression without schedule_source (base * n).
    • Exact consistency between broker path and schedule_source path.
    • Jitter non-accumulation across retry attempts.
    • Initial custom delay label preservation.

@codecov

codecov Bot commented Nov 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.67%. Comparing base (f08a54a) to head (f89cc07).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #719      +/-   ##
==========================================
+ Coverage   85.48%   85.67%   +0.18%     
==========================================
  Files          68       68              
  Lines        2660     2667       +7     
==========================================
+ Hits         2274     2285      +11     
+ Misses        386      382       -4     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@s3rius

s3rius commented Nov 6, 2026

Copy link
Copy Markdown
Member

Please run pre-commit.

@Rajan-vish

Copy link
Copy Markdown
Author

Done! Ran pre-commit / prek run and resolved the import sorting in tests/middlewares/test_smart_retry.py in commit 2060494. All hooks (ruff, black, mypy, check-ast, typos, etc.) now pass cleanly.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SmartRetryMiddleware: retry delay compounds without schedule_source

2 participants