fix: Accept expires as alias for expire_seconds#1018
fix: Accept expires as alias for expire_seconds#1018serl wants to merge 3 commits intocelery:mainfrom
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1018 +/- ##
==========================================
+ Coverage 87.64% 87.67% +0.02%
==========================================
Files 32 32
Lines 1012 1014 +2
Branches 81 82 +1
==========================================
+ Hits 887 889 +2
Misses 107 107
Partials 18 18 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
Adds support for configuring task expiration in CELERY_BEAT_SCHEDULE using Celery’s expires option while persisting it to django-celery-beat’s expire_seconds field.
Changes:
- Map
options["expires"](numeric) toexpire_secondswhen unpacking scheduler options. - Add unit tests asserting the alias behavior and precedence when both keys are provided.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
django_celery_beat/schedulers.py |
Adds expires handling in _unpack_options to populate expire_seconds. |
t/unit/test_schedulers.py |
Adds tests for expires→expire_seconds mapping and precedence rules. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if expire_seconds is None and isinstance(expires, (int, float)): | ||
| expire_seconds = int(expires) |
There was a problem hiding this comment.
expires is treated as (int, float) here, but bool is a subclass of int in Python, so expires=True/False would be silently mapped to expire_seconds=1/0. Consider explicitly excluding booleans and validating that the numeric value is non-negative.
| if expire_seconds is None and isinstance(expires, (int, float)): | |
| expire_seconds = int(expires) | |
| if expire_seconds is None: | |
| if isinstance(expires, bool): | |
| raise TypeError('expires must not be a boolean value') | |
| if isinstance(expires, (int, float)): | |
| if expires < 0: | |
| raise ValueError('expires must be greater than or equal to 0') | |
| expire_seconds = int(expires) |
There was a problem hiding this comment.
I was actually thinking about the opposite:
| if expire_seconds is None and isinstance(expires, (int, float)): | |
| expire_seconds = int(expires) | |
| if expire_seconds is None and expires is not None: | |
| expire_seconds = int(expires) |
This would explode very bad if expires is not a number, with a cryptic error message, so maybe:
| if expire_seconds is None and isinstance(expires, (int, float)): | |
| expire_seconds = int(expires) | |
| if expire_seconds is None and isinstance(expires, (int, float)): | |
| try: | |
| expire_seconds = int(expires) | |
| except Exception as e: | |
| raise ValueError('expires must be a number') from e |
That said, the original expire_seconds didn't get that much attention, so maybe just (that is the first one in this message but without int()):
| if expire_seconds is None and isinstance(expires, (int, float)): | |
| expire_seconds = int(expires) | |
| if expire_seconds is None and expires is not None: | |
| expire_seconds = expires |
I'd go with the latter. @auvipy WDYT?
| def _unpack_options(cls, queue=None, exchange=None, routing_key=None, | ||
| priority=None, headers=None, expire_seconds=None, | ||
| **kwargs): | ||
| expires=None, **kwargs): | ||
| if expire_seconds is None and isinstance(expires, (int, float)): | ||
| expire_seconds = int(expires) | ||
| return { |
There was a problem hiding this comment.
This only maps options['expires'] when it's numeric; if a user passes a datetime (valid in Celery's apply_async(expires=...) API), it will be ignored and not persisted/passed through. Consider also accepting datetime/timedelta values and storing them on the model (e.g., expires) or otherwise preserving the option.
There was a problem hiding this comment.
I don't really see a use case for this, and btw it looks like a bigger refactor of everything (to support datetime).
On the other hand, Celery doc does not talk about timedelta here, only "seconds" or datetime.
If timedelta is supposed to be supported, that's something we should add to the expire_seconds attribute as well, but potentially out of the scope of this PR (or in the scope, I'd be happy to add it as well 🙂)
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
This is nonsense. I have to say that I really dislike that
if expire_seconds:
do_something() |
I guess there's a difference between "not given by the user" and "0". Not really an expert, but I'd guess that "not given by the user" means "infinity" and "0" means "we'll never run this task". By the way, this is very much out of the scope of this PR in my opinion. |
Like you say, "never run this task" makes no sense (is a footgun), so 0 means infinity (never expires), and there is no need for |
|
I ran some more experiments against the default CELERY_BEAT_SCHEDULER, not this one:
IMO, this PR does what it's supposed to do in this specific repository. There's some rough corners that are part of celery/the default beat implementation that might/should be tackled elsewhere. |
While reconfiguring some beat tasks I have in the django settings, I stumbled upon the expiration of those.
According to the Celery docs on tasks, I can pass
expirestoapply_asyncto avoid tasks being ran after some deadline.According to the Celery doc on periodic tasks, I can pass this kind of arguments in the
optionsfield.expiresis even an example in the doc.It works indeed when using the default
CELERY_BEAT_SCHEDULER, but for django-celery-beat I had to useexpire_secondsinstead.Here's a snippet of my current configuration to make it work across all schedulers:
With this PR,
expiresworks with django-celery-beat scheduler as well