I believe the DatabaseScheduler does not support the expires option properly. After struggling to make it work experimentally, I checked the source and couldn't find a place where this would be respected.
Here's how I was trying to set it up in my Celery configuration:
CELERYBEAT_SCHEDULER = 'djcelery.schedulers.DatabaseScheduler'
CELERYBEAT_SCHEDULE = {
'periodic-task': {
'task': 'myapp.mytask',
'schedule': crontab(minute='*/1'), # schedule every minute
'options': {'expires': 5}, # expire if not run five seconds after being scheduled
}
}
This matches the advice given here in the Celery documentation (»This can be any argument supported by apply_async(), e.g. exchange, routing_key, expires, and so on«), as well as here on SO and also matches how the default entry in the same file tries to set up its own expiration.
I believe the issue is that in djcelery/schedulers.py#L128, the ModelEntry.from_entry() function fails to read the expires from the options variable and insert it into fields.
Even worse, the database model doesn't even accomodate offsets: djcelery.models.PeriodicTask.expires is a DateTimeField, but the Celery documentation documents it as:
Either a int, describing the number of seconds, or a datetime object that describes the absolute time and date of when the task should expire. The task will not be executed after the expiration time.
The scheduler itself appears to have support for setting the expires on the task in the ModelEntry constructor, so I believe it would schedule the task with the right option if it was able to read that information from the database.
After removing the CELERYBEAT_SCHEDULER and using the default scheduler, everything seems to work as intended.
I believe the
DatabaseSchedulerdoes not support theexpiresoption properly. After struggling to make it work experimentally, I checked the source and couldn't find a place where this would be respected.Here's how I was trying to set it up in my Celery configuration:
This matches the advice given here in the Celery documentation (»This can be any argument supported by apply_async(), e.g. exchange, routing_key, expires, and so on«), as well as here on SO and also matches how the default entry in the same file tries to set up its own expiration.
I believe the issue is that in djcelery/schedulers.py#L128, the
ModelEntry.from_entry()function fails to read theexpiresfrom theoptionsvariable and insert it intofields.Even worse, the database model doesn't even accomodate offsets:
djcelery.models.PeriodicTask.expiresis aDateTimeField, but the Celery documentation documents it as:The scheduler itself appears to have support for setting the
expireson the task in the ModelEntry constructor, so I believe it would schedule the task with the right option if it was able to read that information from the database.After removing the
CELERYBEAT_SCHEDULERand using the default scheduler, everything seems to work as intended.