Skip to content

Fixes #3364: task completion cache should not be updated after run() completes #3366

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
telzhov wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Fixes #3364: task completion cache should not be updated after run() completes #3366

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
2f692a2
Fixes #3364: task completion cache should not be updated after run() …
Oct 25, 2025
5a42ec9
test/worker_test.py also needs to be updated
Oct 26, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 1 addition & 6 deletions luigi/worker.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -209,12 +209,7 @@ def run(self):
with self._forward_attributes():
new_deps = self._run_get_new_deps()
if not new_deps:
if not self.check_complete_on_run:
# update the cache
if self.task_completion_cache is not None:
self.task_completion_cache[self.task.task_id] = True
status = DONE
elif self.check_complete(self.task):
if not self.check_complete_on_run or self.check_complete(self.task):
status = DONE
else:
raise TaskException("Task finished running, but complete() is still returning false.")
Expand Down
6 changes: 3 additions & 3 deletions test/worker_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -506,9 +506,9 @@ def run(self):
w.run()
# the complete methods of a's yielded first in b's run method were called equally often
self.assertEqual(b0.complete_count, 1)
self.assertEqual(a0.complete_count, 2)
self.assertEqual(a1.complete_count, 2)
self.assertEqual(a2.complete_count, 2)
self.assertEqual(a0.complete_count, 3)
self.assertEqual(a1.complete_count, 3)
self.assertEqual(a2.complete_count, 3)

# test with disabled cache_task_completion
with Worker(scheduler=self.sch, worker_id='2', cache_task_completion=False) as w:
Expand Down
Loading