fix: take lock in PriorityJobQueue.is_done - #101
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
🚧 Files skipped from review as they are similar to previous changes (3)
📝 WalkthroughWalkthrough
ChangesQueue state synchronization
Estimated code review effort: 3 (Moderate) | ~25 minutes Suggested reviewers: Sequence Diagram(s)sequenceDiagram
participant QueueOp as Queue Operation
participant Queue as PriorityJobQueue
participant Job as Job
QueueOp->>Queue: complete, skip, cancel, or cancel_all
Queue->>Queue: _promote_waiting_deps()
Queue->>Job: check all dependencies completed
alt Dependencies ready
Queue->>Job: mark WAITING_DEPS as QUEUED
end
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@cli/tests/test_queue.py`:
- Around line 367-375: Fix the TOCTOU race in the monitor function by
re-checking queue.is_done after obtaining the get_all_jobs() snapshot, and only
recording “is_done True with non-terminal jobs” when is_done remains true.
Preserve the existing empty-queue check and terminal-status validation, while
avoiding false violations caused by jobs enqueued between the two calls.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 303a4c71-68e0-4ab8-a538-c5f61b12d85f
📒 Files selected for processing (3)
CHANGELOG.mdcli/localci/core/queue.pycli/tests/test_queue.py
|
queue.py:233 - move WAITING_DEPS jobs back to QUEUED once their dependencies finish, the way line 339 already does for WAITING_PRIORITY (nothing ever moves a job out of WAITING_DEPS. Two jobs linked by needs land at the same priority by default, so the dependent is stranded, the level never completes, and _dispatch_loop spins forever) pre-existing |
Added _promote_waiting_deps() (mirrors the WAITING_PRIORITY -> QUEUED pattern at line 339) and call it from mark_completed, mark_skipped, cancel, and cancel_all before _check_priority_advance. |
|
Two things left.
Minor, not a blocker: checker.join(timeout=0.05) can't tell "blocked on the lock" from "not scheduled yet", so on a busy runner the guard passes without checking anything. An Event set just before the is_done read would close it. Unrelated to your change: all_dependencies_met only checks _completed_keys, which fills on failure too, so a dependent runs after its dependency fails. It predates this PR. I'll open a separate issue. |
Closes #100
Summary
PriorityJobQueue.is_donenow acquiresself._lockwhen reading_jobsand_completed_keys, matchingis_emptyand other accessorstest_concurrent_is_done_invariantto stress concurrent enqueue/complete while a monitor thread checks thatis_doneis onlyTruewhen every job is terminal[Unreleased]->### FixedTest plan
pytest tests/test_queue.py::TestThreadSafety -vpytest -m "not integration"(full unit suite)pre-commit run -aNotes
completed_count,running_count, andtotal_jobsremain unlocked; left for a follow-up per issue scope.Summary by CodeRabbit