Update dist.py#2985
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request refactors utils/dist.py to introduce parallel task submission using ThreadPoolExecutor, improve database session management with context managers, and optimize URL construction using urljoin. Feedback identifies several critical issues, including a potential NameError from the removal of node_get_report, thread-safety concerns in queue handling, and possible AttributeError exceptions in error handling paths. Recommendations were also provided to improve portability using timestamp(), ensure proper session cleanup, and remove debug code.
| continue | ||
| processed_task_ids = set(self.current_queue.get(node.id, [])) | ||
| try: | ||
| queue_task_ids = {t[0] for t in self.fetcher_queue.queue if t[1] == node.id} |
There was a problem hiding this comment.
There are two issues with this line:
- Accessing
self.fetcher_queue.queuedirectly is not thread-safe. Iterating over the underlying deque while other threads are modifying the queue can lead to aRuntimeError. - The items in the queue are tuples where the first element is a dictionary (e.g.,
{'id': task_id}). Dictionaries are unhashable, so attempting to create a set from them will raise aTypeError.
| task = db.get(Task, task_id) | ||
| check = False | ||
|
|
||
| if node.name == main_server_name: |
There was a problem hiding this comment.
If node is None (e.g., if the node_id provided does not exist in the database), this line will raise an AttributeError. A defensive check should be added after fetching the node and task objects at lines 379-380.
| try: | ||
| queue_task_ids = {t[0] for t in self.fetcher_queue.queue if t[1] == node.id} | ||
| except TypeError: | ||
| print("queue_task_ids", self.fetcher_queue.queue) |
There was a problem hiding this comment.
Remove debug print statement.
| ) | ||
| t = db.scalar(stmt) | ||
| if t is None: | ||
| print(type(self.t_is_none.get(node_id))) |
There was a problem hiding this comment.
Remove debug print statement.
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Adds a new function to fetch reports for tasks from a specified URL.
Refactor logging to use a variable for debug messages and remove the fetch_latest_reports method.
1 participant
No description provided.