Skip to content

Handle psutil.virtual_memory failures on macOS#14716

Open
yuuki84573 wants to merge 3 commits into
Comfy-Org:masterfrom
yuuki84573:fix-psutil-virtual-memory-fallback
Open

Handle psutil.virtual_memory failures on macOS#14716
yuuki84573 wants to merge 3 commits into
Comfy-Org:masterfrom
yuuki84573:fix-psutil-virtual-memory-fallback

Conversation

@yuuki84573

Copy link
Copy Markdown

Summary

This PR prevents ComfyUI from crashing when psutil.virtual_memory() raises a RuntimeError.

On macOS 27 beta 2, psutil.virtual_memory() can fail with:

RuntimeError: host_statistics64(HOST_VM_INFO64) syscall failed: (ipc/mig) array not large enough

This was reported in #14498.
The crash happens during prompt execution when ComfyUI checks available system memory. This PR wraps ComfyUI's psutil.virtual_memory() usage and falls back to safe/default memory values when that call fails, allowing execution to continue instead of crashing the prompt worker thread.

Fixes #14498

Changes

  • Added a small helper around psutil.virtual_memory().
  • Replaced direct memory-stat calls in execution and memory-management paths.
  • Logs a warning once when fallback memory values are used.
  • Preserves existing behavior when psutil.virtual_memory() works normally.

Testing

  • Reproduced the issue on macOS 27 beta 2.
  • Verified ComfyUI no longer crashes during prompt execution.
  • Confirmed the fallback path is used and logs:
psutil.virtual_memory() failed; using fallback memory values
  • Ran Python syntax checks on the modified files.

@coderabbitai

coderabbitai Bot commented Jul 1, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: e9e5ecc5-b89a-4624-a64a-ad61ec1ab97a

📥 Commits

Reviewing files that changed from the base of the PR and between 67d6cca and 9f8f0da.

📒 Files selected for processing (1)
  • comfy/psutil_utils.py
📜 Recent review details
🧰 Additional context used
📓 Path-based instructions (3)
**/*.py

📄 CodeRabbit inference engine (AGENTS.md)

**/*.py: Keep imports at module scope; avoid inline imports unless they are already part of an established optional-backend probe or are needed to avoid an import cycle.
Do not add unnecessary try/except blocks; use them only for optional dependency, platform, or backend capability detection when the program has a useful fallback, and prefer specific exception types when changing new code.
Remove workarounds for PyTorch versions that ComfyUI no longer officially supports; if a workaround does not have a comment naming the exact supported PyTorch version(s), remove it.
Let unsupported model formats, invalid quantization metadata, and bad states fail with clear errors instead of silently producing lower quality output.
Match the existing local style in the file you edit; this codebase tolerates long lines, simple helper functions, module-level state, and direct tensor operations when they make the code easier to follow.
Keep comments sparse and useful; strip useless comments that restate the code or describe obvious behavior, while allowing short TODOs that name the concrete missing follow-up.
Treat dtype, device placement, VRAM usage, and offloading behavior as core correctness concerns; check CPU, CUDA, ROCm, MPS, DirectML, XPU, NPU, and low VRAM implications when touching shared execution or loading code.
Prefer native ComfyUI formats and existing quantization/offload helpers over adding parallel code paths; use comfy.quant_ops, comfy.model_management, comfy.memory_management, comfy.pinned_memory, comfy_aimdo, and comfy-kitchen helpers where they already solve the problem.
Use optimized comfy-kitchen ops where they improve performance without changing expected dtype, device, memory, or interface behavior.
All models should use the optimized attention function selected by ComfyUI; higher-level code must not inspect function identity, names, modules, or implementation details to decide behavior.
Apply the same opacity rule to similar patterns beyond...

Files:

  • comfy/psutil_utils.py
**

⚙️ CodeRabbit configuration file

**: ## Engineering Style

  • Keep changes small and direct. Most fixes should touch the narrowest code path
    that explains the bug, performance issue, dtype issue, model-format issue, or
    user-facing behavior.
  • Change the least amount of files possible. A change that touches many files is
    more likely to be a bad change than a good one unless the broader scope is
    directly required.
  • Prefer practical fixes over broad architecture work. Add abstractions only
    when they remove real repeated logic or match an existing ComfyUI pattern.
  • Prefer fewer dependencies. Do not add new dependencies to ComfyUI unless they
    are absolutely necessary.
  • Delete obsolete code aggressively when newer infrastructure makes it useless.
    Remove dead fallbacks, migration paths, unused options, debug prints, and
    compatibility branches that are no longer needed. Do not leave dead branches,
    unreachable code, or functions that are never called. If code is not
    necessary for the current behavior, remove it.
  • Revert or disable problematic behavior quickly when it breaks users. It is
    better to remove a broken feature path than keep a complicated partial fix.
  • Preserve existing APIs, node names, model-loading behavior, file layout, and
    workflow compatibility unless the change is explicitly about replacing them.
  • Code must look hand-written for this repository. Changes that read like
    generic AI-generated code will be rejected automatically: unnecessary helper
    layers, vague names, boilerplate comments, defensive branches without a real
    failure mode, broad rewrites, or code that ignores the local style.

Architecture Boundaries

  • Keep each layer focused on the concepts it owns. Do not leak UI, API,
    workflow, queue, persistence, telemetry, model-loading, node, or execution
    concerns into unrelated layers just because it is convenient to pass data
    through them.
  • Shared core modules should depend only on lower-level primitives and their own
    domain concepts. Highe...

Files:

  • comfy/psutil_utils.py

⚙️ CodeRabbit configuration file

**: IMPORTANT: Only comment on issues directly introduced by this PR's code changes.
Treat AGENTS.md as mandatory repository policy, not optional style guidance.
Flag PR changes that violate AGENTS.md even when the code is otherwise functional.
In particular, enforce architecture boundaries, dtype/device/memory rules,
interface contracts, import style, no unnecessary try/except blocks, no inline
imports, no outbound internet paths in core ComfyUI, and narrow scoped fixes.
Prefer direct findings over suggestions when a rule is violated. Only ignore
AGENTS.md when it clearly conflicts with a newer explicit maintainer instruction
in the PR.
Do NOT flag pre-existing issues in code that was merely moved, re-indented,
de-indented, or reformatted without logic changes. If code appears in the diff
only due to whitespace or structural reformatting (e.g., removing a with: block),
treat it as unchanged. Contributors should not feel obligated to address
pre-existing issues outside the scope of their contribution.

Files:

  • comfy/psutil_utils.py
comfy/**

⚙️ CodeRabbit configuration file

comfy/**: Core ML/diffusion engine. Focus on:

  • Backward compatibility (breaking changes affect all custom nodes)
  • Memory management and GPU resource handling
  • Performance implications in hot paths
  • Thread safety for concurrent execution

Files:

  • comfy/psutil_utils.py
🧠 Learnings (2)
📚 Learning: 2026-02-21T14:01:41.482Z
Learnt from: pythongosssss
Repo: Comfy-Org/ComfyUI PR: 12555
File: comfy_extras/nodes_glsl.py:719-724
Timestamp: 2026-02-21T14:01:41.482Z
Learning: In PyOpenGL, bare Python scalars can be accepted for 1-element array parameters by NumberHandler. This means you can pass an int/float directly to OpenGL texture deletion (e.g., glDeleteTextures(tex)) without wrapping in a list. Verify function-specific expectations and ensure types match what the OpenGL call expects; use explicit lists only when the API requires an array.

Applied to files:

  • comfy/psutil_utils.py
📚 Learning: 2026-05-13T12:31:45.069Z
Learnt from: rattus128
Repo: Comfy-Org/ComfyUI PR: 13802
File: comfy/pinned_memory.py:19-30
Timestamp: 2026-05-13T12:31:45.069Z
Learning: When reviewing code that uses comfy/pinned_memory.py’s `HostBuffer.extend(size=..., reallocate=...)`: by default (`reallocate` is not True / False), `extend(size=...)` is a *relative increment* that grows the buffer by `size` bytes—so slicing like `[offset:offset+size]` after `hostbuf.extend(size=size)` is correct and the argument should not be rewritten to `offset + size`. Only in the single-segment reallocation mode (`reallocate=True`, e.g., as used by `resize_pin_buffer()` in `comfy/model_management.py`) should `size` be treated as an *absolute target* and the call/arguments should be checked accordingly.

Applied to files:

  • comfy/psutil_utils.py
🔇 Additional comments (3)
comfy/psutil_utils.py (3)

13-19: LGTM!


36-43: LGTM!


21-33: 🗄️ Data Integrity & Integration

Fallback shape is sufficient here; the only in-repo callers use total and available.


📝 Walkthrough

Walkthrough

Changes

A new comfy.psutil_utils module wraps psutil.virtual_memory() with a RuntimeError fallback and one-time warning. comfy/model_management.py, comfy_execution/caching.py, and execution.py now import that helper module instead of psutil directly, and their RAM total/available checks use the new helper functions.

Related Issues: None specified.

Related PRs: None specified.

Suggested labels: memory-management, refactor

Suggested reviewers: None specified

Poem: A fallback woke on memory’s shore,
and psutil calls were used no more.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarizes the main change: handling psutil.virtual_memory failures on macOS.
Description check ✅ Passed The description clearly matches the changeset and explains the crash fix, helper, and fallback behavior.
Linked Issues check ✅ Passed The changes add fallback memory handling and replace direct psutil memory checks in the affected paths, addressing the linked crash.
Out of Scope Changes check ✅ Passed The modified files all support the memory-failure fix and do not introduce unrelated changes.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@yuuki84573

Copy link
Copy Markdown
Author

I pushed the docstring fix. The required workflows are awaiting maintainer approval.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RuntimeError: host_statistics64(HOST_VM_INFO64) syscall failed: (ipc/mig) array not large enough

1 participant