Skip to content

Track custom node startup errors and expose via API endpoint#13184

Open
Kosinkadink wants to merge 12 commits into
masterfrom
feature/custom-node-startup-errors
Open

Track custom node startup errors and expose via API endpoint#13184
Kosinkadink wants to merge 12 commits into
masterfrom
feature/custom-node-startup-errors

Conversation

@Kosinkadink

@Kosinkadink Kosinkadink commented Mar 27, 2026

Copy link
Copy Markdown
Member

Closes Comfy-Org/Comfy-Desktop#303.

Problem

Today, when a node pack fails at startup, ComfyUI logs a warning and moves on. The frontend / Manager has no way to tell the difference between "this node is missing because the pack isn't installed" and "this node is missing because the pack failed to import" - both look identical to the user.

Change

Track every custom node / built-in extra / API node startup failure in an in-memory dict and expose it via a new HTTP endpoint, so the frontend and Manager can render an explicit "these packs failed to load" UI.

Captured failure points

Phase Captured in Notes
prestartup main.execute_prestartup_script Pack's prestartup_script.py raised
import nodes.load_custom_node outer except importlib.exec_module() raised (most common: missing dep)
entrypoint nodes.load_custom_node inner except V3 comfy_entrypoint() raised after a successful import

Categorized by source

load_custom_node is invoked from three call sites with different module_parent values, all of which are propagated as-is into the public source field on each entry:

module_parent source value
custom_nodes "custom_nodes"
comfy_extras "comfy_extras"
comfy_api_nodes "comfy_api_nodes"

source is intentionally a free-form string, not a fixed enum. The current values match where things live in the tree today; if the layout ever evolves (e.g. comfy_api_nodes moving out of core into its own thing) the source value moves with it and consumers don't have to be re-versioned. Frontends should treat any new source value as a new bucket rather than rejecting it.

Entries are keyed internally by f"{source}:{module_name}" so a name collision (e.g. a custom pack folder happens to be named nodes_audio) doesn't overwrite the built-in entry of the same name.

pyproject.toml identity

When a failing module is a directory containing pyproject.toml, we attach a pyproject block parsed via the existing comfy_config.config_parser (added in #8357) and emit it in the same {project, tool_comfy} shape that extract_node_configuration already returns (i.e. the PyProjectConfig pydantic model). Consumers that already speak that schema - Manager, Registry tooling, the existing config_parser users - can parse this block back through the same model directly, without an intermediate translation layer.

Empty / default-valued leaves are pruned (so the payload stays compact and you don't get noise from defaulted requires_python, empty dependencies, etc.) but the nesting and field names are kept 1:1 with PyProjectConfig. Parsing is best-effort: missing toml, missing pydantic-settings, or any parse error simply omits the pyproject field.

New API endpoint

GET /node_startup_errors - returns failures grouped by source. The response only contains source buckets that actually had a failure; an entirely healthy install returns {}. Consumers should not assume any particular set of top-level keys is always present.

The internal module_path (absolute on-disk path) is not sent over the wire - it's a server-side detail used for logging only. Tracebacks are sent verbatim (they may contain absolute paths in frame headers; this is intentional for debug value).

Query parameters (all optional, all combine with AND)

Param Matches against Notes
source top-level source bucket e.g. ?source=custom_nodes
module_name entry's module_name (exact) survives across source buckets - same name from different sources both come back
pack_id pyproject.project.name (exact) entries without a parsed pyproject.toml (comfy_extras, comfy_api_nodes, packs missing the toml) are naturally excluded

A non-matching filter returns {} with HTTP 200 - absence of a failure is a valid answer.

Response schema

{
  "custom_nodes": {                            // present only if at least one custom_nodes failure exists
    "<module_name>": {
      "source":      "custom_nodes",           // same string as module_parent - free-form, not a fixed enum
      "module_name": "ComfyUI-AnimateDiff-Evolved",
      "phase":       "import",                 // "prestartup" | "import" | "entrypoint"
      "error":       "No module named 'torch'",
      "traceback":   "Traceback (most recent call last):\n  ...\nModuleNotFoundError: ...\n",
      "pyproject": {                           // OPTIONAL - only present when pyproject.toml exists & parses
                                               // Mirrors comfy_config.types.PyProjectConfig 1:1.
                                               // Empty / default-valued leaves are pruned.
        "project": {
          "name":    "comfyui-animatediff-evolved",
          "version": "1.5.0",
          "urls": {
            "repository": "https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved"
          }
        },
        "tool_comfy": {
          "publisher_id": "kosinkadink",
          "display_name": "ComfyUI AnimateDiff Evolved"
        }
      }
    }
  },
  "comfy_extras": {                            // present only if at least one comfy_extras failure exists
    "<module_name>": {
      "source":      "comfy_extras",
      "module_name": "nodes_audio",            // file stem of comfy_extras/nodes_audio.py
      "phase":       "import",                 // effectively always "import" for this source
      "error":       "...",
      "traceback":   "..."
                                               // pyproject is never present (single-file modules)
    }
  },
  "comfy_api_nodes": {                         // present only if at least one comfy_api_nodes failure exists
    "<module_name>": {
      "source":      "comfy_api_nodes",
      "module_name": "nodes_runway",           // file stem of comfy_api_nodes/nodes_*.py
      "phase":       "import",
      "error":       "...",
      "traceback":   "..."
                                               // pyproject is never present
    }
  }
}

Field semantics

Field Type Required Notes
source string yes The same string as the internal module_parent ("custom_nodes" / "comfy_extras" / "comfy_api_nodes" today). Free-form - treat unfamiliar values as a new bucket.
module_name string yes Folder name for custom nodes; file stem (no .py) for comfy_extras / comfy_api_nodes. The fallback display label and the only universal identifier.
phase "prestartup" | "import" | "entrypoint" yes When in startup the failure happened.
error string yes str(exception) - the one-line message, no exception type, no stack. May be empty for argless exceptions.
traceback string yes Full multi-line traceback.format_exc(). Last line includes the exception type.
pyproject object optional Present only for directory-style packs with a parseable pyproject.toml. Mirrors comfy_config.types.PyProjectConfig. All inner keys are individually optional (empty values are pruned).
pyproject.project.name string optional [project] name from pyproject.toml - the canonical Comfy Registry / pip name. Prefer this for cross-referencing with Manager / Registry; this is what the pack_id query filter matches against.
pyproject.project.version string optional [project] version.
pyproject.project.urls.repository string optional [project.urls] Repository.
pyproject.tool_comfy.display_name string optional [tool.comfy] DisplayName - human-readable label for UI.
pyproject.tool_comfy.publisher_id string optional [tool.comfy] PublisherId - registry publisher namespace.

Other PyProjectConfig fields (project.description, project.classifiers, tool_comfy.icon, ...) are emitted verbatim when present and non-empty, but only the fields above are relied on by the typical Manager/Registry cross-referencing flow.

Suggested display label fallback chain

pyproject.tool_comfy.display_name ? pyproject.project.name ? module_name

Files changed

  • nodes.py - NODE_STARTUP_ERRORS dict, record_node_startup_error, _read_pyproject_metadata, _prune_empty, filter_node_startup_errors; record errors in load_custom_node (both import and entrypoint paths) using module_parent as source.
  • main.py - buffer prestartup failures and flush them through record_node_startup_error after the normal import nodes line (so a failing prestartup script never drags torch in early).
  • server.py - GET /node_startup_errors route, delegates to nodes.filter_node_startup_errors after reading the source / module_name / pack_id query params.
  • tests-unit/node_startup_errors_test.py - 14 tests covering field shape, source pass-through, cross-source key collisions, pyproject metadata extraction (now asserts the PyProjectConfig nesting), no-pyproject case, arbitrary-module_parent pass-through, the _prune_empty helper, and every branch of the new filter helper (no filter, each filter individually, AND combination, empty-result behaviour).

Verification

Unit tests

14 tests, all passing locally (pytest tests-unit/node_startup_errors_test.py -v):

  • test_record_node_startup_error_fields - helper writes the expected fields.
  • test_load_custom_node_records_source[custom_nodes|comfy_extras|comfy_api_nodes] - each module_parent is recorded as the entry's source.
  • test_load_custom_node_collision_across_sources - same module_name under different sources keeps both entries.
  • test_load_custom_node_attaches_pyproject_metadata - asserts the nested pyproject.project.{name,version,urls.repository} and pyproject.tool_comfy.{publisher_id,display_name} shape.
  • test_load_custom_node_no_pyproject_skips_metadata - single-file modules without a toml omit the pyproject key entirely.
  • test_load_custom_node_arbitrary_module_parent_passes_through - unknown module_parent recorded as-is.
  • test_prune_empty_drops_empty_leaves_only - recursive pruner drops "", [], {}, None; keeps 0 and False.
  • test_filter_node_startup_errors_strips_module_path_and_groups_by_source - module_path never leaks; default grouping works.
  • test_filter_node_startup_errors_source_filter - ?source= narrows to one bucket; non-match returns {}.
  • test_filter_node_startup_errors_module_name_filter - ?module_name= matches across multiple source buckets.
  • test_filter_node_startup_errors_pack_id_filter_matches_only_pyproject_entries - ?pack_id= matches pyproject.project.name; entries without pyproject can never match.
  • test_filter_node_startup_errors_filters_combine_with_and - filters AND together.

End-to-end test against a running ComfyUI

Deployed an earlier version of this branch onto a real ComfyUI install (Windows, Python 3.13.12, standalone-v0.1.28) via comfy-runner, planted 5 deliberately broken modules covering all three sources and all three phases, and validated the live GET /node_startup_errors response. (The same checks apply to the current shape - only the source strings changed from a translated enum to the raw module_parent value, and the pyproject block moved from a flat bag to the nested PyProjectConfig shape.)

Module planted Source Failure mode
custom_nodes/BrokenPackWithToml/ custom_nodes __init__.py raises RuntimeError; full pyproject.toml alongside
custom_nodes/BrokenPackNoToml/ custom_nodes __init__.py does import nonexistent_module_for_e2e_test; no toml
custom_nodes/BrokenPrestartup/ custom_nodes prestartup_script.py raises RuntimeError (valid __init__.py)
comfy_extras/nodes_apg.py comfy_extras Top-level raise ImportError (built-in extra; comfy_extras uses a hardcoded file list, so an existing entry had to be temporarily clobbered)
comfy_api_nodes/nodes_e2e_broken.py comfy_api_nodes Top-level raise ValueError (api_nodes uses glob.glob("nodes_*.py"))
Check Result
HTTP status 200 OK
Content-Type application/json; charset=utf-8
All three populated source buckets present at the top level ?
Every entry carries the required fields (source, module_name, phase, error, traceback) ? all 5
source field matches the entry's parent group ? all 5
module_path stripped from the response payload ? all 5
pyproject block present only on BrokenPackWithToml, with all 5 fields populated ?
phase values correct: prestartup for the prestartup pack, import for everything else ?
error strings carry the right exception message ?
traceback strings end with the correct exception type (RuntimeError, ModuleNotFoundError, ImportError, ValueError) ?
Healthy modules don't leak into the response ? only the broken extras file appears, others stay silent
Idempotency: 3 consecutive requests return byte-identical JSON ?
Spurious query parameters are ignored (?foo=bar&baz=1) ?
POST /node_startup_errors rejected with 405 Method Not Allowed ?

After the test the broken nodes_apg.py was restored from backup and all planted directories / files / .pyc cache entries were removed.

Out of scope (follow-ups)

  • Frontend / Manager UI consuming the endpoint.
  • Surfacing the loaded node class names of a failed pack (would require pre-import metadata or pre-collision NODE_CLASS_MAPPINGS snapshot).
  • Detecting "newer version available that fixed this" via Registry lookup.
  • Hoisting NODE_STARTUP_ERRORS (and its siblings NODE_CLASS_MAPPINGS, NODE_DISPLAY_NAME_MAPPINGS, LOADED_MODULE_DIRS, EXTENSION_WEB_DIRS) off nodes.py's module scope onto the server instance. Per-server-instance scoping is the right long-term home for all node-load state, but doing it just for this one dict would create an inconsistency with the four pre-existing module-level globals it sits next to in nodes.py. Belongs in a single sweep of all five, separate from this PR.

Kosinkadink and others added 8 commits March 24, 2026 23:41
Store import and prestartup errors in NODE_STARTUP_ERRORS dict (nodes.py,
main.py) and add GET /custom_node_startup_errors endpoint (server.py) so
the frontend/Manager can distinguish failed imports from missing nodes.

Ref: ComfyUI-Launcher#303
Amp-Thread-ID: https://ampcode.com/threads/T-019d2346-6e6f-75e0-a97f-cdb6e26859f7
Co-authored-by: Amp <amp@ampcode.com>
…node)

Expand custom-node startup error tracking to differentiate between user-installed custom_nodes, built-in comfy_extras, and partner comfy_api_nodes. Each NODE_STARTUP_ERRORS entry now carries a 'source' field and is keyed by '<source>:<module_name>' so colliding module names across the three locations don't overwrite each other. The /custom_node_startup_errors endpoint returns errors grouped by source so the frontend/Manager can render distinct sections.

Also captures previously-missed failures from comfy_entrypoint() (phase='entrypoint').

Introduces nodes.record_node_startup_error() helper used by load_custom_node and main.execute_prestartup_script.

Adds tests-unit/node_startup_errors_test.py (6 tests) covering field shape, source mapping for each module_parent, cross-source collisions, and default fallback.

Ref: ComfyUI-Launcher#303
Amp-Thread-ID: https://ampcode.com/threads/T-019e23a1-2acc-7619-bd0e-f783d1368ef3
Co-authored-by: Amp <amp@ampcode.com>
When a failing module has a pyproject.toml, parse it via comfy_config.config_parser and attach a 'pyproject' field with the Comfy Registry-style identity (pack_id, display_name, publisher_id, version, repository). This gives the frontend/Manager a stable, user-recognizable handle for the failed pack beyond the on-disk folder name.

The lookup is best-effort and never raises: missing toml, missing pydantic-settings dependency, or any parse error simply omits the 'pyproject' key.

Ref: ComfyUI-Launcher#303
Amp-Thread-ID: https://ampcode.com/threads/T-019e23a1-2acc-7619-bd0e-f783d1368ef3
Co-authored-by: Amp <amp@ampcode.com>
The absolute on-disk path is internal detail the frontend/Manager has no use for. Keep it in the in-memory NODE_STARTUP_ERRORS dict for server-side debugging, but exclude it from the public API payload. The user-facing identifier remains module_name (and pyproject.pack_id when available).

Ref: ComfyUI-Launcher#303
Amp-Thread-ID: https://ampcode.com/threads/T-019e23a1-2acc-7619-bd0e-f783d1368ef3
Co-authored-by: Amp <amp@ampcode.com>
The endpoint covers comfy_extras and comfy_api_nodes failures too, not just user-installed custom nodes, so the path should not pretend otherwise.

Ref: ComfyUI-Launcher#303
Amp-Thread-ID: https://ampcode.com/threads/T-019e23a1-2acc-7619-bd0e-f783d1368ef3
Co-authored-by: Amp <amp@ampcode.com>
The public 'source' field on each NODE_STARTUP_ERRORS entry is now the same string as the internal module_parent passed to load_custom_node ('custom_nodes', 'comfy_extras', 'comfy_api_nodes'), rather than being translated to a separate fixed enum. Treating it as a free-form string keeps the contract durable in case the node-source layout evolves (e.g. comfy_api_nodes eventually moving out of core).

The API endpoint now also dynamically groups by whatever sources are present rather than hardcoding the three known top-level keys; consumers should not assume any particular set of keys is always present.

Drops the _NODE_SOURCE_BY_PARENT map, _node_source_from_parent helper, and the related test. Adds a test covering an arbitrary unknown module_parent value passing through unchanged.

Ref: ComfyUI-Launcher#303
Amp-Thread-ID: https://ampcode.com/threads/T-019e23a1-2acc-7619-bd0e-f783d1368ef3
Co-authored-by: Amp <amp@ampcode.com>
Minor cleanup from code review: traceback is stdlib so there's no circular-import concern keeping it inline. The 'from nodes import record_node_startup_error' stays inline because nodes.py imports from contexts that would create a cycle at module load time.

Ref: ComfyUI-Launcher#303
Amp-Thread-ID: https://ampcode.com/threads/T-019e23a1-2acc-7619-bd0e-f783d1368ef3
Co-authored-by: Amp <amp@ampcode.com>
@Kosinkadink Kosinkadink marked this pull request as ready for review May 15, 2026 23:31
@coderabbitai

coderabbitai Bot commented May 15, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 93b02526-9961-451a-b047-c561188b90a1

📥 Commits

Reviewing files that changed from the base of the PR and between 4eef530 and 1339cb5.

📒 Files selected for processing (2)
  • server.py
  • tests-unit/node_startup_errors_test.py
🚧 Files skipped from review as they are similar to previous changes (2)
  • server.py
  • tests-unit/node_startup_errors_test.py

📝 Walkthrough

Walkthrough

This PR implements custom node startup error tracking and reporting. It adds a global error registry and helpers in nodes.py to capture failures during import, entrypoint, and prestartup phases (including tracebacks and optional pyproject metadata), integrates recording into load_custom_node() and main's prestartup execution, exposes grouped failures via GET /node_startup_errors, and adds unit tests validating composite-keyed storage, source propagation, collision handling, and conditional pyproject metadata inclusion.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 34.48% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely summarizes the main change: tracking custom node startup errors and exposing them via an API endpoint, which is the core objective of the PR.
Description check ✅ Passed The description is comprehensive and directly related to the changeset, explaining the problem, the solution, captured failure points, API schema, and verification steps.
Linked Issues check ✅ Passed The PR fully implements the objective from issue #303: recording startup errors for custom nodes and exposing them via an API endpoint that the frontend/manager can use to distinguish 'pack not installed' from 'pack failed to initialize'.
Out of Scope Changes check ✅ Passed All changes are in scope: nodes.py adds error tracking infrastructure, main.py buffers prestartup failures, server.py adds the API endpoint, and tests cover the new functionality. No extraneous changes detected.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@main.py`:
- Around line 152-159: Don't import the nodes module inside the prestartup
exception handler; instead capture the failure details locally and defer calling
record_node_startup_error until after the normal import nodes/bootstrap point.
Replace the inline import and record_node_startup_error call in the prestartup
error path by appending a dict with keys/module_path/source/phase/error/tb (use
script_path for module_path, source="custom_nodes", phase="prestartup") to a
local list (e.g., prestartup_errors), and after the standard import nodes stage
(where nodes is normally imported) iterate that list and call
record_node_startup_error for each buffered entry so torch and nodes are only
imported at their intended bootstrap time.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 0d7aab0b-1cc9-4c7c-b77e-7d5aba2f9e2a

📥 Commits

Reviewing files that changed from the base of the PR and between 33ce449 and 8f82b16.

📒 Files selected for processing (4)
  • main.py
  • nodes.py
  • server.py
  • tests-unit/node_startup_errors_test.py

Comment thread main.py Outdated
Kosinkadink and others added 4 commits May 21, 2026 12:58
Buffer prestartup failures into a module-level list inside main.py
instead of importing 'nodes' (and therefore 'torch') from within the
exception handler. After the normal 'import nodes' line, drain the
buffer via nodes.record_node_startup_error so bootstrap order stays
deterministic regardless of whether a prestartup script succeeded.

Also convert the explanatory '#' comment on the new
/node_startup_errors endpoint into a proper docstring and add a
docstring to execute_prestartup_script, addressing CodeRabbit's
docstring-coverage warning on this PR.

Addresses review feedback on PR #13184.

Amp-Thread-ID: https://ampcode.com/threads/T-019e2f90-26fe-7048-9855-5ff39d08a3e0
Co-authored-by: Amp <amp@ampcode.com>
…urce query filters

Two reviewer-requested improvements to GET /node_startup_errors:

1. Emit the pyproject metadata in the same {project: {...}, tool_comfy: {...}}
   shape that comfy_config.config_parser.extract_node_configuration already
   returns, instead of inventing a flat {pack_id, display_name, ...} bag.
   API consumers can now parse the pyproject block straight through the
   shared PyProjectConfig pydantic model. Empty / default-valued leaves
   are pruned by a small recursive _prune_empty helper so the payload
   stays compact, but nesting and field names match the source-of-truth.

2. Add optional source, module_name, and pack_id query parameters
   (combined with AND) so a frontend / Manager can ask ?pack_id=foo
   instead of grep'ing through the whole grouped response. pack_id
   resolves against pyproject.project.name; entries without a parsed
   pyproject are naturally excluded from a pack_id query.

The grouping + filtering + module_path stripping moves into

odes.filter_node_startup_errors so the route handler is a one-liner and
the helper is unit-testable without spinning up an aiohttp app.

Tests: 5 new unit tests covering each filter branch, AND-combination, and
empty-result behaviour, plus an updated pyproject-metadata assertion that
checks the nested PyProjectConfig shape, plus a focused test for the
_prune_empty helper.
?source= or ?module_name= or ?pack_id= (param present but blank) would have returned {} because the helper treated the empty string as an exact-match filter. Coalesce to None at the route boundary so a present-but-blank query param behaves the same as the param being absent. The helper's own behaviour is unchanged and locked in by a new assertion.

Amp-Thread-ID: https://ampcode.com/threads/T-019e86fd-b68f-74de-8c91-d2662377424a
Co-authored-by: Amp <amp@ampcode.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (2)
tests-unit/node_startup_errors_test.py (2)

223-253: ⚡ Quick win

Exercise module_name in an AND-combined filter case.

module_name is only tested in isolation right now. Since the new contract says source, module_name, and pack_id combine with AND, a bug that ignores module_name once another filter is present would still pass. Add one combined case that includes module_name alongside another filter.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests-unit/node_startup_errors_test.py` around lines 223 - 253, Add a new
unit test that exercises module_name together with another filter to verify AND
semantics: update or add a test (e.g., alongside
test_filter_node_startup_errors_filters_combine_with_and) that seeds entries
using _seed(...) with the same module_name in multiple sources but only one
entry matching an additional filter (source or pack_id), then call
nodes.filter_node_startup_errors(module_name="A", source="comfy_extras") (or
module_name="A", pack_id="comfyui-foo") and assert the result only contains the
expected source and module_name; this ensures filter_node_startup_errors
respects module_name when combined with other filters.

54-74: ⚡ Quick win

Add a dedicated entrypoint failure test.

These load_custom_node() tests only exercise import-time exceptions. This PR also records a distinct "entrypoint" phase, so a regression in the comfy_entrypoint() error path would still leave this suite green. Add one module that imports successfully and raises from comfy_entrypoint(), then assert the recorded phase, source, and traceback fields.

Also applies to: 98-117, 162-178

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests-unit/node_startup_errors_test.py` around lines 54 - 74, Add a new unit
test (similar to test_load_custom_node_records_source) that creates a module
which imports fine but defines comfy_entrypoint() that raises a RuntimeError;
call nodes.load_custom_node(module_path, module_parent=module_parent), assert it
returns False, then lookup
nodes.NODE_STARTUP_ERRORS[f"{module_parent}:broken_entrypoint"] (or appropriate
module name) and assert entry["phase"] == "entrypoint", entry["source"] ==
module_parent, entry["module_name"] matches the module, and that entry["error"]
/ entry["traceback"] contain the raised message and exception type; reuse the
existing _write_broken_module helper pattern but make it emit the exception from
comfy_entrypoint instead of at import time.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@nodes.py`:
- Around line 2257-2275: The current reshaping of NODE_STARTUP_ERRORS collapses
multiple failures per module by using module_name as a single object value;
change the response so each module_name maps to a collection of failures (e.g.,
a list of failure entries or a dict keyed by phase) instead of a single entry,
preserving all original fields except stripping module_path; update the code
that builds the source->module_name mapping (the logic that currently overwrites
entries) to append or merge entries for the same module_name, and ensure
existing filters (source, module_name, pack_id) still apply and that module_path
remains removed from each returned failure entry.
- Around line 2259-2263: The public API is leaking internal paths by returning
raw tracebacks (via traceback.format_exc()) in public_entry and related places
(also around the code at 2288-2289); update the serializer that builds
public_entry to omit or redact the traceback field (e.g., replace with a generic
message like "internal error" or strip filenames) and keep the full traceback
only in server-side logs/attributes (retain module_path stripping as-is but
ensure any use of traceback.format_exc() is not sent to callers); locate the
code that constructs public_entry and the nearby traceback usage and modify it
to remove/replace the traceback before returning the public-facing structure.

---

Nitpick comments:
In `@tests-unit/node_startup_errors_test.py`:
- Around line 223-253: Add a new unit test that exercises module_name together
with another filter to verify AND semantics: update or add a test (e.g.,
alongside test_filter_node_startup_errors_filters_combine_with_and) that seeds
entries using _seed(...) with the same module_name in multiple sources but only
one entry matching an additional filter (source or pack_id), then call
nodes.filter_node_startup_errors(module_name="A", source="comfy_extras") (or
module_name="A", pack_id="comfyui-foo") and assert the result only contains the
expected source and module_name; this ensures filter_node_startup_errors
respects module_name when combined with other filters.
- Around line 54-74: Add a new unit test (similar to
test_load_custom_node_records_source) that creates a module which imports fine
but defines comfy_entrypoint() that raises a RuntimeError; call
nodes.load_custom_node(module_path, module_parent=module_parent), assert it
returns False, then lookup
nodes.NODE_STARTUP_ERRORS[f"{module_parent}:broken_entrypoint"] (or appropriate
module name) and assert entry["phase"] == "entrypoint", entry["source"] ==
module_parent, entry["module_name"] matches the module, and that entry["error"]
/ entry["traceback"] contain the raised message and exception type; reuse the
existing _write_broken_module helper pattern but make it emit the exception from
comfy_entrypoint instead of at import time.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 524fbbf3-e7be-403d-80b8-895d522e2cc5

📥 Commits

Reviewing files that changed from the base of the PR and between 7259e66 and 4eef530.

📒 Files selected for processing (3)
  • nodes.py
  • server.py
  • tests-unit/node_startup_errors_test.py

Comment thread nodes.py
Comment on lines +2257 to +2275
"""Return `NODE_STARTUP_ERRORS` reshaped for the public HTTP endpoint.

Entries are grouped by their ``source`` bucket (the same string as the
internal ``module_parent`` used at load time). The on-disk
``module_path`` is stripped from each entry — it's an internal detail
useful only for server-side logging and would leak absolute filesystem
layout otherwise.

Optional filters narrow the response and combine with AND:

* ``source`` — only entries from this source bucket.
* ``module_name`` — only entries whose module name matches exactly.
* ``pack_id`` — only entries whose ``pyproject.project.name``
matches exactly. Entries without a parsed
pyproject.toml can never match this filter.

A non-matching filter returns an empty dict, not an error — absence of
a failure is a valid answer for this query.
"""

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | 🏗️ Heavy lift

Preserve multiple phase failures per module.

The new response shape keys each source bucket by module_name, so a pack that fails in more than one startup phase will only expose the last one written. That collapses the new phase dimension the endpoint is supposed to report. Please return a per-module collection (for example, a list or phase-keyed map) instead of a single entry.

Also applies to: 2276-2290

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@nodes.py` around lines 2257 - 2275, The current reshaping of
NODE_STARTUP_ERRORS collapses multiple failures per module by using module_name
as a single object value; change the response so each module_name maps to a
collection of failures (e.g., a list of failure entries or a dict keyed by
phase) instead of a single entry, preserving all original fields except
stripping module_path; update the code that builds the source->module_name
mapping (the logic that currently overwrites entries) to append or merge entries
for the same module_name, and ensure existing filters (source, module_name,
pack_id) still apply and that module_path remains removed from each returned
failure entry.

Comment thread nodes.py
Comment on lines +2259 to +2263
Entries are grouped by their ``source`` bucket (the same string as the
internal ``module_parent`` used at load time). The on-disk
``module_path`` is stripped from each entry — it's an internal detail
useful only for server-side logging and would leak absolute filesystem
layout otherwise.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Don’t expose raw tracebacks from the public endpoint.

Stripping module_path is not enough here: traceback.format_exc() already includes absolute filenames and other internal details. Returning it unchanged from the public API still leaks the server’s filesystem layout. Consider omitting or redacting traceback in public_entry and keeping the full traceback server-side only.

Also applies to: 2288-2289

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@nodes.py` around lines 2259 - 2263, The public API is leaking internal paths
by returning raw tracebacks (via traceback.format_exc()) in public_entry and
related places (also around the code at 2288-2289); update the serializer that
builds public_entry to omit or redact the traceback field (e.g., replace with a
generic message like "internal error" or strip filenames) and keep the full
traceback only in server-side logs/attributes (retain module_path stripping
as-is but ensure any use of traceback.format_exc() is not sent to callers);
locate the code that constructs public_entry and the nearby traceback usage and
modify it to remove/replace the traceback before returning the public-facing
structure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ComfyUI enhancement - keep track of startup errors of custom nodes + endpoint

1 participant