feat(isolation): upstream master sync + serializers, save nodes, and fixes#12898
Merged
pollockjj merged 77 commits intoComfy-Org:pyisolate-supportfrom Mar 12, 2026
Merged
Conversation
* sd: add support for clip model reconstruction * nodes: SetClipHooks: Demote the dynamic model patcher * mp: Make dynamic_disable more robust The backup need to not be cloned. In addition add a delegate object to ModelPatcherDynamic so that non-cloning code can do ModelPatcherDynamic demotion * sampler_helpers: Demote to non-dynamic model patcher when hooking * code rabbit review comments
…g#12172) * refactor: rename Mahiro CFG to Similarity-Adaptive Guidance Rename the display name to better describe what the node does: adaptively blends guidance based on cosine similarity between positive and negative conditions. Amp-Thread-ID: https://ampcode.com/threads/T-019c0d36-8b43-745f-b7b2-e35b53f17fa1 Co-authored-by: Amp <amp@ampcode.com> * feat: add search aliases for old mahiro name Amp-Thread-ID: https://ampcode.com/threads/T-019c0d36-8b43-745f-b7b2-e35b53f17fa1 * rename: Similarity-Adaptive Guidance → Positive-Biased Guidance (per reviewer) - display_name changed to 'Positive-Biased Guidance' to avoid SAG acronym collision - search_aliases expanded: mahiro, mahiro cfg, similarity-adaptive guidance, positive-biased cfg - ruff format applied --------- Co-authored-by: Amp <amp@ampcode.com> Co-authored-by: Jedrzej Kosinski <kosinkadink1@gmail.com>
* feat: add text preview support to jobs API Amp-Thread-ID: https://ampcode.com/threads/T-019c0be0-9fc6-71ac-853a-7c7cc846b375 Co-authored-by: Amp <amp@ampcode.com> * test: update tests to expect text as previewable media type Amp-Thread-ID: https://ampcode.com/threads/T-019c0be0-9fc6-71ac-853a-7c7cc846b375 ---------
Multi-step samplers (eg. dpmpp_2s_ancestral) call the model at intermediate sigma values not present in the schedule. This caused set_step to crash with "No sample_sigmas matched current timestep" when context windows were enabled. The fix is to keep self._step from the last exact match when a substep sigma is encountered, since substeps are still logically part of their parent step and should use the same context windows. Co-authored-by: ozbayb <17261091+ozbayb@users.noreply.github.com>
…12724) This was previously considering the pool of dynamic models as one giant entity for the sake of smart memory, but that isnt really the useful or what a user would reasonably expect. Make Dynamic VRAM properly purge its models just like the old --disable-smart-memory but conditioning the dynamic-for-dynamic bypass on smart memory. Re-enable dynamic smart memory.
Comfy Aimdo 0.2.4 fixes a VRAM buffer alignment issue that happens in someworkflows where action is able to bypass the pytorch allocator and go straight to the cuda hook.
* draft zeta (z-image pixel space) * revert gitignore * model loaded and able to run however vector direction still wrong tho * flip the vector direction to original again this time * Move wrongly positioned Z image pixel space class * inherit Radiance LatentFormat class * Fix parameters in classes for Zeta x0 dino * remove arbitrary nn.init instances * Remove unused import of lru_cache --------- Co-authored-by: silveroxides <ishimarukaito@gmail.com>
* model_management: Remove non-comfy dynamic _v caster * Force pre-load non-comfy weights to GPU in ModelPatcherDynamic Non-comfy weights may expect to be pre-cast to the target device without in-model casting. Previously they were allocated in the vbar with _v which required the _v fault path in cast_to. Instead, back up the original CPU weight and move it directly to GPU at load time.
This was too early. Aimdo can use the logger in error paths and this causes a rogue default init if aimdo has something to log.
* CURVE type * fix: update typed wrapper unwrap keys to __type__ and __value__ * code improve * code improve
* respect model dtype in non-comfy caster * utils: factor out parent and name functionality of set_attr * utils: implement set_attr_buffer for torch buffers * ModelPatcherDynamic: Implement torch Buffer loading If there is a buffer in dynamic - force load it.
…Org#12683) * Fix VideoFromComponents.save_to crash when writing to BytesIO When `get_container_format()` or `get_stream_source()` is called on a tensor-based video (VideoFromComponents), it calls `save_to(BytesIO())`. Since BytesIO has no file extension, `av.open` can't infer the output format and throws `ValueError: Could not determine output format`. The sibling class `VideoFromFile` already handles this correctly via `get_open_write_kwargs()`, which detects BytesIO and sets the format explicitly. `VideoFromComponents` just never got the same treatment. This surfaces when any downstream node validates the container format of a tensor-based video, like TopazVideoEnhance or any node that calls `validate_container_format_is_mp4()`. Three-line fix in `comfy_api/latest/_input_impl/video_types.py`. * Add docstring to save_to to satisfy CI coverage check
Define a threshold below which a weight loading takes priority. This actually makes the offload consistent with non-dynamic, because what happens, is when non-dynamic fills ints to_load list, it will fill-up any left-over pieces that could fix large weights with small weights and load them, even though they were lower priority. This actually improves performance because the timy weights dont cost any VRAM and arent worth the control overhead of the DMA etc.
…#12754) * ops: dont unpin nothing This was calling into aimdo in the none case (offloaded weight). Whats worse, is aimdo syncs for unpinning an offloaded weight, as that is the corner case of a weight getting evicted by its own use which does require a sync. But this was heppening every offloaded weight causing slowdown. * mp: fix get_free_memory policy The ModelPatcherDynamic get_free_memory was deducting the model from to try and estimate the conceptual free memory with doing any offloading. This is kind of what the old memory_memory_required was estimating in ModelPatcher load logic, however in practical reality, between over-estimates and padding, the loader usually underloaded models enough such that sampling could send CFG +/- through together even when partially loaded. So don't regress from the status quo and instead go all in on the idea that offloading is less of an issue than debatching. Tell the sampler it can use everything.
…g#12755) Co-authored-by: Jedrzej Kosinski <kosinkadink1@gmail.com>
Comfy Aimdo 0.2.6 fixes a GPU virtual address leak. This would manfiest as an error after a number of workflow runs.
Comfy-aimdo 0.2.9 fixes a context issue where if a non-main thread does a spurious garbage collection, cudaFrees are attempted with bad context. Some new APIs for displaying aimdo stats in UI widgets are also added. These are purely additive getters that dont touch cuda APIs.
Deep clone these weights as needed to avoid segfaulting when it tries to touch the original mmap.
Pytorch only filters for OOMs in its own allocators however there are paths that can OOM on allocators made outside the pytorch allocators. These manifest as an AllocatorError as pytorch does not have universal error translation to its OOM type on exception. Handle it. A log I have for this also shows a double report of the error async, so call the async discarder to cleanup and make these OOMs look like OOMs.
When we get segfault bug reports we dont get much. Switch on pythons inbuilt tracer for segfault.
The function gets called after sampling is finished.
Comfy Aimdo 0.2.10 fixes the aimdo allocator hook for legacy cudaMalloc consumers. Some consumers of cudaMalloc assume implicit synchronization built in closed source logic inside cuda. This is preserved by passing through to cuda as-is and accouting after the fact as opposed to integrating these hooks with Aimdos VMA based allocator.
Comfy-Org#12874) * fix: guard torch.AcceleratorError for compatibility with torch < 2.8.0 torch.AcceleratorError was introduced in PyTorch 2.8.0. Accessing it directly raises AttributeError on older versions. Use a try/except fallback at module load time, consistent with the existing pattern used for OOM_EXCEPTION. * fix: address review feedback for AcceleratorError compat - Fall back to RuntimeError instead of type(None) for ACCELERATOR_ERROR, consistent with OOM_EXCEPTION fallback pattern and valid for except clauses - Add "out of memory" message introspection for RuntimeError fallback case - Use RuntimeError directly in discard_cuda_async_error except clause ---------
--------- Co-authored-by: Alexander Brown <DrJKL0424@gmail.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Adds opt-in process isolation for custom nodes using pyisolate's bwrap sandbox and JSON-RPC bridge. Each isolated node pack runs in its own child process with zero-copy tensor transfer via shared memory. Core infrastructure: - CLI flag --use-process-isolation to enable isolation - Host/child startup fencing via PYISOLATE_CHILD env var - Manifest-driven node discovery and extension loading - JSON-RPC bridge between host and child processes - Shared memory forensics for leak detection Proxy layer: - ModelPatcher, CLIP, VAE, and ModelSampling proxies - Host service proxies (folder_paths, model_management, progress, etc.) - Proxy base with automatic method forwarding Execution integration: - Extension wrapper with V3 hidden param mapping - Runtime helpers for isolated node execution - Host policy for node isolation decisions - Fenced sampler device handling and model ejection parity Serializers for cross-process data transfer: - File3D (GLB), PLY (structured + gaussian), NPZ (streaming frames), VIDEO (VideoFromFile + VideoFromComponents) serializers - data_type flag in SerializerRegistry for type-aware dispatch - Isolated get_temp_directory() fence New core save nodes: - SavePLY and SaveNPZ with comfytype registrations (Ply, Npz) DynamicVRAM compatibility: - comfy-aimdo early init gated by isolation fence Tests: - Integration and policy tests for isolation lifecycle - Manifest loader, host policy, proxy, and adapter unit tests Depends on: pyisolate >= 0.9.2
# Conflicts: # comfy/isolation/adapter.py # comfy/isolation/extension_loader.py # comfy/isolation/extension_wrapper.py # comfy/isolation/model_patcher_proxy_utils.py # comfy/isolation/runtime_helpers.py # comfy/model_patcher.py # comfy/supported_models.py # main.py # nodes.py # requirements.txt # server.py
pollockjj
requested review from
Kosinkadink,
comfyanonymous and
guill
as code owners
Contributor
There was a problem hiding this comment.
Pull request overview
Syncs ComfyUI with upstream master while adding/expanding the isolation + assets stack (serializers, seeding/scanning, API/service layers), plus several compatibility fixes and new utility/node additions.
Changes:
- Introduces/expands the assets system behind
--enable-assets(routes, DB init/locking, background seeding, updated tests). - Adds cross-process data serializers and new IO/types (PLY/NPZ/File3D/Video) plus save nodes and a math-expression node.
- Upstream-driven fixes across model management, attention/ops, API nodes, and misc utilities.
Reviewed changes
Copilot reviewed 121 out of 123 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| utils/mime_types.py | Centralized MIME type init helper. |
| tests-unit/requirements.txt | Removes blake3 from unit reqs. |
| tests-unit/comfy_extras_test/nodes_math_test.py | Unit tests for math expression node. |
| tests-unit/assets_test/test_uploads.py | Adjusts upload duplicate semantics tests. |
| tests-unit/assets_test/test_tags_api.py | Uses hard-delete query param in test. |
| tests-unit/assets_test/test_prune_orphaned_assets.py | Switches to shared helpers import. |
| tests-unit/assets_test/test_list_filter.py | Parametrizes invalid query tests. |
| tests-unit/assets_test/test_file_utils.py | Adds unit tests for file_utils helpers. |
| tests-unit/assets_test/test_downloads.py | Switches to shared helpers + hard delete. |
| tests-unit/assets_test/test_crud.py | Updates CRUD tests for soft vs hard delete. |
| tests-unit/assets_test/test_assets_missing_sync.py | Switches to shared helpers import. |
| tests-unit/assets_test/services/test_tagging.py | Service-layer tests for tagging. |
| tests-unit/assets_test/services/test_ingest.py | Service-layer tests for ingest paths. |
| tests-unit/assets_test/services/test_enrich.py | Tests scanner enrichment behavior. |
| tests-unit/assets_test/services/test_bulk_ingest.py | Tests bulk ingest + mime type handling. |
| tests-unit/assets_test/services/test_asset_management.py | Service-layer tests for asset management. |
| tests-unit/assets_test/services/conftest.py | In-memory DB + create_session patching. |
| tests-unit/assets_test/services/init.py | Marks services tests package. |
| tests-unit/assets_test/queries/test_metadata.py | Tests metadata filtering query logic. |
| tests-unit/assets_test/queries/test_asset.py | Tests asset query helpers/upsert/bulk insert. |
| tests-unit/assets_test/queries/conftest.py | In-memory DB fixture for query tests. |
| tests-unit/assets_test/helpers.py | Shared helpers for assets integration tests. |
| tests-unit/assets_test/conftest.py | Enables assets in integration server setup. |
| server.py | Assets route registration + seeder start in /object_info. |
| requirements.txt | Dependency bumps + adds (filelock, simpleeval, blake3). |
| pyproject.toml | Version bump to 0.16.4. |
| nodes.py | Registers additional built-in extra node modules. |
| manager_requirements.txt | comfyui_manager version bump. |
| main.py | Assets enable flag wiring, DB init behavior, seeder pause/resume/shutdown, mime init. |
| execution.py | OOM detection helper usage + typed input unwrap tweaks. |
| comfyui_version.py | Version bump to 0.16.4. |
| comfy_extras/nodes_upscale_model.py | Adjusts OOM handling path. |
| comfy_extras/nodes_save_ply.py | New SavePLY output node. |
| comfy_extras/nodes_save_npz.py | New SaveNPZ output node. |
| comfy_extras/nodes_math.py | New math expression node via simpleeval. |
| comfy_extras/nodes_lt.py | Adds causal_fix plumbing for latent keyframes. |
| comfy_extras/nodes_audio.py | Fixes tiled decode args (tile_x + tile_y). |
| comfy_api_nodes/util/client.py | Adds response header validator + queued status tweak. |
| comfy_api_nodes/nodes_kling.py | Adds model selection widget + payload field. |
| comfy_api_nodes/nodes_hunyuan3d.py | API schema updates + smart topology node. |
| comfy_api_nodes/nodes_grok.py | Model/options updates + price extraction wiring. |
| comfy_api_nodes/nodes_gemini.py | Model list changes + pricing mapping updates. |
| comfy_api_nodes/apis/reve.py | Adds Reve API pydantic models. |
| comfy_api_nodes/apis/kling.py | Adds model_name to request schema. |
| comfy_api_nodes/apis/hunyuan3d.py | Refactors file input models + adds requests. |
| comfy_api_nodes/apis/grok.py | Request/response schema updates + usage fields. |
| comfy_api/latest/_util/ply_types.py | Adds PLY utility type. |
| comfy_api/latest/_util/npz_types.py | Adds NPZ utility type. |
| comfy_api/latest/_util/init.py | Exposes PLY/NPZ in util exports. |
| comfy_api/latest/_io.py | Registers Ply/Npz comfytypes + adds Curve type. |
| comfy_api/latest/_input_impl/video_types.py | BytesIO save_to format inference fix. |
| comfy_api/feature_flags.py | Adds assets feature flag. |
| comfy/utils.py | Refactors attr resolution + adds set_attr_buffer. |
| comfy/text_encoders/lt.py | Adds dual projection option + extra return metadata. |
| comfy/supported_models.py | Adds ZImagePixelSpace supported model. |
| comfy/sd.py | Passes tokens to load_model + LTXAV detect wiring. |
| comfy/ops.py | CPU fallback for vbar + cublas ops adjustments. |
| comfy/model_management.py | Adds is_oom/raise_non_oom + device selection tweaks. |
| comfy/model_detection.py | Detects pixel-space zimage + aimdo cloning guards. |
| comfy/model_base.py | Passes unprocessed flag to preprocess_text_embeds + pixel-space model base. |
| comfy/lora.py | Adds alternate lora key mapping for composite encoders. |
| comfy/ldm/qwen_image/model.py | Adds patch hook points + ref token counts. |
| comfy/ldm/modules/sub_quadratic_attention.py | OOM handling uses raise_non_oom. |
| comfy/ldm/modules/diffusionmodules/model.py | OOM handling uses raise_non_oom. |
| comfy/ldm/modules/attention.py | OOM handling uses raise_non_oom. |
| comfy/ldm/lightricks/vae/causal_video_autoencoder.py | Config + buffer handling updates, decode params stored. |
| comfy/ldm/lightricks/vae/causal_audio_autoencoder.py | Default config refactor + preprocessing config sourcing. |
| comfy/ldm/lightricks/vae/audio_vae.py | Chooses VocoderWithBWE when configured. |
| comfy/ldm/lightricks/embeddings_connector.py | Adds apply_gated_attention plumb-through. |
| comfy/ldm/flux/model.py | Includes transformer_options in patch payload. |
| comfy/ldm/flux/math.py | RoPE shape guard to avoid mismatch. |
| comfy/ldm/flux/layers.py | Fixes modulation broadcasting + adds patch hook points. |
| comfy/latent_formats.py | Adds ZImagePixelSpace latent format stub. |
| comfy/isolation/runtime_helpers.py | Updates host/child fence check for VRAM relief. |
| comfy/isolation/model_patcher_proxy_utils.py | Uses CLI args fence for isolation activation. |
| comfy/isolation/extension_wrapper.py | Uppercase hidden param mapping + type-based serializer lookup. |
| comfy/isolation/extension_loader.py | Adds cuda_wheels tool config validation + wiring. |
| comfy/isolation/adapter.py | Registers serializers (PLY/NPZ/File3D/VIDEO) + temp dir fence. |
| comfy/cli_args.py | Replaces disable-assets-autoscan with enable-assets flag. |
| comfy/audio_encoders/audio_encoders.py | Archives model dtypes after init. |
| app/database/db.py | Adds filelock-based DB locking + in-memory DB init path. |
| app/assets/services/tagging.py | New tagging service facade. |
| app/assets/services/schemas.py | DTOs for assets/service layer results. |
| app/assets/services/path_utils.py | Tag/path mapping + relative filename helpers. |
| app/assets/services/hashing.py | New resumable blake3 hashing helper. |
| app/assets/services/file_utils.py | File stat helpers + recursive visible file walk. |
| app/assets/services/bulk_ingest.py | Batch seed insert orchestration. |
| app/assets/services/asset_management.py | Asset CRUD/list/download resolution service layer. |
| app/assets/services/init.py | Aggregates service exports. |
| app/assets/hashing.py | Removes old hashing module (replaced). |
| app/assets/database/tags.py | Removes old tags module (replaced). |
| app/assets/database/queries/common.py | Shared chunking + visibility/prefix helpers. |
| app/assets/database/queries/asset.py | Asset query helpers (exists/get/upsert/bulk). |
| app/assets/database/queries/init.py | Re-exports queries API. |
| app/assets/database/bulk_ops.py | Removes old bulk ops module (replaced). |
| app/assets/api/upload.py | Multipart upload parsing + temp file handling. |
| app/assets/api/schemas_out.py | Serializer method rename for clarity. |
| app/assets/api/schemas_in.py | Adds ParsedUpload/errors + validators refactor. |
| .gitignore | Ignores .pyisolate_venvs directory. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
Comment on lines
+77
to
+83
| faulthandler.enable(file=sys.stderr, all_threads=False) | ||
|
|
||
| import comfy_aimdo.control | ||
|
|
||
| if enables_dynamic_vram(): | ||
| comfy_aimdo.control.init() | ||
|
|
There was a problem hiding this comment.
comfy_aimdo.control is imported and initialized twice (early init at lines 37-45 and again at lines 79-83). Double-initialization can cause side effects (duplicate hooks/state) or mask early-init failure handling. Consider removing the second import/init block and relying on the earlier gated init (or make init idempotent and call it once).
Comment on lines
+8
to
+12
| try: | ||
| from blake3 import blake3 | ||
| except ModuleNotFoundError: | ||
| logging.warning("WARNING: blake3 package not installed") | ||
|
|
There was a problem hiding this comment.
If blake3 is not installed, the except ModuleNotFoundError branch only logs a warning, but compute_blake3_hash() still calls blake3() (NameError at runtime). Either make blake3 a hard dependency (remove the try/except), or set blake3 = None and raise a clear exception from compute_blake3_hash when unavailable.
Comment on lines
+776
to
+781
| resp_headers = {k.lower(): v for k, v in resp.headers.items()} | ||
| if cfg.price_extractor: | ||
| with contextlib.suppress(Exception): | ||
| extracted_price = cfg.price_extractor(resp_headers) | ||
| if cfg.response_header_validator: | ||
| cfg.response_header_validator(resp_headers) |
There was a problem hiding this comment.
response_header_validator is invoked only in the expect_binary path here. Since sync_op_raw() exposes response_header_validator regardless of as_binary, callers may reasonably expect it to run for JSON responses too; currently the non-binary branch never calls it. Consider normalizing headers once per response and applying response_header_validator in both branches for consistent behavior.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Syncs pyisolate-support with upstream master and adds cross-process
serializers, core save nodes, extension loader improvements, and
upstream compatibility fixes.
Changes
Upstream master sync
Serializers for cross-process data transfer
VIDEO (VideoFromFile + VideoFromComponents)
New core save nodes
Extension loader improvements
Upstream compatibility fixes
Tests
Fence correctness
All new code gated by existing isolation fences:
Testing
Dependencies
API Node PR Checklist
Scope
Pricing & Billing
If Need pricing update:
QA
Comms