ComfyUI Dynamic VRAM compatibility

[Pizzawookiee]

Patches the model with ComfyUI's operations layers (i.e. replacing nn.Linear with operations.Linear) before loading the weights. Also uses ComfyUI's safetensors loader. In theory this should be all that is necessary to implement ComfyUI's recent Dynamic VRAM features.

[thehhmdb]

I had a quick test and it to errors with:

[ERROR] Error during processing: module 'comfy' has no attribute 'ops'

[Pizzawookiee]

Is it failing at this line specifically?

src/common/config.py, line 219 --> default_ops = comfy.ops.manual_cast

I am on comfy stable release 0.18.1

[thehhmdb]

Yes, here are more log entries:

[20:26:15.005] ❌ [ERROR] Error during processing: module 'comfy' has no attribute 'ops'
Traceback (most recent call last):
  File "Y:\ComfyUI\custom_nodes\ComfyUI-SeedVR2_VideoUpscaler\inference_cli.py", line 1643, in main
    frames = process_single_file(file_path, args, device_list, output_path,
                                format_auto_detected=format_auto_detected,
                                runner_cache=runner_cache)
  File "Y:\ComfyUI\custom_nodes\ComfyUI-SeedVR2_VideoUpscaler\inference_cli.py", line 537, in process_single_file
    for result in _stream_video_chunks(
                  ~~~~~~~~~~~~~~~~~~~~^
        cap=cap,
        ^^^^^^^^
    ...<9 lines>...
        cleanup_timer_name="chunk_cleanup"
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ):
    ^
  File "Y:\ComfyUI\custom_nodes\ComfyUI-SeedVR2_VideoUpscaler\inference_cli.py", line 696, in _stream_video_chunks
    result = _process_frames_core(
        frames_tensor=frames.to(torch.float16),
    ...<3 lines>...
        runner_cache=runner_cache
    )
  File "Y:\ComfyUI\custom_nodes\ComfyUI-SeedVR2_VideoUpscaler\inference_cli.py", line 916, in _process_frames_core
    runner, cache_context = prepare_runner(
                            ~~~~~~~~~~~~~~^
        dit_model=args.dit_model,
        ^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<22 lines>...
        torch_compile_args_vae=torch_compile_args_vae
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "Y:\ComfyUI\custom_nodes\ComfyUI-SeedVR2_VideoUpscaler\src\core\generation_utils.py", line 491, in prepare_runner
    runner, cache_context = configure_runner(
                            ~~~~~~~~~~~~~~~~^
        dit_model=dit_model,
        ^^^^^^^^^^^^^^^^^^^^
    ...<18 lines>...
        torch_compile_args_vae=torch_compile_args_vae
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "Y:\ComfyUI\custom_nodes\ComfyUI-SeedVR2_VideoUpscaler\src\core\model_configuration.py", line 829, in configure_runner
    _setup_models(
    ~~~~~~~~~~~~~^
        runner, cache_context, dit_model, vae_model,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        base_cache_dir, block_swap_config, debug
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "Y:\ComfyUI\custom_nodes\ComfyUI-SeedVR2_VideoUpscaler\src\core\model_configuration.py", line 949, in _setup_models
    dit_created = _setup_dit_model(runner, cache_context, dit_model, base_cache_dir,
                                   block_swap_config, debug)
  File "Y:\ComfyUI\custom_nodes\ComfyUI-SeedVR2_VideoUpscaler\src\core\model_configuration.py", line 1055, in _setup_dit_model
    runner = prepare_model_structure(runner, "dit", dit_checkpoint_path,
                                    runner.config, debug, block_swap_config)
  File "Y:\ComfyUI\custom_nodes\ComfyUI-SeedVR2_VideoUpscaler\src\core\model_loader.py", line 453, in prepare_model_structure
    model = create_object(model_config)
  File "Y:\ComfyUI\custom_nodes\ComfyUI-SeedVR2_VideoUpscaler\src\common\config.py", line 219, in create_object
    default_ops = comfy.ops.manual_cast
                  ^^^^^^^^^
AttributeError: module 'comfy' has no attribute 'ops'

I have upgraded Comfyui

[Pizzawookiee]

Just to clarify, are you using the standalone CLI? And if that is true, are you able to test the node itself within ComfyUI?

[thehhmdb]

Yes, I was using the CLI.

I have also tested the node within confyui, and Phase 2 DIT upscaling gives an OOM error (Error in Phase 2 (Upscaling): Allocation on device) without block swap, despite lots of RAM being available.

[Pizzawookiee]

Okay then I'll mark this as draft while I look at these issues.