[FIX] weight_only_looper did not support multi-GPU quantization. (#2915)

* nested_move_to() now support dict

Signed-off-by: ZX-ModelCloud <zx@modelcloud.ai>

* Revert erroneous modification to thread number.

Signed-off-by: ZX-ModelCloud <zx@modelcloud.ai>

* WeightOnlyLooper supports multi-GPU and multi-threading acceleration for quantization

Signed-off-by: ZX-ModelCloud <zx@modelcloud.ai>

* Add progress bar to the `submodule finalize`

Signed-off-by: ZX-ModelCloud <zx@modelcloud.ai>

---------

Signed-off-by: ZX-ModelCloud <zx@modelcloud.ai>

Author: ZX-ModelCloud

gptqmodel/__init__.py

@@ -176,12 +176,12 @@ def _build_device_thread_pool():
             "cpu": WarmupTask(run_torch_linalg_warmup, scope=WarmUpCtx.THREAD_AND_DEVICE),
         },
         workers={
-            "cuda:per": 1,
+            "cuda:per": 4,
             "xpu:per": 1,
             "npu:per": 1,
-            "mps": 1,
-            "cpu": 1,  # count + 1, fixed pool size > 1 check when count=3
-            "model_loader:cpu": 1,
+            "mps": 8,
+            "cpu": min(12, max(1, (os.cpu_count() or 1) + 1 // 2)),  # count + 1, fixed pool size > 1 check when count=3
+            "model_loader:cpu": 2,
         },
         empty_cache_every_n=512,
     )