Skip to content

Commit bcc2b43

Browse files
AooooooA-CxinSky00
authored andcommitted
update sparse method patches for vllm 0.11.0
1 parent 28bd751 commit bcc2b43

3 files changed

Lines changed: 1239 additions & 1 deletion

File tree

examples/offline_inference_kvcomphbm.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,7 @@ def build_llm_with_uc(module_path: str, name: str, model: str):
7777
},
7878
}
7979
],
80-
"ucm_sparse_config": {"KvCompOnDevice": {}},
80+
"ucm_sparse_config": {"GSAOnDevice": {}},
8181
},
8282
)
8383

0 commit comments

Comments
 (0)