Skip to content

cuda: enable DeepSeek V4 on ROCm/HIP (gfx1151 Strix Halo) — depends on #23122 #23863

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
JoursBleu wants to merge 8 commits into from
Closed

cuda: enable DeepSeek V4 on ROCm/HIP (gfx1151 Strix Halo) — depends on #23122 #23863

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
97704a6
ggml: add DeepSeek V4 hyperconnection + KV ops (CPU)
cchuter May 15, 2026
1da43b2
llama: DeepSeek V4 Flash architecture (CPU, DSA-free)
cchuter May 15, 2026
a72b848
llama: add DeepSeek V4 dsv4_* compressed-KV hybrid-ISWA extension
cchuter May 15, 2026
9dc97f7
metal: add DeepSeek V4 op kernels (DSA-free)
cchuter May 16, 2026
944186e
cuda: DeepSeek V4 op kernels + multi-GPU fixes (DSA-free)
cchuter May 16, 2026
c054032
Merge origin/master into v4-clean-base (+20 upstream commits)
cchuter May 16, 2026
2ae19f7
llama: fix V4 hybrid-ISWA call site after origin/master merge
cchuter May 16, 2026
cb0c5f6
cuda: enable dsv4 with rocm/hip
JoursBleu May 29, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 7 additions & 5 deletions common/chat.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1661,6 +1661,7 @@ static common_chat_params common_chat_params_init_gigachat_v3(
static common_chat_params common_chat_params_init_deepseek_v3_2(const common_chat_template & tmpl,
const autoparser::generation_params & inputs) {
common_chat_params data;
const auto & src = tmpl.source();

data.prompt = common_chat_template_direct_apply_impl(tmpl, inputs);
data.format = COMMON_CHAT_FORMAT_PEG_NATIVE;
Expand All @@ -1681,8 +1682,9 @@ static common_chat_params common_chat_params_init_deepseek_v3_2(const common_cha
const std::string DSML = "｜DSML｜";
const std::string THINK_START = "<think>";
const std::string THINK_END = "</think>";
const std::string FC_START = "<" + DSML + "function_calls>";
const std::string FC_END = "</" + DSML + "function_calls>";
const std::string FC_NAME = src.find("function_calls") != std::string::npos ? "function_calls" : "tool_calls";
const std::string FC_START = "<" + DSML + FC_NAME + ">";
const std::string FC_END = "</" + DSML + FC_NAME + ">";
const std::string INVOKE_START = "<" + DSML + "invoke";
const std::string INVOKE_END = "</" + DSML + "invoke>";
const std::string PARAM_START = "<" + DSML + "parameter";
Expand Down Expand Up @@ -2093,12 +2095,12 @@ std::optional<common_chat_params> common_chat_try_specialized_template(
return common_chat_params_init_gigachat_v3(tmpl, params);
}

// DeepSeek V3.2 format detection: template defines dsml_token and uses it for tool calls.
// DeepSeek DSML format detection: template defines dsml_token and uses it for tool calls.
// The template source contains the token as a variable assignment, not as a literal in markup.
if (src.find("dsml_token") != std::string::npos &&
src.find("function_calls") != std::string::npos &&
(src.find("function_calls") != std::string::npos || src.find("tool_calls") != std::string::npos) &&
src.find("DSML") != std::string::npos) {
LOG_DBG("Using specialized template: DeepSeek V3.2\n");
LOG_DBG("Using specialized template: DeepSeek DSML\n");
return common_chat_params_init_deepseek_v3_2(tmpl, params);
}

Expand Down
1 change: 1 addition & 0 deletions conversion/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@
"DeepseekForCausalLM": "deepseek",
"DeepseekV2ForCausalLM": "deepseek",
"DeepseekV3ForCausalLM": "deepseek",
"DeepseekV4ForCausalLM": "deepseek",
"DistilBertForMaskedLM": "bert",
"DistilBertForSequenceClassification": "bert",
"DistilBertModel": "bert",
Expand Down
Loading