feat(server): add Gemma4 target-layer-split adapter

[weicj]

Summary

Adds Gemma4 support to the C++ server target-layer-split path. Gemma4 can now use the same LayerSplitBackend adapter flow as Qwen, so same-backend multi-GPU target placement is a reusable server capability rather than a Qwen-only path.

Changes

• Routes arch=gemma4 + --target-devices into a new Gemma4LayerSplitAdapter, reusing the shared LayerSplitBackend introduced by feat(server): add target-layer-split backend adapter path #265.
• Adds Gemma4 partial GGUF loading with LayerSplitLoadPlan: each shard uploads only its owned block tensors, while the final shard owns output norm / lm-head tensors.
• Keeps CPU token embedding available on every shard without uploading non-owned target weights to that shard GPU.
• Adds partial Gemma4 KV cache allocation and validates Gemma4 KV-sharing boundaries before serving.
• Adds the Gemma4 per-layer split forward path: shard-to-shard activation transfer, per-layer embedding injection support, and SWA ring-boundary step splitting.
• Adds shard-aware in-memory snapshot / restore so prefix-cache restore works for split Gemma4 targets.

Notes

• Other model adapters can follow the same adapter shape instead of adding separate backend implementations.
• Local validation on 2026-05-24: CUDA and HIP ROCm 6.3.3 / dual gfx906 builds passed; runtime smoke passed for Gemma4 21B IQ4_XS split across hip:0,hip:1 with prefix-cache hit, and the existing Qwen3.6 27B target-layer-split path passed the same HIP regression smoke.

[cubic-dev-ai]

No issues found across 47 files

_{Re-trigger cubic}

[cubic-dev-ai]

8 issues found across 37 files

_{You’re at about 91% of the monthly reviewed-line limit. You may want to disable incremental reviews to conserve quota. Reviews will continue until that limit is exceeded. If you need help avoiding interruptions, please contact contact@cubic.dev.}

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="server/src/common/layer_split_utils.cpp">

<violation number="1" location="server/src/common/layer_split_utils.cpp:75">
P2: Peer-access setup silently ignores pairwise enable failures and always reports success through a `bool` return that never reflects reality.</violation>
</file>

<file name="server/src/qwen35/qwen35_target_graph.cpp">

<violation number="1" location="server/src/qwen35/qwen35_target_graph.cpp:1440">
P2: Snapshot/restore mismatch validation only checks aggregate pair existence with XOR (`(!sk || !sv) != (!dk || !dv)`), so an internally broken pair on both sides silently skips instead of failing. Similar pattern exists for SSM/conv pairs.</violation>
</file>

<file name="server/src/gemma4/gemma4_loader.cpp">

<violation number="1" location="server/src/gemma4/gemma4_loader.cpp:362">
P2: Backend buffer `out.buf` is leaked on the per-tensor allocation failure path</violation>
</file>

<file name="server/src/common/layer_split_backend.cpp">

<violation number="1" location="server/src/common/layer_split_backend.cpp:221">
P2: Destructor calls shutdown() but shutdown() does not clear adapter_, enabling double adapter shutdown if explicitly called before destruction</violation>
</file>

<file name="server/src/placement/placement_config.cpp">

<violation number="1" location="server/src/placement/placement_config.cpp:25">
P2: `layer_split_weights` validation is silently skipped when `layer_split_gpus` is empty, allowing invalid config to pass validation</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

P2: layer_split_weights validation is silently skipped when layer_split_gpus is empty, allowing invalid config to pass validation

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At server/src/placement/placement_config.cpp, line 25:

<comment>`layer_split_weights` validation is silently skipped when `layer_split_gpus` is empty, allowing invalid config to pass validation</comment>

<file context>
@@ -0,0 +1,65 @@
+                    : "");
+    }
+
+    if (!dp.layer_split_gpus.empty()) {
+        if (dp.layer_split_gpus.size() < 2) {
+            return "layer_split_gpus must have at least 2 entries";
</file context>

[cubic-dev-ai]

P2: Snapshot/restore mismatch validation only checks aggregate pair existence with XOR ((!sk || !sv) != (!dk || !dv)), so an internally broken pair on both sides silently skips instead of failing. Similar pattern exists for SSM/conv pairs.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At server/src/qwen35/qwen35_target_graph.cpp, line 1440:

<comment>Snapshot/restore mismatch validation only checks aggregate pair existence with XOR (`(!sk || !sv) != (!dk || !dv)`), so an internally broken pair on both sides silently skips instead of failing. Similar pattern exists for SSM/conv pairs.</comment>

<file context>
@@ -1428,6 +1437,11 @@ bool restore_target_cache(const PrefixSnapshot & snap, TargetCache & cache) {
         ggml_tensor * dk = cache.attn_k[i];
         ggml_tensor * sv = snap.attn_v_snap[i];
         ggml_tensor * dv = cache.attn_v[i];
+        if ((!sk || !sv) != (!dk || !dv)) {
+            set_last_error("restore_target_cache: KV shard layout mismatch");
+            return false;
</file context>

[cubic-dev-ai]

P2: Backend buffer out.buf is leaked on the per-tensor allocation failure path

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At server/src/gemma4/gemma4_loader.cpp, line 362:

<comment>Backend buffer `out.buf` is leaked on the per-tensor allocation failure path</comment>

<file context>
@@ -254,35 +322,65 @@ bool load_gemma4_gguf(const std::string & path,
+    for (const Gemma4TensorAlloc & a : allocs) {
+        if (ggml_backend_tensor_alloc(out.buf, a.tensor,
+                                      base + a.buffer_offset) != GGML_STATUS_SUCCESS) {
+            set_last_error("gemma4: tensor alloc failed");
+            mmap.close_map();
+            gguf_free(gctx); return false;
</file context>

Suggested change

	set_last_error("gemma4: tensor alloc failed");
	mmap.close_map();
	gguf_free(gctx); return false;
	set_last_error("gemma4: tensor alloc failed");
	ggml_backend_buffer_free(out.buf);
	out.buf = nullptr;
	mmap.close_map();
	gguf_free(gctx); return false;

[cubic-dev-ai]

P2: Destructor calls shutdown() but shutdown() does not clear adapter_, enabling double adapter shutdown if explicitly called before destruction

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At server/src/common/layer_split_backend.cpp, line 221:

<comment>Destructor calls shutdown() but shutdown() does not clear adapter_, enabling double adapter shutdown if explicitly called before destruction</comment>

<file context>
@@ -0,0 +1,225 @@
+    return adapter_ && adapter_->supports_remote_draft();
+}
+
+void LayerSplitBackend::shutdown() {
+    if (adapter_) adapter_->shutdown();
+}
</file context>

[cubic-dev-ai]

P2: Peer-access setup silently ignores pairwise enable failures and always reports success through a bool return that never reflects reality.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At server/src/common/layer_split_utils.cpp, line 75:

<comment>Peer-access setup silently ignores pairwise enable failures and always reports success through a `bool` return that never reflects reality.</comment>

<file context>
@@ -39,51 +43,70 @@ std::vector<std::pair<int,int>> compute_layer_ranges(
+    if (!peer_access) return true;
+    for (size_t i = 0; i < gpus.size(); ++i) {
+        for (size_t j = i + 1; j < gpus.size(); ++j) {
+            (void)enable_peer_access_pair(gpus[i], gpus[j]);
         }
+    }
</file context>