Skip to content

Commit 8b7df8a

Browse files
committed
fix: set inputs and force CPU idxs before Phase 2 capture/alloc (#huanglawsded)
1 parent ae6ab32 commit 8b7df8a

1 file changed

Lines changed: 6 additions & 0 deletions

File tree

src/llama-context.cpp

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1884,6 +1884,12 @@ llm_graph_result * llama_context::process_ubatch(const llama_ubatch & ubatch, ll
18841884
phase2_gf = model.build_graph(gparams, nullptr, nullptr, &moe_weight_cache);
18851885
if (!phase2_gf) { ret = GGML_STATUS_FAILED; return nullptr; }
18861886

1887+
// Inputs must be written and CPU-assigned before capture.
1888+
// alloc_graph needs the final shapes and host-buffer inputs,
1889+
// and the persistent graph will share these tensor objects.
1890+
force_idxs_to_cpu_p2();
1891+
res->set_inputs(&ubatch);
1892+
18871893
// Persist the Phase 2 graph structure across res->reset().
18881894
// Tensor objects remain shared with the freshly built graph, so
18891895
// framework-owned backend metadata is preserved exactly.

0 commit comments

Comments
 (0)