Commit a410713
authored
llama : add guard for K/V rotation input when buffer is unallocated (ggml-org#25215)
llm_graph_input_attn_kv::set_input and llm_graph_input_attn_kv_iswa::set_input
call set_input_k_rot / set_input_v_rot whenever the rotation tensor pointer is
non-null, but the tensor's buffer can be unallocated (NULL) when a graph only
stores K/V without attending -- e.g. DFlash speculative decoding's KV-injection
pass. set_input_k_rot then calls ggml_backend_buffer_is_host() on a NULL buffer
and aborts with GGML_ASSERT(buffer).
Guard the four k_rot/v_rot inputs with the same "&& ->buffer" check that the
adjacent kq_mask inputs already use in these two functions. When the buffer is
unallocated there is no data to upload, so skipping is correct.
Fixes ggml-org#25191
Signed-off-by: liminfei-amd <91481003+liminfei-amd@users.noreply.github.com>1 parent 6658925 commit a410713
1 file changed
Lines changed: 6 additions & 6 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
494 | 494 | | |
495 | 495 | | |
496 | 496 | | |
497 | | - | |
| 497 | + | |
498 | 498 | | |
499 | 499 | | |
500 | 500 | | |
501 | | - | |
| 501 | + | |
502 | 502 | | |
503 | 503 | | |
504 | 504 | | |
| |||
592 | 592 | | |
593 | 593 | | |
594 | 594 | | |
595 | | - | |
| 595 | + | |
596 | 596 | | |
597 | 597 | | |
598 | 598 | | |
599 | | - | |
| 599 | + | |
600 | 600 | | |
601 | 601 | | |
602 | 602 | | |
603 | | - | |
| 603 | + | |
604 | 604 | | |
605 | 605 | | |
606 | 606 | | |
607 | | - | |
| 607 | + | |
608 | 608 | | |
609 | 609 | | |
610 | 610 | | |
| |||
0 commit comments