Commit 6653f64
fix(context): keep n_outputs == n_tokens after n_seqs rounding for mean/rank pooling
graph_reserve() rounds n_tokens up to a multiple of n_seqs, but callers
(e.g. the pp reserve via n_outputs_pp) compute n_outputs from the
unrounded n_tokens. For embedding models with mean/rank pooling,
build_pooling multiplies t_embd (reduced to [n_outputs,...] via out_ids)
by inp_mean of shape [n_tokens, n_seqs], which requires n_outputs ==
n_tokens; the rounding broke that invariant and tripped
GGML_ASSERT(ggml_can_mul_mat) during graph reserve (e.g. llama-embed /
nemotron-embed at n_ubatch=1280, n_seqs=3 -> n_tokens rounded to 1281).
Re-establish the invariant after rounding.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>1 parent 100bce6 commit 6653f64
1 file changed
Lines changed: 10 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2287 | 2287 | | |
2288 | 2288 | | |
2289 | 2289 | | |
| 2290 | + | |
| 2291 | + | |
| 2292 | + | |
| 2293 | + | |
| 2294 | + | |
| 2295 | + | |
| 2296 | + | |
| 2297 | + | |
| 2298 | + | |
| 2299 | + | |
2290 | 2300 | | |
2291 | 2301 | | |
2292 | 2302 | | |
| |||
0 commit comments