Skip to content

Commit bddfd2b

Browse files
authored
server: refactor batch construction (#24843)
* server: refactor batch construction * wip * wip 2 * wip 3 * wip 4 * add abort_all_slots * handle batch full more carefully * fix assert * rm debug log * small nits * (debug) add timings * debug: force llama_synchronize for accurate timings * address comments * disable DEBUG_TIMINGS
1 parent 0d135df commit bddfd2b

1 file changed

Lines changed: 534 additions & 302 deletions

File tree

0 commit comments

Comments
 (0)