Commit bddfd2b
authored
server: refactor batch construction (#24843)
* server: refactor batch construction
* wip
* wip 2
* wip 3
* wip 4
* add abort_all_slots
* handle batch full more carefully
* fix assert
* rm debug log
* small nits
* (debug) add timings
* debug: force llama_synchronize for accurate timings
* address comments
* disable DEBUG_TIMINGS1 parent 0d135df commit bddfd2b
1 file changed
Lines changed: 534 additions & 302 deletions
0 commit comments