Skip to content

Commit 0c9a6e1

Browse files
kmbandyclaude
andcommitted
feat(wp): break batch range at routing boundaries under WP_PAGED_BATCH (fixes H3/H4)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014ZRfPpL8XFzk1hep9MMg9P
1 parent e5cb058 commit 0c9a6e1

1 file changed

Lines changed: 9 additions & 0 deletions

File tree

src/weight-pager/wp-eval-cb.cpp

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -246,7 +246,16 @@ bool weight_pager_eval_cb(struct ggml_tensor * t, bool ask, void * user_data) {
246246
const uint64_t sync_fallbacks_before =
247247
batch_eval_cb ? pager->sync_fallback_count() : 0;
248248
bool routing_tls_set = false;
249+
const bool paged_batch = batch_eval_cb && wp_paged_batch_enabled();
249250
auto eval_cb_op_return = [&]() -> bool {
251+
// WP_PAGED_BATCH: break the batch range at every routing boundary
252+
// (each MUL_MAT_ID and its ids-producer). Ending the range here forces
253+
// the scheduler's compute+sync, so the router's ids are materialized
254+
// before the next range reads them, and the routing op runs isolated
255+
// (fixes the read-before-produce H3 + TLS take-steal H4 faults).
256+
if (paged_batch && pager->is_routing_break(t)) {
257+
return true;
258+
}
250259
if (batch_eval_cb &&
251260
pager->batch_safe() &&
252261
!routing_tls_set &&

0 commit comments

Comments
 (0)