Skip to content

Commit 6de7e39

Browse files
committed
feat: add comprehensive training profiling reports and updates to 1B model benchmark scripts and core NN components
1 parent 0120b7f commit 6de7e39

45 files changed

Lines changed: 209121 additions & 290 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

cppmega_mlx/models/hybrid_lm.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -470,6 +470,7 @@ def __call__(
470470
*,
471471
kv_cache: ContiguousKVCache | None = None,
472472
attention_layer_idx: int | None = None,
473+
doc_ids: mx.array | None = None,
473474
) -> mx.array:
474475
self.validate_backend()
475476
residual = hidden_states
@@ -478,6 +479,7 @@ def __call__(
478479
mask,
479480
kv_cache=kv_cache,
480481
attention_layer_idx=attention_layer_idx,
482+
doc_ids=doc_ids,
481483
)
482484
updated = residual + delta
483485
if self.mhc is not None:

0 commit comments

Comments
 (0)