Skip to content

Commit 3c7583e

Browse files
stefpiawni
andauthored
remove redundant repeats definition
Co-authored-by: Awni Hannun <awni.hannun@gmail.com>
1 parent 704bab7 commit 3c7583e

1 file changed

Lines changed: 0 additions & 1 deletion

File tree

llms/llama/llama.py

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,6 @@ def __init__(self, args: ModelArgs):
5555
def shard(self, group: mx.distributed.Group):
5656
self.n_heads = self.n_heads // group.size()
5757
self.n_kv_heads = self.n_kv_heads // group.size()
58-
self.repeats = self.n_heads // self.n_kv_heads
5958

6059
self.wq = shard_linear(self.wq, "all-to-sharded", group=group)
6160
self.wk = shard_linear(self.wk, "all-to-sharded", group=group)

0 commit comments

Comments
 (0)