Commit e6ca062
authored
[Fix] Update KVCacheLayout to set row counts. (#920)
## Purpose
Fix the `IndexError: list index out of range` issue when running
`GLM-4.7-W8A8-floatmtp`.
## Modification
Use the number of unique layer IDs,
`len(set(self.layer_name_to_id.values()))`, as the row count in KV cache
layout construction instead of `self.local_num_hidden_layers`.1 parent 42afd8b commit e6ca062
1 file changed
Lines changed: 4 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
88 | 88 | | |
89 | 89 | | |
90 | 90 | | |
91 | | - | |
92 | | - | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
93 | 95 | | |
94 | 96 | | |
95 | 97 | | |
| |||
0 commit comments