Skip to content

Commit 285dcec

Browse files
committed
Fix IndexError with DeepSpeed ZeRO-3 when kernels rotary is active (#45414)
* Fix `IndexError: pop from an empty deque` under DeepSpeed ZeRO-3 When `kernels` is installed, `@use_kernelized_func` attaches a `rotary_fn` child `nn.Module` to attention layers. DeepSpeed ZeRO-3's parameter coordinator traces the module graph at init and expects every registered submodule to be invoked during forward. The model's forward still calls the plain Python `apply_rotary_pos_emb`, so `rotary_fn` is never executed and the trace desynchronizes, raising `IndexError: pop from an empty deque` on the second forward. Skip attaching the kernelized submodule when ZeRO-3 is enabled; users running under ZeRO-3 fall back to the Python implementation, which is what they were getting before #41147. Fixes #45137 * Add dates to new model cards to satisfy check-repository-consistency
1 parent 16f5c6c commit 285dcec

4 files changed

Lines changed: 12 additions & 3 deletions

File tree

docs/source/en/model_doc/pp_chart2table.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ specific language governing permissions and limitations under the License.
1313
rendered properly in your Markdown viewer.
1414
1515
-->
16-
*This model was released on 2025-05-20 and added to Hugging Face Transformers on 2026-03-18.*
16+
*This model was released on 2025-05-20 and added to Hugging Face Transformers on 2026-03-20.*
1717

1818
# PP-Chart2Table
1919

docs/source/en/model_doc/slanext.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ specific language governing permissions and limitations under the License.
1313
rendered properly in your Markdown viewer.
1414
1515
-->
16-
*This model was released on 2025-03-07 and added to Hugging Face Transformers on 2026-03-19.*
16+
*This model was released on 2025-03-07 and added to Hugging Face Transformers on 2026-03-21.*
1717

1818
# SLANeXt
1919

docs/source/en/model_doc/uvdoc.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ specific language governing permissions and limitations under the License.
1313
rendered properly in your Markdown viewer.
1414
1515
-->
16-
*This model was released on 2023-02-06 and added to Hugging Face Transformers on 2026-03-19.*
16+
*This model was released on 2023-02-06 and added to Hugging Face Transformers on 2026-03-21.*
1717

1818
# UVDoc
1919

src/transformers/integrations/hub_kernels.py

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -452,6 +452,15 @@ def decorator(cls):
452452

453453
def new_init(self, *args, **kwargs):
454454
orig_init(self, *args, **kwargs)
455+
# Skip attaching the kernelized submodule under DeepSpeed ZeRO-3: the coordinator traces
456+
# the module graph at init time, and a child `nn.Module` that is not actually invoked
457+
# during forward (e.g. when the model keeps calling the plain Python `apply_rotary_pos_emb`)
458+
# breaks the parameter fetch trace and raises `IndexError: pop from an empty deque`.
459+
# See https://github.com/huggingface/transformers/issues/45137
460+
from .deepspeed import is_deepspeed_zero3_enabled
461+
462+
if is_deepspeed_zero3_enabled():
463+
return
455464
for fn in module_names:
456465
# we hardcode the name of the function to "rotary_fn" for now
457466
setattr(self, "rotary_fn", fn)

0 commit comments

Comments
 (0)