Perhaps there are some issues in the model forward part.

[zhangshan-zs94]

Firstly, thank you for your work.
It appears that past_key_values is being reset here, which results in super().forward() having no information to work with. I believe this part might need to be removed.

OpenCoconut/opencoconut/models/qwen2.py

Line 292 in 5519bf4

# FIXME: cannot reuse past_key_values from generating thoughts

Another point supporting this view is that the handling of past_key_values in infer_forward is not aligned with the current train_forward.