Skip to content

Megatron-Swift利用mcore-bridge训练的模型保存检查点时没有存储lm_head.weight #9753

Description

@randydl

Checklist / 检查清单

  • I have searched existing issues, and this is a new bug report. / 我已经搜索过现有的 issues,确认这是一个新的 bug report。

Bug Description / Bug 描述

在tie_word_embeddings=False的情况下,应该存储lm_head.weight。DeepSpeed训练是正确的,可以看到model.safetensors.index.json里面是有的,但是用megatron训练是没有的。

How to Reproduce / 如何复现

如上,训练llava onevision 1.5模型

Additional Information / 补充信息

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions