Skip to content

GLM5.2 微调调用dsa的时候,不开重计算参数不对齐,开启重计算是正常的 #143

Description

@qq1243196045

Checklist / 检查清单

  • I have searched existing issues, and this is a new bug report. / 我已经搜索过现有的 issues,确认这是一个新的 bug report。

Bug Description / Bug 描述

Image 如图,开启重计算的时候会走else分支,此时会判断是否为dsa并传入x 和qr,而不开重计算的时候不会传入,并且_checkpointed_attention_forward中也没有判断dsa的逻辑,导致调用接口失败 报错如下: Image

How to Reproduce / 如何复现

微调glm5.2,并且不开启recompute参数
--recompute_granularity full
--recompute_method uniform
--recompute_num_layers 1
若增加以上三个参数,则可以规避该问题

Additional Information / 补充信息

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions