Add InternLM model and tokenizer support by fjaijf · Pull Request #4495 · PaddlePaddle/PaddleFormers

fjaijf · 2026-05-20T01:26:26Z

Before submitting

Lint code. If there are lint issues, please format the code first.

# Install and register `pre-commit` in the project folder
pip install pre-commit && pre-commit install

# Process previous code files separately
pre-commit run --file XXXX.py

Add test cases into tests folder. If there are codecov issues, please add tests cases first.

PR types

PR changes

Description

Add InternLM model and tokenizer support

模型前向精度对齐，前10个token的logits diff为0.08
模型推理生成正常
模型sft loss为
[2026-05-20 01:25:59,358] [ INFO] - loss: 2.46186185, learning_rate: 5e-06, global_step: 1, current_memory_allocated: 13.973276853561401, current_memory_reserved: 26.491707801818848, max_memory_allocated: 26.48780107498169, max_memory_reserved: 26.491707801818848, interval_runtime: 12.092, interval_samples_per_second: 0.3308, interval_steps_per_second: 0.0827, ppl: 11.726624443222343, progress_or_epoch: 0.3333, cpu_used_memory: 16.1, cpu_available_memory: 30.93
[2026-05-20 01:25:59,358] [ INFO] - [DataLoad global_step: 1] data_load_time: 44.99 ms (accumulated over 4 micro-batches)
[2026-05-20 01:25:59,360] [ INFO] - Not using packing mode for data iteration.
[2026-05-20 01:26:00,303] [ INFO] - loss: 2.7151227, learning_rate: 1e-05, global_step: 2, current_memory_allocated: 13.973276853561401, current_memory_reserved: 26.491707801818848, max_memory_allocated: 26.48780107498169, max_memory_reserved: 26.491707801818848, interval_runtime: 0.9452, interval_samples_per_second: 4.2321, interval_steps_per_second: 1.058, ppl: 15.106463521258013, progress_or_epoch: 0.6667, cpu_used_memory: 16.1, cpu_available_memory: 30.92
[2026-05-20 01:26:00,303] [ INFO] - [DataLoad global_step: 2] data_load_time: 0.66 ms (accumulated over 4 micro-batches)
[2026-05-20 01:26:01,250] [ INFO] - loss: 2.56071281, learning_rate: 1.5e-05, global_step: 3, current_memory_allocated: 13.973276853561401, current_memory_reserved: 26.491707801818848, max_memory_allocated: 26.48780107498169, max_memory_reserved: 26.491707801818848, interval_runtime: 0.9468, interval_samples_per_second: 4.2245, interval_steps_per_second: 1.0561, ppl: 12.945041382596827, progress_or_epoch: 1.0, cpu_used_memory: 16.11, cpu_available_memory: 30.91

paddle-bot · 2026-05-20T01:26:32Z

Thanks for your contribution!

Paddle-CI-Bot · 2026-05-20T02:06:08Z

PaddleFormers Log Analysis

Run #28766137471 · Attempt 1

日志分析报告

流水线名称	问题标签	修复建议	日志片段
Unittest GPU CI	其他（streamer 输出不一致）	本次 PR 未改动 `generate()` 或 streamer 相关代码，与 PR 本身无直接关联，建议 rerun 确认是否偶发；若仍失败需排查 `input_kwargs` 张量是否被第一次 `generate()` 调用 in-place 修改导致第二次 `skip_prompt` 跳过 token 数不匹配	报错代码

失败的测试case:

tests/generation/test_streamers.py::StreamerTester::test_text_streamer_matches_non_streaming

根本原因分析:

PR 仅新增 InternLM 模型实现（internlm/modeling.py、configuration.py、tokenizer.py）并将其注册到 AutoConfig/AutoModelForCausalLM，未触碰 generate() 或 TextStreamer 路径。失败断言为 ' NASA' != 'ਸ̄ founderczasT poorfahrшла\r NASA'，streamer_text 只是 greedy_text 的尾部子串，说明第二次 model.generate(**input_kwargs, streamer=streamer) 调用时 input_kwargs 中的 input_ids 张量可能被第一次调用 in-place 截断（PaddleFormers generate() 在某些路径下会修改传入的 input_ids 视图），导致 skip_prompt 少跳了 token，streamer 只输出了末尾部分。该问题与本 PR 新增的 InternLM 代码无因果关系。

修复建议:

先 rerun 本次 CI，确认是否偶发。
若持续复现，在 test_text_streamer_matches_non_streaming 中对 input_kwargs 做深拷贝再传入第二次调用：
```
import copy
model.generate(**copy.deepcopy(input_kwargs), streamer=streamer)
```
或在 generate() 内部确保不 in-place 修改传入的 input_ids，保持两次调用输入完全一致。

🔍 准确性记录：请点击评论底部 😊 图标，选择 👍（准确）或 👎（有误），将自动记录到 CI 监控系统

_{🔄 每次 Re-run 后自动更新}

codecov-commenter · 2026-05-21T03:16:47Z

Codecov Report

❌ Patch coverage is 64.74227% with 171 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (develop@08d8839). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
paddleformers/transformers/internlm/tokenizer.py	0.00%	98 Missing ⚠️
paddleformers/transformers/internlm/modeling.py	79.81%	67 Missing ⚠️
...eformers/transformers/internlm/auto_dist_config.py	33.33%	4 Missing ⚠️
...ddleformers/transformers/internlm/configuration.py	95.45%	2 Missing ⚠️

❌ Your patch status has failed because the patch coverage (64.74%) is below the target coverage (75.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff             @@
##             develop    #4495   +/-   ##
==========================================
  Coverage           ?   46.53%           
==========================================
  Files              ?      480           
  Lines              ?    91124           
  Branches           ?        0           
==========================================
  Hits               ?    42409           
  Misses             ?    48715           
  Partials           ?        0

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

fjaijf · 2026-06-30T07:25:37Z

/re-run all-failed

zjjlivein · 2026-06-30T07:30:20Z

/re-run all-failed

fjaijf · 2026-06-30T08:27:53Z

/re-run all-failed

liuhao2638

已完成这轮代码审查，发现几处需要合入前修复的问题，具体细节已放在行级评论里。

优先级：P3 非行级：PR 标题/描述不是 diff 中的具体代码行。当前标题 internlm test 只表达测试，但实际改动是新增 InternLM config/model/tokenizer/Auto 映射支持，容易让维护者误判变更范围。建议将标题改为 Add InternLM model and tokenizer support，并在描述中补充支持范围、主要测试项、已知风险或 CI 依赖状态。

^{Powered by Nyanpasu with gpt-5.5 xhigh, please check the suggestions carefully.}

liuhao2638 · 2026-07-03T04:52:29Z

+    "internlm.configuration": ["InternLMConfig"],
+    "internlm.modeling": [
+        "InternLMModel",
+        "InternLMForCausalLM",
+        "InternLMForCausalLMPipe",
+    ],
+    "internlm.tokenizer": ["InternLMTokenizer"],


优先级：P1

这里新增 InternLM 映射时把原来的 ernie4_5.tokenizer 条目删掉了。paddleformers.transformers.ernie4_5.__init__ 仍导出 Ernie4_5Tokenizer，删除顶层 import_structure 后，from paddleformers.transformers import Ernie4_5Tokenizer 以及 AutoTokenizer 的本地类查找都会找不到该已有公共类，属于兼容性回归。请保留 ERNIE 4.5 tokenizer 条目，再追加 InternLM 条目。

Suggested change

"internlm.configuration": ["InternLMConfig"],

"internlm.modeling": [

"InternLMModel",

"InternLMForCausalLM",

"InternLMForCausalLMPipe",

],

"internlm.tokenizer": ["InternLMTokenizer"],

"ernie4_5.tokenizer": ["Ernie4_5Tokenizer"],

"internlm.configuration": ["InternLMConfig"],

"internlm.modeling": [

"InternLMModel",

"InternLMForCausalLM",

"InternLMForCausalLMPipe",

],

"internlm.tokenizer": ["InternLMTokenizer"],

liuhao2638 · 2026-07-03T04:52:55Z

+        if token_ids_1 is None:
+            return [1] + ([0] * len(token_ids_0)) + [1]
+        return [1] + ([0] * len(token_ids_0)) + [1, 1] + ([0] * len(token_ids_1)) + [1]


优先级：P1

build_inputs_with_special_tokens() 会根据 add_bos_token/add_eos_token 决定实际插入哪些特殊 token，但这里固定返回 [BOS] + tokens + [EOS] 的 mask。默认 add_eos_token=False 时，单句输入实际长度是 len(token_ids_0)+1，这里却返回 len(token_ids_0)+2；pair 输入也同样多标了不存在的 EOS/BOS，return_special_tokens_mask=True 或数据 collator 使用该 mask 时会和 input_ids 长度不一致。请让 mask 按同一组开关构造。

Suggested change

if token_ids_1 is None:

return [1] + ([0] * len(token_ids_0)) + [1]

return [1] + ([0] * len(token_ids_0)) + [1, 1] + ([0] * len(token_ids_1)) + [1]

mask = ([1] if self.add_bos_token else []) + ([0] * len(token_ids_0))

if token_ids_1 is not None:

mask += [0] * len(token_ids_1)

if self.add_eos_token:

mask += [1]

return mask

liuhao2638 · 2026-07-03T04:54:00Z

+        if past_key_values is not None:
+            key_states, value_states = past_key_values.update(key_states, value_states, self.layer_idx)
+
+        cos, sin = position_embeddings
+        query_states, key_states = apply_rotary_pos_emb(query_states, key_states, cos, sin, position_ids)


优先级：P1

这里先把未加 RoPE 的 key_states 写入 DynamicCache.update()，再对 update() 返回的整段 key 统一按当前 position_ids 做 RoPE，会破坏缓存生成。DynamicLayer.update() 会把本次 key 拼到缓存并返回完整 self.keys，下一步 decode 时 key_states.shape[-2] 已经是 past+1，而 position_ids 只包含当前 token；apply_rotary_pos_emb() 的 fallback 会从 cos[:, :k_len] 取位置 0..past，导致历史 key 被重新用错误位置旋转，且缓存中保存的仍是未旋转 key。请按 Llama/Qwen 的顺序先只对本步 q/k 应用 RoPE，再把已经旋转后的 k 写入 cache。

Suggested change

if past_key_values is not None:

key_states, value_states = past_key_values.update(key_states, value_states, self.layer_idx)

cos, sin = position_embeddings

query_states, key_states = apply_rotary_pos_emb(query_states, key_states, cos, sin, position_ids)

cos, sin = position_embeddings

query_states, key_states = apply_rotary_pos_emb(query_states, key_states, cos, sin, position_ids)

if past_key_values is not None:

key_states, value_states = past_key_values.update(key_states, value_states, self.layer_idx)

risemeup1111

已复查新提交，当前 diff 中可见的代码问题已修复，未发现新的阻塞性问题。本地环境缺少 paddle，无法运行新增 InternLM 单测；目前 CI 还有任务运行中，建议等待结果完成后再合入。

优先级：P3 非行级：PR 标题/描述不是 diff 中的具体代码行。当前标题仍是 internlm test，与实际新增 InternLM config/model/tokenizer/Auto 映射支持不够匹配。建议改为 Add InternLM model and tokenizer support，并在描述中补充支持范围、主要测试项、已知风险或 CI 状态。

^{Powered by Nyanpasu with gpt-5.5 xhigh, please check the suggestions carefully.}

risemeup1111

已复查当前状态：PR 标题已更新为与变更内容匹配的描述，之前可见的代码问题已在后续提交中修复，当前检查均已通过。未发现需要继续阻塞合入的问题。

^{Powered by Nyanpasu with gpt-5.5 xhigh, please check the suggestions carefully.}

internlm test

c69eea5

paddle-bot Bot added the contributor label May 20, 2026

bug fixed

8422401

fjaijf added 2 commits June 8, 2026 15:56

fix internlm bug

c97dd88

fix a little bug

88d1605

Merge remote-tracking branch 'origin/develop' into feat/internlm_test

36a74e0

fjaijf added 3 commits July 2, 2026 19:01

trigger ci

c5eb83f

trigger ci

b0ff5d8

trigger ci

011e501

liuhao2638 suggested changes Jul 3, 2026

View reviewed changes

fix according to reveiw

8467ff3

risemeup1111 reviewed Jul 6, 2026

View reviewed changes

trigger ci

543dd67

fjaijf changed the title ~~internlm test~~ Add InternLM model and tokenizer support Jul 9, 2026

risemeup1111 approved these changes Jul 9, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add InternLM model and tokenizer support#4495

Add InternLM model and tokenizer support#4495
fjaijf wants to merge 10 commits into
PaddlePaddle:developfrom
fjaijf:feat/internlm_test

fjaijf commented May 20, 2026 •

edited

Loading

Uh oh!

paddle-bot Bot commented May 20, 2026

Uh oh!

Paddle-CI-Bot commented May 20, 2026 •

edited

Loading

Uh oh!

codecov-commenter commented May 21, 2026

Uh oh!

fjaijf commented Jun 30, 2026

Uh oh!

zjjlivein commented Jun 30, 2026

Uh oh!

fjaijf commented Jun 30, 2026

Uh oh!

liuhao2638 left a comment

Uh oh!

liuhao2638 Jul 3, 2026

Uh oh!

liuhao2638 Jul 3, 2026

Uh oh!

liuhao2638 Jul 3, 2026

Uh oh!

risemeup1111 left a comment

Uh oh!

risemeup1111 left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

-        if token_ids_1 is None:
-            return [1] + ([0] * len(token_ids_0)) + [1]
-        return [1] + ([0] * len(token_ids_0)) + [1, 1] + ([0] * len(token_ids_1)) + [1]
+        mask = ([1] if self.add_bos_token else []) + ([0] * len(token_ids_0))
+        if token_ids_1 is not None:
+            mask += [0] * len(token_ids_1)
+        if self.add_eos_token:
+            mask += [1]
+        return mask

Uh oh!

Conversation

fjaijf commented May 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Before submitting

PR types

PR changes

Description

Add InternLM model and tokenizer support

Uh oh!

paddle-bot Bot commented May 20, 2026

Uh oh!

Paddle-CI-Bot commented May 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PaddleFormers Log Analysis

日志分析报告

Uh oh!

codecov-commenter commented May 21, 2026

Codecov Report

Uh oh!

fjaijf commented Jun 30, 2026

Uh oh!

zjjlivein commented Jun 30, 2026

Uh oh!

fjaijf commented Jun 30, 2026

Uh oh!

liuhao2638 left a comment

Choose a reason for hiding this comment

Uh oh!

liuhao2638 Jul 3, 2026

Choose a reason for hiding this comment

Uh oh!

liuhao2638 Jul 3, 2026

Choose a reason for hiding this comment

Uh oh!

liuhao2638 Jul 3, 2026

Choose a reason for hiding this comment

Uh oh!

risemeup1111 left a comment

Choose a reason for hiding this comment

Uh oh!

risemeup1111 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

fjaijf commented May 20, 2026 •

edited

Loading

Paddle-CI-Bot commented May 20, 2026 •

edited

Loading