File tree Expand file tree Collapse file tree 1 file changed +1
-1
lines changed
Expand file tree Collapse file tree 1 file changed +1
-1
lines changed Original file line number Diff line number Diff line change @@ -78,7 +78,7 @@ GraphGen 首先根据源文本构建细粒度的知识图谱,然后利用期
7878## GraphGen的效果
7979### Pretrain
8080
81- 受 Kimi-K2 的 技术报告 (https://arxiv.org/pdf/2507.20534 ) (Improving Token Utility with Rephrasing) 和 ByteDance Seed 的 [ Reformulation for Pretraining Data Augmentation] ( https://arxiv.org/abs/2507.15752 ) (MGA 框架)启发,GraphGen 引入了一套** 重述流水线(rephrase pipeline)** ——利用大语言模型对语料进行改写,生成同一知识内容的多种表达变体,替代传统的简单重复训练。
81+ 受 Kimi-K2 的 技术报告 (https://arxiv.org/pdf/2507.20534 ) (Improving Token Utility with Rephrasing) 和 ByteDance Seed 的 [ Reformulation for Pretraining Data Augmentation] ( https://arxiv.org/abs/2502.04235 ) (MGA 框架)启发,GraphGen 引入了一套** 重述流水线(rephrase pipeline)** ——利用大语言模型对语料进行改写,生成同一知识内容的多种表达变体,替代传统的简单重复训练。
8282
8383** 实验设置:** 使用 Qwen3-0.6B 模型,基于 [ SlimPajama-6B] ( https://huggingface.co/datasets/DKYoon/SlimPajama-6B ) 数据集从头训练。
8484
You can’t perform that action at this time.
0 commit comments