feat: add datasets_v2 module and SFT-V2 training pipeline#4435

Open

weiyixuanxx wants to merge 22 commits into

PaddlePaddle:developfrom

weiyixuanxx:dev_dataset_v2

Commits on May 12, 2026

feat: add datasets_v2 module and SFT-V2 training pipeline

weiyixuanxx
and
claude
committed

Commits on May 15, 2026

First version of changes: the YAML config file requires setting stage: SFT_v2. Not all features are supported yet, and the overall pipeline still needs to be reviewed.

weiyixuanxx
and
claude
committed

Commits on May 18, 2026

Commits on May 22, 2026

Commits on Jun 2, 2026

Merge branch 'dev_dataset_v2' of https://github.com/weiyixuanxx/PaddleFormers into dev_dataset_v2
weiyixuanxx
committed

Commits on Jun 3, 2026

feat(datasets_v2): add true streaming support and refine V2 pipeline

weiyixuanxx
and
claude
committed