Skip to content

Commit a2bfa13

Browse files
[feat] train: add keep_checkpoint_max config knob
Add TrainConfig.keep_checkpoint_max (default 0 = keep all) so long runs can cap the number of retained model.ckpt-<step> directories. Documented in usage/train.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 0917e22 commit a2bfa13

2 files changed

Lines changed: 3 additions & 0 deletions

File tree

docs/source/usage/train.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,7 @@ LR策略可以支持按epoch更新或者按step更新
6363
- num_epochs: 训练的epoch数
6464
- save_checkpoints_steps: 保存模型的步数间隔,保存模型后会做一次评估
6565
- save_checkpoints_epochs: 保存模型的Epoch数间隔,保存模型后会做一次评估,与save_checkpoints_steps不能同时设置
66+
- keep_checkpoint_max: 最多保留的最近checkpoint数量,超出后会异步删除最旧的checkpoint,默认0表示全部保留;当exporter_type为best时,会额外保留指标最优的checkpoint
6667
- fine_tune_checkpoint: 增量训练的checkpoint路径,也可以设置checkpoint目录,将会使用目录下最新的checkpoint
6768
- fine_tune_ckpt_var_map: 需要restore的参数列表文件路径,文件的每一行是{variable_name in current model}\\t{variable name in old model ckpt}
6869
- 需要设置fine_tune_ckpt_var_map的情形:

tzrec/protos/train.proto

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,5 +70,7 @@ message TrainConfig {
7070
optional uint32 gradient_accumulation_steps = 18;
7171
// dense gradient clipping config
7272
optional GradClipping grad_clipping = 19;
73+
// maximum number of recent checkpoints to keep; 0 keeps all.
74+
optional uint32 keep_checkpoint_max = 20 [default = 0];
7375
// TBD: qcomm config
7476
}

0 commit comments

Comments
 (0)