Skip to content

Use released OSim tokenizer for SFT#20

Draft
sunnweiwei wants to merge 1 commit into
mainfrom
agent/qwen3-no-template-followup
Draft

Use released OSim tokenizer for SFT#20
sunnweiwei wants to merge 1 commit into
mainfrom
agent/qwen3-no-template-followup

Conversation

@sunnweiwei

@sunnweiwei sunnweiwei commented Jul 8, 2026

Copy link
Copy Markdown
Owner

This switches the SFT reproduction path away from code-level chat-template rewriting.

Changes:

  • Reverts the Qwen3 chat-template override in sft/dataset.py.
  • Makes run_sft.sh set actor_rollout_ref.model.tokenizer_path to the released OSim mid tokenizer by model size (cmu-lti/osim-4b-mid for Qwen3-4B, cmu-lti/osim-8b-mid for Qwen3-8B), while keeping ACTOR_TOKENIZER_PATH override support.
  • Makes the SFT dataloader and SFT actor checkpoint metadata honor tokenizer_path, so saved checkpoints keep the selected tokenizer/chat template.

Validation:

  • bash -n run_sft.sh
  • python3 -m py_compile train_sft.py sft/dataset.py sft/sft_data.py sft/fsdp_worker.py sft/trainer.py
  • Docker tokenizer check with Qwen/Qwen3-4B-Base + cmu-lti/osim-4b-mid: prefix-stable tokenization passes.
  • Docker 8-GPU 1-step SFT smoke with Qwen/Qwen3-4B-Base + cmu-lti/osim-4b-mid: training and checkpoint save completed; saved tokenizer template matches cmu-lti/osim-4b-mid, with no last_query_index/<think>.

@sunnweiwei
sunnweiwei force-pushed the agent/qwen3-no-template-followup branch from ee49075 to fc6e00b Compare July 8, 2026 20:08
@sunnweiwei sunnweiwei changed the title Preserve Qwen3 chat template during SFT Use Qwen3 non-thinking chat template for SFT Jul 8, 2026
@sunnweiwei
sunnweiwei force-pushed the agent/qwen3-no-template-followup branch from fc6e00b to 13f85e7 Compare July 8, 2026 20:31
@sunnweiwei sunnweiwei changed the title Use Qwen3 non-thinking chat template for SFT Use Qwen3 instruct chat template for SFT Jul 8, 2026
@sunnweiwei
sunnweiwei force-pushed the agent/qwen3-no-template-followup branch from 13f85e7 to 47370e8 Compare July 8, 2026 20:43
@sunnweiwei
sunnweiwei force-pushed the agent/qwen3-no-template-followup branch from 47370e8 to fe570ab Compare July 8, 2026 23:35
@sunnweiwei sunnweiwei changed the title Use Qwen3 instruct chat template for SFT Use released OSim tokenizer for SFT Jul 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant