add qwen35 rl config and fix mix bug

[hhaAndroid]

No description provided.

[hhaAndroid]

@claude review

[claude]

Claude: ## Summary

This PR adds a Qwen3.5-VL 35B GRPO mixed-data RL config, a new "qwen3.5-vl-rl" chat template (with <think> tag for chain-of-thought), configurable template_name in Qwen3VLTokenizeFnConfig, and a bug fix in replay_buffer.py.

Issues

Critical

• qwen3_vl_tokenize_fn.py:908 — Default template_name is "qwen3_vl" (underscore) but CHAT_TEMPLATE_MAP uses "qwen3-vl" (hyphen). This will cause a KeyError at runtime for anyone using the default.

Nit

• Config file: unclosed file handle (open() without context manager), mid-file import, Chinese comment, PEP 8 spacing.

Bug Fix Review

The replay_buffer.py fix is correct — multimodal_train_info must be assigned back to data regardless of whether pixel_values is present, otherwise mixed text+image data loses position IDs.

Verdict

REQUEST_CHANGES — the template_name default value mismatch is a runtime bug that needs fixing.

[hhaAndroid]

@claude review

[claude]

Claude: Re-review of updated PR (3 commits).

Summary

This PR adds a Qwen3.5-VL 35B GRPO mixed-data RL config, a new "qwen3-vl-rl" chat template (with <think> tag for chain-of-thought), renames template_name → chat_template across tokenize functions, adds chat_template to BaseMLLMTokenizeFnConfig, and fixes a bug in replay_buffer.py where mixed text+image data lost multimodal metadata.

Issues

Warning

• base_mllm_tokenize_fn.py:263 — chat_template is added as a required field (no default) to the base config. The two in-tree subclasses override with defaults so they're safe, but this is a breaking change for any out-of-tree subclass. Also, the Literal type enumerating all templates creates tight coupling with the template registry — a plain str validated at runtime may be more maintainable.

Nit

• rl_qwen3p5_vl_35B_grpo_mixdata.py:1-8 — Import ordering (PEP 8): import json should be grouped with stdlib imports.
• rl_qwen3p5_vl_35B_grpo_mixdata.py:74 — Unclosed file handle (open() without context manager). Suggest with open(...) as f: json.load(f).
• templates/__init__.py:57 — The qwen3-vl-rl template forces <think> mode and carries unused tool fields — a brief comment explaining the design choice would help future readers.

Bug Fix ✓

• replay_buffer.py:280-283 — The fix is correct. De-denting data["multimodal_train_info"] = multimodal_train_info ensures metadata (e.g., position_id) is preserved even when pixel_values is absent.

Previous Issue Resolved ✓

• The template_name default value mismatch ("qwen3_vl" underscore vs "qwen3-vl" hyphen) flagged in the previous review has been fixed by the rename to chat_template with correct defaults.

Verdict

COMMENT — The previous critical issue is resolved. The remaining items are warnings and nits. The BaseMLLMTokenizeFnConfig coupling concern is worth considering but not blocking.

[hhaAndroid]

@claude please approve it

[claude]

All previous critical issues have been resolved. The template_name → chat_template rename with correct defaults fixes the runtime KeyError, and the replay_buffer bug fix is correct. Remaining items are minor nits. LGTM.

[claude]

Claude: Approved the PR as requested. All previously flagged critical issues (template_name default mismatch) have been resolved in the latest commits. The replay_buffer bug fix is correct, and the remaining items are minor nits only.