Commit 82cf2dd
[vllm, rollout] fix: Use different seeds for vllm (verl-project#5179)
### What does this PR do?
The PR fixes the issue that all vLLM replicas using the same seed at
startup, hindering its exploration capacity when vLLM runs in
deterministic mode and dispatching logic is also deterministic.
### Test
Passed.
### API and Usage Example
N/A
### Design & Code Changes
The PR fixes the issue that all vLLM replicas using the same seed at
startup, hindering its exploration capacity when vLLM runs in
deterministic mode and dispatching logic is also deterministic.
Previously, it to worked because the request dispatching logic is
indeterministic which causes vLLM replicas to go through different RNG
trajectories, but the variability is far less than having different
seeds at startup.
### Checklist Before Submitting
> [!IMPORTANT]
> Please check all the following items before requesting a review,
otherwise the reviewer might deprioritize this PR for review.
- [x] Read the [Contribute
Guide](https://github.com/volcengine/verl/blob/main/CONTRIBUTING.md).
- [x] Apply [pre-commit
checks](https://github.com/volcengine/verl/blob/main/CONTRIBUTING.md#code-linting-and-formatting):
`pre-commit install && pre-commit run --all-files --show-diff-on-failure
--color=always`
- [x] Add / Update [the
documentation](https://github.com/volcengine/verl/tree/main/docs).
- [x] Add unit or end-to-end test(s) to [the CI
workflow](https://github.com/volcengine/verl/tree/main/.github/workflows)
to cover all the code. If not feasible, explain why: ...
- [x] Once your PR is ready for CI, send a message in [the `ci-request`
channel](https://verl-project.slack.com/archives/C091TCESWB1) in [the
`verl` Slack
workspace](https://join.slack.com/t/verl-project/shared_invite/zt-3855yhg8g-CTkqXu~hKojPCmo7k_yXTQ).
(If not accessible, please try [the Feishu group
(飞书群)](https://applink.larkoffice.com/client/chat/chatter/add_by_link?link_token=772jd4f1-cd91-441e-a820-498c6614126a).)
- [x] If your PR is related to the `recipe` submodule, please also
update the reference to the submodule commit via `git submodule update
--remote` or `cd recipe && git pull origin main`.
Co-authored-by: Victor Cui <vcui@nvidia.com>1 parent aaac32e commit 82cf2dd
1 file changed
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
282 | 282 | | |
283 | 283 | | |
284 | 284 | | |
285 | | - | |
| 285 | + | |
286 | 286 | | |
287 | 287 | | |
288 | 288 | | |
| |||
0 commit comments