Skip to content

Commit 82cf2dd

Browse files
victordionvcuinv
andauthored
[vllm, rollout] fix: Use different seeds for vllm (verl-project#5179)
### What does this PR do? The PR fixes the issue that all vLLM replicas using the same seed at startup, hindering its exploration capacity when vLLM runs in deterministic mode and dispatching logic is also deterministic. ### Test Passed. ### API and Usage Example N/A ### Design & Code Changes The PR fixes the issue that all vLLM replicas using the same seed at startup, hindering its exploration capacity when vLLM runs in deterministic mode and dispatching logic is also deterministic. Previously, it to worked because the request dispatching logic is indeterministic which causes vLLM replicas to go through different RNG trajectories, but the variability is far less than having different seeds at startup. ### Checklist Before Submitting > [!IMPORTANT] > Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review. - [x] Read the [Contribute Guide](https://github.com/volcengine/verl/blob/main/CONTRIBUTING.md). - [x] Apply [pre-commit checks](https://github.com/volcengine/verl/blob/main/CONTRIBUTING.md#code-linting-and-formatting): `pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=always` - [x] Add / Update [the documentation](https://github.com/volcengine/verl/tree/main/docs). - [x] Add unit or end-to-end test(s) to [the CI workflow](https://github.com/volcengine/verl/tree/main/.github/workflows) to cover all the code. If not feasible, explain why: ... - [x] Once your PR is ready for CI, send a message in [the `ci-request` channel](https://verl-project.slack.com/archives/C091TCESWB1) in [the `verl` Slack workspace](https://join.slack.com/t/verl-project/shared_invite/zt-3855yhg8g-CTkqXu~hKojPCmo7k_yXTQ). (If not accessible, please try [the Feishu group (飞书群)](https://applink.larkoffice.com/client/chat/chatter/add_by_link?link_token=772jd4f1-cd91-441e-a820-498c6614126a).) - [x] If your PR is related to the `recipe` submodule, please also update the reference to the submodule commit via `git submodule update --remote` or `cd recipe && git pull origin main`. Co-authored-by: Victor Cui <vcui@nvidia.com>
1 parent aaac32e commit 82cf2dd

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

verl/workers/rollout/vllm_rollout/vllm_async_server.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -282,7 +282,7 @@ async def launch_server(self, master_address: str = None, master_port: int = Non
282282
"gpu_memory_utilization": self.config.gpu_memory_utilization,
283283
"disable_log_stats": self.config.disable_log_stats,
284284
"tensor_parallel_size": self.config.tensor_model_parallel_size,
285-
"seed": self.config.get("seed", 0),
285+
"seed": self.replica_rank,
286286
"override_generation_config": json.dumps(override_generation_config),
287287
"quantization": quantization,
288288
"hf_overrides": hf_overrides,

0 commit comments

Comments
 (0)