[Data][LLM] Fix max_pending_requests default to track vLLM's GPU-dependent max_num_seqs

[Aydin-ab]

Summary

vLLMEngineStageUDF computed its default max_pending_requests from engine_kwargs.get("max_num_seqs", 128) * pp_size * 1.1. The hardcoded 128 fallback is stale — vLLM's actual default for max_num_seqs is GPU-dependent via AsyncEngineArgs.get_batch_defaults (vllm/engine/arg_utils.py):

GPU	vLLM default	Ray assumed
A10G / other <70 GiB	256	128
A100 80 GiB (explicitly excluded from large-GPU path)	256	128
H100 / MI300x (≥70 GiB, non-A100)	1024	128
CPU	256 × world_size	128

When users don't set max_num_seqs explicitly (the common case), the semaphore silently caps inflight requests far below vLLM's real capacity — e.g. ~141 vs 1024 on H100, ~14% utilization.

Fix

Move the default resolution into vLLMEngineWrapper, which already calls AsyncEngineArgs.create_engine_config() and has access to the resolved scheduler_config.max_num_seqs and parallel_config.pipeline_parallel_size. The UDF passes max_pending_requests=None through and reads the resolved value back.

Semantics:

• max_pending_requests=None (default): auto-resolve from vLLM's resolved engine config
• positive int: explicit limit (unchanged)
• non-positive (e.g. -1): disable semaphore (unchanged)

This aligns with ProcessorConfig.max_pending_requests's stated intent: "If not specified, will use the default value from the backend engine."

Test plan

• Existing test_vllm_engine_udf_basic updated to reflect that the UDF now reads max_pending_requests from the (mocked) wrapper and passes None through when the caller didn't supply a value.
• Other tests that pass explicit max_pending_requests=10 are unaffected (positive int path unchanged).
• test_vllm_wrapper_semaphore exercises max_pending_requests=2 (positive int) — unaffected.
• Local pytest run not possible without a ray C-extension build; CI will exercise the GPU tests.

🤖 Generated with Claude Code

[gemini-code-assist]

Code Review

This pull request refactors the calculation of max_pending_requests within the vLLMEngineStage. The logic for resolving the default value has been moved from the UDF to the vLLMEngineWrapper, where it now dynamically calculates the limit based on vLLM's resolved engine configuration (specifically max_num_seqs and pipeline_parallel_size). This change ensures that the request concurrency limit correctly tracks GPU-dependent capacities rather than relying on hardcoded defaults. I have no feedback to provide as there were no review comments.

[jeffreywang-anyscale]

🚢