[Bugfix][Router] Sanitize prefiller payload in disagg-prefill (stream, min_tokens)

[lcz1998]

Problem

In disaggregated-prefill mode (route_disaggregated_prefill_request), send_request_to_prefiller forces max_tokens=1 but otherwise inherits the client's request body. Two fields in that body can break the prefill step.

1) stream  →  aiohttp.ContentTypeError on every streaming request

When the client sends a streaming request (stream: true), the prefiller (vLLM) returns text/event-stream, which aiohttp's response.json() refuses to decode by default (it requires application/json mimetype):

[ERROR] HTTP error in prefiller: 200, message='Attempt to decode JSON with unexpected mimetype: text/event-stream; charset=utf-8', url='http://<prefiller>:8100/v1/completions'
Traceback (most recent call last):
  File ".../services/request_service/request.py", line 932, in route_disaggregated_prefill_request
    await send_request_to_prefiller(...)
  File ".../services/request_service/request.py", line 693, in send_request_to_prefiller
    return await response.json()
aiohttp.client_exceptions.ContentTypeError: 200, message='Attempt to decode JSON with unexpected mimetype: text/event-stream; charset=utf-8'

Trivially reproducible with vllm bench serve (which sends stream=true by default):

vllm bench serve \
  --backend openai --base-url http://router:8005 \
  --endpoint /v1/completions --model qwen --tokenizer "$MODEL_DIR" \
  --dataset-name random --random-input-len 8192 --random-output-len 3000 \
  --num-prompts 50 --max-concurrency 4 --request-rate inf \
  --temperature 0 --ignore-eos

Every streaming request fails the prefill step.

2) min_tokens  →  400 from prefiller when min_tokens > 1

vLLM's SamplingParams rejects requests where min_tokens > max_tokens. We just forced max_tokens=1, so any client that legitimately sets min_tokens > 1 (a vLLM extension) would get a 400 from the prefiller. The decode phase wouldn't even start.

Root cause

send_request_to_prefiller only sanitizes max_tokens / max_completion_tokens. Other fields that could conflict with the forced max_tokens=1 semantics — stream, stream_options, min_tokens — are passed through as-is.

The orchestrated path (route_orchestrated_disaggregated_request, request.py:779-782) already strips stream / stream_options for the same reason. send_request_to_prefiller simply missed the same treatment.

Fix

Sanitize the prefiller payload (on the local copy req_data, so the decode-phase request_json is untouched):

req_data["max_tokens"] = 1
if "max_completion_tokens" in req_data:
    req_data["max_completion_tokens"] = 1
# Avoid min_tokens > max_tokens=1 conflict in vLLM SamplingParams.
req_data.pop("min_tokens", None)
# Force non-streaming: max_tokens=1 needs no SSE, and SSE would break response.json() below.
req_data["stream"] = False
req_data.pop("stream_options", None)

Why this is safe

• req_data = req_data.copy() is a shallow copy; assignments / pops on top-level scalar keys do not affect the caller's request_json (the decode phase still sees the original stream, min_tokens, etc.).
• max_tokens=1 already makes streaming pointless; no information is lost.
• The prefill response body is not consumed by the caller in the disagg-prefill flow (only await-ed for completion), so forcing non-streaming changes nothing observable.
• The orchestrated disagg-prefill flow (which does parse kv_transfer_params from the response) was already non-streaming and is unchanged by this PR.
• Forcing non-streaming also avoids unnecessary SSE encoding/decoding overhead on the prefiller hop.

Verification

• black / isort clean on the edited file (matches .pre-commit-config.yaml versions).
• After applying the fix, the same vllm bench serve reproducer above runs end-to-end against a disagg-prefill router without ContentTypeErrors.

---

• Make sure the code changes pass the pre-commit checks.
• Sign-off your commit by using -s when doing git commit
• Try to classify PRs for easy understanding of the type of changes, such as [Bugfix], [Feat], and [CI].

[gemini-code-assist]

Code Review

This pull request updates the send_request_to_prefiller function in request.py to explicitly disable streaming for prefill requests by setting stream to False and removing stream_options from the request data. This prevents aiohttp ContentTypeError exceptions when parsing the response as JSON, since a prefill request with max_tokens=1 does not require Server-Sent Events (SSE). There are no review comments, and I have no additional feedback to provide.

[lcz1998]

@ruizhang0101 could you take a look when you have a moment? This PR is blocked on the first-time-contributor "Approve and run workflows" gate plus a maintainer review. It's a small (+5/-0) bugfix that makes vllm bench serve (and any stream=true client) work in disagg-prefill mode. Thanks!

[ruizhang0101]

LGTM

[lcz1998]

LGTM

Sorry, I accidentally closed and reopened the PR. It looks like the workflows require maintainer approval again.

@ruizhang0101 could you please approve/run the checks again and merge if everything is green? Thanks!