[Router] Add prefix_min_match_length threshold to PrefixAwareRouter

[ywc668]

Summary

Adds a configurable --prefix-min-match-length option to the prefix-aware
routing logic. When the longest prefix match for an incoming request is
shorter than this threshold, the request falls back to QPS-based routing
instead of being pinned to the matched endpoint.

Partially addresses #957.

Motivation

With prefixaware routing, requests sharing a long common prefix (e.g. a
large shared system prompt) are all routed to the same endpoint. This is
the intended behavior for KV-cache reuse, but it can create load hotspots:
a popular prefix concentrates traffic on a single engine while others sit
idle. This option lets operators require a minimum match length before
the prefix affinity kicks in, so short/incidental matches don't override
load balancing.

Changes

• parser.py: new --prefix-min-match-length arg (int, default 0).
• routing_logic.py: PrefixAwareRouter.__init__ now takes
  prefix_min_match_length; route_request falls back to _qps_routing
  when match_length < prefix_min_match_length.
• app.py / initialize_routing_logic: thread the new arg through.
• New test file src/tests/test_prefixaware_router.py covering the
  fallback, the matched-endpoint, and the default-behavior paths.

Design decisions

A few deliberate choices worth surfacing for review:

1. Default 0 preserves existing behavior. With the default, even a
   match_length of 0 (no prefix match at all) still uses the matched
   endpoint and random selection — identical to behavior before this
   change. The threshold is strictly opt-in.
2. The fallback path does not write to the trie. When a request falls
   back to QPS routing, hashtrie.insert is intentionally skipped, so a
   below-threshold request doesn't pollute prefix state for future
   requests. A test asserts insert is not awaited on this path.
3. Scope is limited to prefix-min-match-length. This PR intentionally
   does not touch other prefix-aware routing knobs or the QPS routing logic
   itself. Broader hotspot-mitigation strategies are left for follow-up so
   this change stays small and reviewable.

Testing

src/tests/test_prefixaware_router.py adds three async tests:

• below-threshold match falls back to QPS and picks the lowest-QPS engine,
  and does not write to the trie;
• above-threshold match uses the matched endpoint and writes to the trie;
• default (threshold 0) uses the matched endpoint even with no match.

PR Checklist

• DCO sign-off present on the commit (Signed-off-by).
• PR title is prefixed with [Router].
• New tests added for the new behavior.
• Opened as draft — pending self-review / maintainer feedback on
  the design decisions above.

[gemini-code-assist]

Code Review

This pull request introduces a minimum prefix match length threshold for prefix-aware routing. A new CLI argument, --prefix-min-match-length, allows the router to fall back to QPS-based routing if the longest prefix match is shorter than the specified value. The PrefixAwareRouter and its initialization logic have been updated to support this parameter, and new tests verify the fallback behavior. Feedback suggests clarifying the help text for the new argument, as the match length is calculated in chunks (defaulting to 128 characters), which may affect the expected precision of the threshold.

[AndrewTsao]

@ywc668 非常感谢你的工作。我们可能需要同步更新一下 helm 相关的配置，比如 https://github.com/vllm-project/production-stack/blob/main/helm/templates/deployment-router.yaml 配上对应的参数及说明。

Thank you very much for your work. We may need to synchronously update the Helm-related configurations, such as https://github.com/vllm-project/production-stack/blob/main/helm/templates/deployment-router.yaml, with the corresponding parameters and descriptions.

[ruizhang0101]

Hi thanks for the contribution! Could you add this param to helm template, values schema and readme? Otherwise LGTM.

[ywc668]

@AndrewTsao @ruizhang0101
Sure, thank you for the review! I will add helm related changes.

[ywc668]

Hi thanks for the contribution! Could you add this param to helm template, values schema and readme? Otherwise LGTM.

Added prefixMinMatchLength to the Helm chart — values.yaml, values.schema.json, templates/deployment-router.yaml, and the README parameter table. It's rendered conditionally, so the default of 0 keeps the router arg unset (CLI default is also 0), preserving existing behavior. Ready for another look.

[ruizhang0101]

LGTM

[ruizhang0101]

Could you fix the pre-commit issue when you have time?

[ywc668]

Could you fix the pre-commit issue when you have time?

Fixed the schema mismatch (the values.schema.json description now matches what's auto-generated from values.yaml). The remaining e2e failures (k8s-discovery-e2e-test, Secure-Minimal-Example) appear unrelated to this PR — they're failing on the baseline roundrobin routing test (which this PR doesn't touch), with pod-level OOMKills and PVC cleanup race conditions in the test harness. Happy to look into them if you think they're blocking, but they look like CI environment issues rather than something my change introduced.

[ruizhang0101]

LGTM