perf(router): shallow-copy wildcard deployments instead of deepcopy in pattern router

[devin-ai-integration]

Relevant issues

Follow-up to #33721, which patched only the GET /v1/models caller. This addresses the shared root cause

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

line_profiler on PatternMatchRouter._return_pattern_matched_deployments, simulating a /v1/models listing that routes 500 wildcard-expanded model names through a router with two wildcard patterns (openai/*, bedrock/*). Same script for both runs; the only change is this diff

Before, at commit 07e07e6 (deepcopy)

Function: PatternMatchRouter._return_pattern_matched_deployments at line 100
Total time: 0.0883387 s

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
   100                                           def _return_pattern_matched_deployments(...):
   101       500        151.3      0.3      0.2      new_deployments = []
   102      1000        339.8      0.3      0.4      for deployment in deployments:
   103       500      85327.8    170.7     96.6          new_deployment = copy.deepcopy(deployment)
   104      1000       1861.4      1.9      2.1          new_deployment["litellm_params"]["model"] = PatternMatchRouter.set_deployment_model_name(
   105       500        138.3      0.3      0.2              matched_pattern=matched_pattern,
   106       500        151.1      0.3      0.2              litellm_deployment_litellm_model=deployment["litellm_params"]["model"],
   107                                                   )
   108       500        160.0      0.3      0.2          new_deployments.append(new_deployment)
   110       500        208.9      0.4      0.2      return new_deployments

After, at commit a0efa6f (shallow copy)

Function: PatternMatchRouter._return_pattern_matched_deployments at line 99
Total time: 0.00393085 s

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
    99                                           def _return_pattern_matched_deployments(...):
   100      1000        341.2      0.3      8.7      return [
   101      1500        471.4      0.3     12.0          {
   102       500        116.9      0.2      3.0              **deployment,
   103      1500        476.7      0.3     12.1              "litellm_params": {
   104       500        130.8      0.3      3.3                  **deployment["litellm_params"],
   105      1000       1704.1      1.7     43.4                  "model": PatternMatchRouter.set_deployment_model_name(...),
   109                                                       },
   110                                                   }
   111      1500        421.1      0.3     10.7          for deployment in deployments
   112                                                   ]

Function-level time drops from 88.3 ms to 3.9 ms for the 500-model listing (about 22x). The 170.7 us/call copy.deepcopy on line 103 disappears; what remains is dominated by set_deployment_model_name, the actual wildcard substitution. Measured end to end, the same 500-listing loop through pattern_router.route goes from ~203.6 us to ~40.9 us per matched model (about 5x), since route also does sorted_patterns and a regex match per call

Type

🚄 Infrastructure

Changes

PatternMatchRouter._return_pattern_matched_deployments deep-copied every deployment it returned so it could rewrite litellm_params["model"] with the wildcard substitution without corrupting the stored pattern template. deepcopy was overkill: the only field written is litellm_params["model"], so a shallow copy of the deployment dict plus a shallow copy of its litellm_params is enough to isolate the write while leaving every other nested value shared by reference

return [
    {
        **deployment,
        "litellm_params": {
            **deployment["litellm_params"],
            "model": PatternMatchRouter.set_deployment_model_name(
                matched_pattern=matched_pattern,
                litellm_deployment_litellm_model=deployment["litellm_params"]["model"],
            ),
        },
    }
    for deployment in deployments
]

This is the same shallow-copy-with-nested-litellm_params pattern already used for default_deployment in router.py (the one whose comment reads "Shallow copy with nested litellm_params copy (100x+ faster than deepcopy)")

route is the hot path for every wildcard request, not just /v1/models. It backs _common_checks_available_deployment -> _try_early_resolve_deployments_for_model_not_in_names -> get_deployments_by_pattern (per-completion routing), get_model_list, get_model_group_info -> _set_model_group_info, and get_deployment_credentials_with_provider. #33721 removed only the /v1/models caller by adding get_configured_token_limits; the deepcopy still fired on all the others until this change

QA runbook

Not an e2e test change; the added tests live in tests/local_testing/test_router_pattern_matching.py

test_route_does_not_mutate_stored_template - two sequential route() calls for different wildcard requests must each return the correct substituted litellm_params.model while the stored template keeps its openai/* value. Fails if a naive shallow-outer-only copy is used (it mutates the shared nested dict)

test_route_returns_shallow_copy_not_deepcopy - the returned deployment gets a fresh litellm_params dict (distinct object from the template) but a nested value that is not rewritten (metadata) stays the same object by reference. Fails if reverted to copy.deepcopy

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/63644b811622411ea805c21bc4917595

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[greptile-apps]

Greptile Summary

This PR replaces copy.deepcopy with a two-level shallow dict spread in PatternMatchRouter._return_pattern_matched_deployments, dropping total function time from ~88 ms to ~4 ms (22×) for a 500-model wildcard listing. The import copy statement is also removed since it is no longer used.

• The shallow copy creates a fresh outer deployment dict and a fresh litellm_params dict, so the model field substitution is fully isolated from the stored template — verified by two new regression tests covering sequential routing and reference identity.
• All other nested values inside litellm_params (e.g. metadata, credential dicts, list fields) remain shared by reference with the stored template; if any downstream caller mutates one of those objects in-place, the stored pattern would be silently corrupted for all future requests on that wildcard.

Confidence Score: 4/5

Safe to merge; the optimization is correct for the stated invariant that only litellm_params["model"] is ever written on the returned deployment.

The shallow copy correctly isolates the one field that is actually mutated and the two new tests demonstrate this end-to-end. The remaining concern is that other nested objects inside litellm_params (metadata, credential fields) are now shared by reference with the stored template, so any future or undiscovered caller that mutates those objects in-place would silently corrupt the stored wildcard pattern for all subsequent requests.

litellm/router_utils/pattern_match_deployments.py — the shallow-copy contract (only litellm_params["model"] is ever written by callers) should be documented with an inline comment to prevent future regressions.

Important Files Changed

Filename	Overview
litellm/router_utils/pattern_match_deployments.py	Replaces copy.deepcopy with a two-level shallow dict spread in _return_pattern_matched_deployments, correctly isolating the litellm_params["model"] write while sharing all other nested references; the unused import copy is also removed.
tests/local_testing/test_router_pattern_matching.py	Adds two focused regression tests: one verifying that sequential route() calls don't mutate the stored template, and one asserting that the returned litellm_params is a distinct dict while shared nested objects (metadata) remain the same object by reference.

_{Reviews (1): Last reviewed commit: "perf(router): shallow-copy wildcard depl..." | Re-trigger Greptile}

[greptile-apps]

Shared nested objects silently mutable by callers

The shallow copy isolates the top-level dict and litellm_params, but every other nested value inside litellm_params (e.g. metadata, list-typed fields, credential dicts) is still shared by reference with the stored template. Any caller that does deployment["litellm_params"]["metadata"]["key"] = value or appends to a list field will silently corrupt the stored pattern template for all future requests on that wildcard. The tests verify route() itself is safe, but they cannot prevent downstream consumers from mutating these shared objects. Worth documenting explicitly with an inline comment (matching the one in router.py) to guard against future regressions.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_pattern_route_shallow_copy (a0efa6f) with litellm_internal_staging (f759c75)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (07e07e6) during the generation of this report, so f759c75 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩