Skip to content

Add custom proxy support for OpenAI-compatible APIs#312

Open
gbladislau wants to merge 1 commit into
huggingface:mainfrom
gbladislau:add_custom_proxys_support
Open

Add custom proxy support for OpenAI-compatible APIs#312
gbladislau wants to merge 1 commit into
huggingface:mainfrom
gbladislau:add_custom_proxys_support

Conversation

@gbladislau

Copy link
Copy Markdown

PR motivation

This removes the model name check for custom proxys, if for example the naming there is different then the ones used in the HF Router. Thus giving more custom-ability for the user.

Summary

Adds a generic custom-proxy/ local model provider for OpenAI-compatible proxy endpoints.

This allows users to run models behind an internal/custom OpenAI-style API, for example:

CUSTOM_PROXY_BASE_URL="https://proxy.example.com/api/v1/gpt/session-id"
CUSTOM_PROXY_API_KEY="<optional>"
ml-intern --model "custom-proxy/Azure AI/gpt-5.5" "your prompt"

Changes

  • Adds custom-proxy/ to supported local/custom model prefixes.
  • Resolves custom-proxy/ to LiteLLM OpenAI-compatible params.
  • Preserves CUSTOM_PROXY_BASE_URL exactly instead of appending /v1.
  • Allows spaces in custom-proxy/ model names, e.g. custom-proxy/Azure AI/gpt-5.5.
  • Documents custom proxy usage in the README.
  • Adds unit coverage for custom proxy parameter resolution and CLI model validation.

This removes the model name check for custom proxys, if for example
the naming there is different then the ones used in the HF Router.
Thus giving more custom-ability for the user.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant