Skip to content

Document auto route provider selection#1841

Merged
Wauplin merged 1 commit into
mainfrom
mention-auto-route-provider-selection
Jul 18, 2025
Merged

Document auto route provider selection#1841
Wauplin merged 1 commit into
mainfrom
mention-auto-route-provider-selection

Conversation

@Wauplin

@Wauplin Wauplin commented Jul 18, 2025

Copy link
Copy Markdown
Contributor

Document that it's possible to select provider like this:

client.chat.completions.create(
-     model="deepseek-ai/DeepSeek-R1-0528", # automatically select provider based on hf.co/settings/inference-providers
+     model="deepseek-ai/DeepSeek-R1-0528:nebius", # manually select Nebius AI
      ...
)

I removed a lot of mentions to hf.co/settings/inference-providers. It's still valid but I feel that it's a bit niche compared to appending e.g. :groq. In addition to this, all LLM/VLM snippets should be updated in the next hub-docs rebuild (see https://huggingface.co/moonshotai/Kimi-K2-Instruct?inference_api=true&inference_provider=groq&language=python).

@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@burtenshaw burtenshaw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great improvement! LGTM.

@Wauplin

Wauplin commented Jul 18, 2025

Copy link
Copy Markdown
Contributor Author

Thanks for the quick review :)

@Wauplin
Wauplin merged commit ff558ce into main Jul 18, 2025
2 checks passed
@Wauplin
Wauplin deleted the mention-auto-route-provider-selection branch July 18, 2025 14:25

Our routing system automatically selects the most popular available provider for your chosen model,
or to your preferred provider if you've sorted the providers from your [user settings](https://huggingface.co/settings/inference-providers).
Our routing system automatically selects the most popular available provider for your chosen model. You can also select the provider of your choice by appending it to the model id (e.g. `"deepseek-ai/DeepSeek-V3-0324:fireworks-ai"`).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or by going to /settings/inference-providers (i would still showcase it a bit)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants