From 337ce3c69195541346f1dd8c092114d3fb6b286a Mon Sep 17 00:00:00 2001 From: Julien Chaumond Date: Mon, 14 Jul 2025 17:54:09 +0200 Subject: [PATCH 01/11] missing word --- docs/inference-providers/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/inference-providers/index.md b/docs/inference-providers/index.md index 2f4b94065d..0e269c974c 100644 --- a/docs/inference-providers/index.md +++ b/docs/inference-providers/index.md @@ -5,7 +5,7 @@ -Hugging Face’s Inference Providers give developers access to hundreds of machine learning models, powered by world-class inference providers. They are also integrated into our client SDKs (for JS and Python), making it easy to explore serverless inference of models your favorite providers. +Hugging Face’s Inference Providers give developers access to hundreds of machine learning models, powered by world-class inference providers. They are also integrated into our client SDKs (for JS and Python), making it easy to explore serverless inference of models on your favorite providers. ## Partners From ba3b8414d1a791e7205d94b2c3bddc9ebcf241e2 Mon Sep 17 00:00:00 2001 From: Julien Chaumond Date: Mon, 14 Jul 2025 18:01:54 +0200 Subject: [PATCH 02/11] billing: this is not the case anymore --- docs/inference-providers/pricing.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/docs/inference-providers/pricing.md b/docs/inference-providers/pricing.md index 467dff4276..5f787c6e61 100644 --- a/docs/inference-providers/pricing.md +++ b/docs/inference-providers/pricing.md @@ -44,8 +44,6 @@ See the [Organization Billing section](#organization-billing) below for more det **PRO users and Enterprise Hub organizations** can continue using the API after exhausting their monthly credits. This ensures uninterrupted access to models for production workloads. -If you have remaining credits, we estimate costs for providers that aren’t fully integrated with our billing system. These estimates are usually higher than the actual cost to prevent abuse, which is why PAYG is currently disabled for those providers. - Hugging Face charges you the same rates as the provider, with no additional fees. We just pass through the provider costs directly. From d61ffb5a0281b758d1ac7b0a91a210418f5b76b1 Mon Sep 17 00:00:00 2001 From: Julien Chaumond Date: Mon, 14 Jul 2025 18:02:40 +0200 Subject: [PATCH 03/11] only 2 are listed? --- docs/inference-providers/pricing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/inference-providers/pricing.md b/docs/inference-providers/pricing.md index 5f787c6e61..f3facb1542 100644 --- a/docs/inference-providers/pricing.md +++ b/docs/inference-providers/pricing.md @@ -54,7 +54,7 @@ You can track your spending anytime on your [billing page](https://huggingface.c ## Hugging Face Billing vs Custom Provider Key (Detailed Comparison) -The documentation above assumes you are making routed requests to external providers. In practice, there are 3 different ways to run inference, each with unique billing implications: +The documentation above assumes you are making routed requests to external providers. In practice, there are 2 different ways to run inference, each with unique billing implications: - **Hugging Face Routed Requests**: This is the default method for using Inference Providers. Simply use the JavaScript or Python `InferenceClient`, or make raw HTTP requests with your Hugging Face User Access Token. Your request is automatically routed through Hugging Face to the provider's platform. No separate provider account is required, and billing is managed directly by Hugging Face. This approach lets you seamlessly switch between providers without additional setup. From c8c370b1e771676721849d54435b8e63ab5c3d17 Mon Sep 17 00:00:00 2001 From: Julien Chaumond Date: Mon, 14 Jul 2025 18:03:42 +0200 Subject: [PATCH 04/11] =?UTF-8?q?not=20true=20anymore=20either=20?= =?UTF-8?q?=F0=9F=98=B1?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/inference-providers/pricing.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/inference-providers/pricing.md b/docs/inference-providers/pricing.md index f3facb1542..0fe454dce0 100644 --- a/docs/inference-providers/pricing.md +++ b/docs/inference-providers/pricing.md @@ -79,7 +79,6 @@ As you may have noticed, you can select to work with `"hf-inference"` provider. For instance, a request to [black-forest-labs/FLUX.1-dev](https://huggingface.co/black-forest-labs/FLUX.1-dev) that takes 10 seconds to complete on a GPU machine that costs $0.00012 per second to run, will be billed $0.0012. -The `"hf-inference"` provider is currently the default provider when working with the JavaScript and Python SDKs. Note that this default might change in the future. ## Billing for Team and Enterprise organizations From a1a1f6fa549080bd725c0f6ed2057770af8de481 Mon Sep 17 00:00:00 2001 From: Julien Chaumond Date: Mon, 14 Jul 2025 18:10:31 +0200 Subject: [PATCH 05/11] mention the current focus of hf-inference --- docs/inference-providers/pricing.md | 1 + .../templates/providers/hf-inference.handlebars | 2 ++ 2 files changed, 3 insertions(+) diff --git a/docs/inference-providers/pricing.md b/docs/inference-providers/pricing.md index 0fe454dce0..b4e78164a4 100644 --- a/docs/inference-providers/pricing.md +++ b/docs/inference-providers/pricing.md @@ -79,6 +79,7 @@ As you may have noticed, you can select to work with `"hf-inference"` provider. For instance, a request to [black-forest-labs/FLUX.1-dev](https://huggingface.co/black-forest-labs/FLUX.1-dev) that takes 10 seconds to complete on a GPU machine that costs $0.00012 per second to run, will be billed $0.0012. +As of July 2025, hf-inference focuses mostly on CPU inference (e.g. embedding, text-ranking, text-classification, or smaller LLMs that have historical importance like BERT or GPT-2). ## Billing for Team and Enterprise organizations diff --git a/scripts/inference-providers/templates/providers/hf-inference.handlebars b/scripts/inference-providers/templates/providers/hf-inference.handlebars index 21dbef8884..cd63f1423d 100644 --- a/scripts/inference-providers/templates/providers/hf-inference.handlebars +++ b/scripts/inference-providers/templates/providers/hf-inference.handlebars @@ -13,4 +13,6 @@ All supported HF Inference models can be found [here](https://huggingface.co/mod HF Inference is the serverless Inference API powered by Hugging Face. This service used to be called "Inference API (serverless)" prior to Inference Providers. If you are interested in deploying models to a dedicated and autoscaling infrastructure managed by Hugging Face, check out [Inference Endpoints](https://huggingface.co/docs/inference-endpoints/index) instead. +As of July 2025, hf-inference focuses mostly on CPU inference (e.g. embedding, text-ranking, text-classification, or smaller LLMs that have historical importance like BERT or GPT-2). + {{{tasksSection}}} \ No newline at end of file From 3da7ac5ffb83c38a887e22c23845244eb6b1ef03 Mon Sep 17 00:00:00 2001 From: Julien Chaumond Date: Mon, 14 Jul 2025 18:11:06 +0200 Subject: [PATCH 06/11] mention org settings here too TODO: add a thumbnail of team usage --- docs/inference-providers/hub-integration.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/docs/inference-providers/hub-integration.md b/docs/inference-providers/hub-integration.md index 080250f5e4..36a6507ee6 100644 --- a/docs/inference-providers/hub-integration.md +++ b/docs/inference-providers/hub-integration.md @@ -65,3 +65,12 @@ In your user account settings, you are able to: + +## Organization Settings + +Similar settings can be found in your Organization settings. Additionally, you can see a graph of your team member's usage over time, helpful to centralize usage billing at the team level. + +
+ + +
From 1c35c31d342bb53700d06cc0f8b8474419e36a69 Mon Sep 17 00:00:00 2001 From: Julien Chaumond Date: Mon, 14 Jul 2025 18:23:12 +0200 Subject: [PATCH 07/11] minor tweaks --- docs/inference-providers/guides/function-calling.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/inference-providers/guides/function-calling.md b/docs/inference-providers/guides/function-calling.md index c5df6855ec..d292267255 100644 --- a/docs/inference-providers/guides/function-calling.md +++ b/docs/inference-providers/guides/function-calling.md @@ -37,7 +37,7 @@ client = OpenAI( -In the Hugging Face Hub client, we'll use the `provider` parameter to specify the provider we want to use for the request. +In the Hugging Face Hub client, we'll use the `provider` parameter to specify the provider we want to use for the request. By default, it is `"auto"`. ```python import json @@ -477,7 +477,7 @@ Streaming is not supported by all providers. You can check the provider's docume ## Next Steps -Now that you've seen how to use function calling with Inference Providers, you can start building your own assistants! Why not try out some of these ideas: +Now that you've seen how to use function calling with Inference Providers, you can start building your own agents and assistants! Why not try out some of these ideas: - Try smaller models for faster responses and lower costs - Build an agent that can fetch real-time data From 3f37113e1dae6bcdbaef9b10da1f20e239d247f2 Mon Sep 17 00:00:00 2001 From: Julien Chaumond Date: Mon, 14 Jul 2025 18:23:20 +0200 Subject: [PATCH 08/11] better more precise name --- docs/inference-providers/guides/function-calling.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/inference-providers/guides/function-calling.md b/docs/inference-providers/guides/function-calling.md index d292267255..f11f15147b 100644 --- a/docs/inference-providers/guides/function-calling.md +++ b/docs/inference-providers/guides/function-calling.md @@ -400,7 +400,7 @@ Here, we're forcing the model to call the `get_current_weather` function, and no -Currently, Hugging Face Hub does not support the `tool_choice` parameters that specify which function to call. +Currently, `huggingface_hub.InferenceClient` does not support the `tool_choice` parameters that specify which function to call. From 23cff41fcfef4e315a4403a08c8b18e75eaa0911 Mon Sep 17 00:00:00 2001 From: Julien Chaumond Date: Mon, 14 Jul 2025 18:23:32 +0200 Subject: [PATCH 09/11] link to @SBrandeis's table --- docs/inference-providers/guides/function-calling.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/inference-providers/guides/function-calling.md b/docs/inference-providers/guides/function-calling.md index f11f15147b..c789a86b59 100644 --- a/docs/inference-providers/guides/function-calling.md +++ b/docs/inference-providers/guides/function-calling.md @@ -471,7 +471,7 @@ Streaming allows you to process responses as they arrive, show real-time progres -Streaming is not supported by all providers. You can check the provider's documentation to see if it supports streaming. +Streaming is not supported by all providers. You can check the provider's documentation to see if it supports streaming, or you can refer to this [dynamic compatibility table on hf.co](https://huggingface.co/inference-providers/models). From 5070a89ca64cd647f0a4449e2da5a94f8d0fc37b Mon Sep 17 00:00:00 2001 From: Julien Chaumond Date: Tue, 15 Jul 2025 15:58:25 +0200 Subject: [PATCH 10/11] Update docs/inference-providers/guides/function-calling.md Co-authored-by: Lucain --- docs/inference-providers/guides/function-calling.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/inference-providers/guides/function-calling.md b/docs/inference-providers/guides/function-calling.md index c789a86b59..c43563c183 100644 --- a/docs/inference-providers/guides/function-calling.md +++ b/docs/inference-providers/guides/function-calling.md @@ -471,7 +471,7 @@ Streaming allows you to process responses as they arrive, show real-time progres -Streaming is not supported by all providers. You can check the provider's documentation to see if it supports streaming, or you can refer to this [dynamic compatibility table on hf.co](https://huggingface.co/inference-providers/models). +Streaming is not supported by all providers. You can check the provider's documentation to see if it supports streaming, or you can refer to this [dynamic compatibility table](https://huggingface.co/inference-providers/models). From e5bbebc15377f6f71a25802498a2cb02943513c5 Mon Sep 17 00:00:00 2001 From: Julien Chaumond Date: Tue, 15 Jul 2025 16:13:00 +0200 Subject: [PATCH 11/11] Update docs/inference-providers/guides/function-calling.md Co-authored-by: vb --- docs/inference-providers/guides/function-calling.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/inference-providers/guides/function-calling.md b/docs/inference-providers/guides/function-calling.md index c43563c183..dca307a123 100644 --- a/docs/inference-providers/guides/function-calling.md +++ b/docs/inference-providers/guides/function-calling.md @@ -471,7 +471,7 @@ Streaming allows you to process responses as they arrive, show real-time progres -Streaming is not supported by all providers. You can check the provider's documentation to see if it supports streaming, or you can refer to this [dynamic compatibility table](https://huggingface.co/inference-providers/models). +Streaming is not supported by all providers. You can check the provider's documentation to see if it supports streaming, or you can refer to this [dynamic model compatibility table](https://huggingface.co/inference-providers/models).