From 588daf9c643be4b48cc2cedbaaba22d5e746502a Mon Sep 17 00:00:00 2001 From: burtenshaw Date: Thu, 26 Jun 2025 14:20:41 +0200 Subject: [PATCH 01/15] add more detail to the billing page --- docs/inference-providers/pricing.md | 55 ++++++++++++++++++++++------- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/docs/inference-providers/pricing.md b/docs/inference-providers/pricing.md index 3199c74bc1..bcc8533713 100644 --- a/docs/inference-providers/pricing.md +++ b/docs/inference-providers/pricing.md @@ -1,31 +1,62 @@ # Pricing and Billing -Inference Providers is a production-ready service involving external partners and is therefore a paid product. However, as a Hugging Face user, you get monthly credits to run experiments. The amount of credits you get depends on your type of account: +Access 200+ models from leading AI inference providers with centralized, transparent, pay-as-you-go pricing. No infrastructure management required—just pay for what you use, with no markup from Hugging Face. -| Tier | Included monthly credits | -| ---------------------------- | ------------------------------------ | -| Free Users | subject to change, less than $0.10 | -| PRO Users | $2.00 | -| Enterprise Hub Organizations | $2.00 per seat, shared among members | +## Free Credits to Get Started + +Every Hugging Face user receives monthly credits to experiment with Inference Providers: + +| Account Type | Monthly Credits | Value | +| ---------------------------- | --------------- | -------- | +| Free Users | Limited credits | ~$0.10 | +| PRO Users | Full access | **$2.00** | +| Enterprise Hub Organizations | Per-seat access | **$2.00 per seat** | + + + +Your monthly credits automatically apply when you route requests through Hugging Face. For Enterprise organizations, credits are shared among all members. + + + +## How Billing Works: Choose Your Approach + +Inference Providers offers flexibility in how you're billed. Understanding these options upfront helps you choose the best approach for your needs: + +| Feature | **Routed by Hugging Face** | **Custom Provider Key** | **Direct Calls** | +| :--- | :--- | :--- | :--- | +| **How it Works** | Your request routes through HF to the provider | You set a custom provider key in HF settings | You provide the provider key directly in your code | +| **Billing** | Pay-as-you-go on your HF account | Billed directly by the provider | Billed directly by the provider | +| **Monthly Credits** | **✅ Yes** - Credits apply to eligible providers | **❌ No** - Credits don't apply | **❌ No** - Credits don't apply | +| **Provider Account Needed** | **❌ No** - We handle everything | **✅ Yes** - You need provider accounts | **✅ Yes** - You need provider accounts | +| **Best For** | Simplicity, experimentation, consolidated billing | More billing control, using non-integrated providers | Full control, bypassing HF routing | +| **Integration** | SDKs, Playground, widgets, Data AI Studio | SDKs, Playground, widgets, Data AI Studio | SDKs only | + +### Which Option Should I Choose? + +- **Start with Routed by Hugging Face** if you want simplicity and to use your monthly credits +- **Use Custom Provider Key** if you need specific provider features or you're consistently using the same provider +- **Use Direct Calls** if you want to bypass Hugging Face routing entirely + +## Pay-as-you-Go Details To benefit from Enterprise Hub included credits, you need to explicitly specify the organization to be billed when performing the inference requests. See the [Organization Billing section](#organization-billing) below for more details. -## Pay-as-you-Go +**PRO users and Enterprise Hub organizations** can continue using the API after exhausting their monthly credits. This ensures uninterrupted access to models for production workloads. -**PRO users and Enterprise Hub organizations** can continue using the API once their monthly included credits are exhausted. This billing model, known as "Pay-as-you-Go" (PAYG), is charged on top of the monthly subscription. PAYG is only available for providers that are integrated with our billing system. We're actively working to integrate all providers, but in the meantime, any providers that are not yet integrated will be blocked once the free-tier limit is reached. +**Current Status**: Pay-as-you-Go is available for providers integrated with our billing system. We're actively integrating remaining providers—those not yet integrated will be blocked once free credits are exhausted. If you have remaining credits, we estimate costs for providers that aren’t fully integrated with our billing system. These estimates are usually higher than the actual cost to prevent abuse, which is why PAYG is currently disabled for those providers. -You can track your spending on your [billing page](https://huggingface.co/settings/billing). - Hugging Face charges you the same rates as the provider, with no additional fees. We just pass through the provider costs directly. -## Routed requests vs direct calls +You can track your spending anytime on your [billing page](https://huggingface.co/settings/billing). + +## Routed Requests vs Direct Calls (Detailed Comparison) The documentation above assumes you are making routed requests to external providers. In practice, there are 3 different ways to run inference, each with unique billing implications: @@ -51,7 +82,7 @@ For instance, a request to [black-forest-labs/FLUX.1-dev](https://huggingface.co The `"hf-inference"` provider is currently the default provider when working with the JavaScript and Python SDKs. Note that this default might change in the future. -## Organization billing +## Billing forEnterprise Hub organizations For Enterprise Hub organizations, it is possible to centralize billing for all of your users. Each user still uses their own User Access Token but the requests are billed to your organization. This can be done by passing `"X-HF-Bill-To: my-org-name"` as a header in your HTTP requests. From cd9801b13b9cddf1f6df93ff273c9f220bafbd66 Mon Sep 17 00:00:00 2001 From: burtenshaw Date: Thu, 26 Jun 2025 15:03:55 +0200 Subject: [PATCH 02/15] add first basic guide on inference --- .../guides/first-api-call.md | 131 ++++++++++++++++++ 1 file changed, 131 insertions(+) create mode 100644 docs/inference-providers/guides/first-api-call.md diff --git a/docs/inference-providers/guides/first-api-call.md b/docs/inference-providers/guides/first-api-call.md new file mode 100644 index 0000000000..3b3f5c5dac --- /dev/null +++ b/docs/inference-providers/guides/first-api-call.md @@ -0,0 +1,131 @@ +# Your First Inference Provider Call + +In this guide we're going to help you make your first API call with Inference Providers. + +Many developers avoid using open source AI models because they assume deployment is complex. This guide will show you how to use a state-of-the-art model in under five minutes, with no infrastructure setup required. + +We're going to use the [FLUX.1-schnell](https://huggingface.co/black-forest-labs/FLUX.1-schnell) model, which is a powerful text-to-image model. + + + +This guide assumes you have a Hugging Face account. If you don't have one, you can create one for free at [huggingface.co](https://huggingface.co). + + + +## Step 1: Find a Model on the Hub + +Visit the [Hugging Face Hub](https://huggingface.co/models) and look for models with the "Inference Providers" filter, you can select the provider that you want. We'll go with `fal`. + +![search image](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/inference-providers-guides/search.png) + +For this example, we'll use [FLUX.1-schnell](https://huggingface.co/black-forest-labs/FLUX.1-schnell), a powerful text-to-image model. Next, navigate to the model page and scroll down to find the inference widget on the right side. + +## Step 2: Try the Interactive Widget + +Before writing any code, try the widget directly on the model page: + +![widget image](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/inference-providers-guides/widget.png) + +Here, you can test the model directly in the browser from any of the available providers. You can also copy relevant code snippets to use in your own projects. + +1. Enter a prompt like "A serene mountain landscape at sunset" +2. Click **"Generate"** +3. Watch as the model creates an image in seconds + +This widget uses the same endpoint you're about to implement in code. + + + +You'll need a Hugging Face account (free at [huggingface.co](https://huggingface.co)) and remaining credits to use the model. + + + +## Step 3: From Clicks to Code + +Now let's replicate this with Python. Click the **"View Code Snippets"** button in the widget to see the generated code snippets. + +![code snippets image](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/inference-providers-guides/code-snippets.png) + +You will need to populate this snippet with a valid Hugging Face User Access Token. You can find your User Access Token in your [settings page](https://huggingface.co/settings/tokens). + +Set your token as an environment variable: + +```bash +export HF_TOKEN="your_token_here" +``` + +The Python or TypeScript code snippet will use the token from the environment variable. + + + + + +Install the required package: + +```bash +pip install huggingface_hub +``` + +You can now use the code snippet to generate an image: + +```python +import os +from huggingface_hub import InferenceClient + +client = InferenceClient( + provider="fal-ai", + api_key=os.environ["HF_TOKEN"], +) + +# output is a PIL.Image object +image = client.text_to_image( + "Astronaut riding a horse", + model="black-forest-labs/FLUX.1-schnell", +) +``` + + + + + +Install the required package: + +```bash +npm install @huggingface/inference +``` + +```typescript +import { InferenceClient } from "@huggingface/inference"; + +const client = new InferenceClient(process.env.HF_TOKEN); + +const image = await client.textToImage({ + provider: "fal-ai", + model: "black-forest-labs/FLUX.1-schnell", + inputs: "Astronaut riding a horse", + parameters: { num_inference_steps: 5 }, +}); +/// Use the generated image (it's a Blob) +``` + + + + +## What Just Happened? + +Nice work! You've successfully used a production-grade AI model without any complex setup. In just a few lines of code, you: + +- Connected to a powerful text-to-image model +- Generated a custom image from text +- Saved the result locally + +The model you just used runs on professional infrastructure, handling scaling, optimization, and reliability automatically. + +## Next Steps + +Now that you've seen how easy it is to use AI models, you might wonder: +- What was that "provider" system doing behind the scenes? +- How does billing work? +- What other models can you use? + +Continue to the next guide to understand the provider ecosystem and make informed choices about authentication and billing. \ No newline at end of file From 3ee3d4ffd7dc68e2d0bf687909b22f7fbe60c4d7 Mon Sep 17 00:00:00 2001 From: burtenshaw Date: Thu, 26 Jun 2025 15:26:58 +0200 Subject: [PATCH 03/15] update index with tts providers --- docs/inference-providers/index.md | 32 +++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/docs/inference-providers/index.md b/docs/inference-providers/index.md index c3970c044d..adec7b7a90 100644 --- a/docs/inference-providers/index.md +++ b/docs/inference-providers/index.md @@ -13,22 +13,22 @@ To learn more about the launch of Inference Providers, check out our [announceme Here is the complete list of partners integrated with Inference Providers, and the supported tasks for each of them: -| Provider | Chat completion (LLM) | Chat completion (VLM) | Feature Extraction | Text to Image | Text to video | -| ---------------------------------------- | :-------------------: | :-------------------: | :----------------: | :-----------: | :-----------: | -| [Cerebras](./providers/cerebras) | ✅ | | | | | -| [Cohere](./providers/cohere) | ✅ | ✅ | | | | -| [Fal AI](./providers/fal-ai) | | | | ✅ | ✅ | -| [Featherless AI](./providers/featherless-ai) | ✅ | ✅ | | | | -| [Fireworks](./providers/fireworks-ai) | ✅ | ✅ | | | | -| [Groq](./providers/groq) | ✅ | | | | | -| [HF Inference](./providers/hf-inference) | ✅ | ✅ | ✅ | ✅ | | -| [Hyperbolic](./providers/hyperbolic) | ✅ | ✅ | | | | -| [Nebius](./providers/nebius) | ✅ | ✅ | ✅ | ✅ | | -| [Novita](./providers/novita) | ✅ | ✅ | | | ✅ | -| [Nscale](./providers/nscale) | ✅ | ✅ | | ✅ | | -| [Replicate](./providers/replicate) | | | | ✅ | ✅ | -| [SambaNova](./providers/sambanova) | ✅ | | ✅ | | | -| [Together](./providers/together) | ✅ | ✅ | | ✅ | | +| Provider | Chat completion (LLM) | Chat completion (VLM) | Feature Extraction | Text to Image | Text to video | Speech to text | +| ---------------------------------------- | :-------------------: | :-------------------: | :----------------: | :-----------: | :-----------: | :-----------: | +| [Cerebras](./providers/cerebras) | ✅ | | | | | | +| [Cohere](./providers/cohere) | ✅ | ✅ | | | | | +| [Fal AI](./providers/fal-ai) | | | | ✅ | ✅ | ✅ | +| [Featherless AI](./providers/featherless-ai) | ✅ | ✅ | | | | | +| [Fireworks](./providers/fireworks-ai) | ✅ | ✅ | | | | | +| [Groq](./providers/groq) | ✅ | | | | | | +| [HF Inference](./providers/hf-inference) | ✅ | ✅ | ✅ | ✅ | | ✅ | +| [Hyperbolic](./providers/hyperbolic) | ✅ | ✅ | | | | | +| [Nebius](./providers/nebius) | ✅ | ✅ | ✅ | ✅ | | | +| [Novita](./providers/novita) | ✅ | ✅ | | | ✅ | | +| [Nscale](./providers/nscale) | ✅ | ✅ | | ✅ | | | +| [Replicate](./providers/replicate) | | | | ✅ | ✅ | ✅ | +| [SambaNova](./providers/sambanova) | ✅ | | ✅ | | | | +| [Together](./providers/together) | ✅ | ✅ | | ✅ | | | ## Why use Inference Providers? From bbe629c88ed0e5c3b3866c4abc26d8f28776ff6e Mon Sep 17 00:00:00 2001 From: burtenshaw Date: Thu, 26 Jun 2025 15:41:23 +0200 Subject: [PATCH 04/15] add building your first app guide --- .../guides/building-first-app.md | 226 ++++++++++++++++++ 1 file changed, 226 insertions(+) create mode 100644 docs/inference-providers/guides/building-first-app.md diff --git a/docs/inference-providers/guides/building-first-app.md b/docs/inference-providers/guides/building-first-app.md new file mode 100644 index 0000000000..5debbc2535 --- /dev/null +++ b/docs/inference-providers/guides/building-first-app.md @@ -0,0 +1,226 @@ +# Building Your First AI App with Inference Providers + +You've learned the basics and understand the provider ecosystem. Now let's build something practical: an **AI Meeting Notes** app that transcribes audio files and generates summaries with action items. + +This project demonstrates real-world AI orchestration using multiple specialized providers within a single application. + +## Project Overview + +Our app will: +1. **Accept audio** as a microphone input through a web interface +2. **Transcribe speech** using a fast speech-to-text model +3. **Generate summaries** using a powerful language model +4. **Deploy to the web** for easy sharing + +**Tech Stack**: Gradio (for the UI) + Inference Providers (for the AI) + +## Step 1: Set Up Authentication + +Before we start coding, authenticate with Hugging Face using the CLI: + +```bash +pip install huggingface_hub +huggingface-cli login +``` + +When prompted, paste your Hugging Face token. This handles authentication automatically for all your inference calls. You can generate one from [your settings page](https://huggingface.co/settings/tokens/new?ownUserPermissions=inference.serverless.write&tokenType=fineGrained). + +## Step 2: Build the User Interface + +Now let's create a simple web interface using Gradio: + +```python +import gradio as gr +from huggingface_hub import InferenceClient + +def process_meeting_audio(audio_file): + """Process uploaded audio file and return transcript + summary""" + if audio_file is None: + return "Please upload an audio file.", "" + + # We'll implement the AI logic next + return "Transcript will appear here...", "Summary will appear here..." + +# Create the Gradio interface +app = gr.Interface( + fn=process_meeting_audio, + inputs=gr.Audio(label="Upload Meeting Audio", type="filepath"), + outputs=[ + gr.Textbox(label="Transcript", lines=10), + gr.Textbox(label="Summary & Action Items", lines=8) + ], + title="🎤 AI Meeting Notes", + description="Upload an audio file to get an instant transcript and summary with action items." +) + +if __name__ == "__main__": + app.launch() +``` + +Here we're using Gradio's `gr.Audio` component to either upload an audio file or use the microphone input. We're keeping things simple with two outputs: a transcript and a summary with action items. + +## Step 3: Add Speech Transcription + +Now let's implement the transcription using `fal.ai` and OpenAI's `whisper-large-v3` model for fast, reliable speech processing: + +```python +def transcribe_audio(audio_file_path): + """Transcribe audio using fal.ai for speed""" + client = InferenceClient(provider="fal-ai") + + # Pass the file path directly - the client handles file reading + transcript = client.automatic_speech_recognition( + audio=audio_file_path, + model="openai/whisper-large-v3" + ) + + return transcript.text +``` + +## Step 4: Add AI Summarization + +Next, we'll use a powerful language model like `Qwen/Qwen3-235B-A22B-FP8` from Qwen via Together AI for summarization: + +```python +def generate_summary(transcript): + """Generate summary using Together AI""" + client = InferenceClient(provider="together") + + prompt = f""" + Analyze this meeting transcript and provide: + 1. A concise summary of key points + 2. Action items with responsible parties + 3. Important decisions made + + Transcript: {transcript} + + Format with clear sections: + ## Summary + ## Action Items + ## Decisions Made + """ + + response = client.chat.completions.create( + model="Qwen/Qwen3-235B-A22B-FP8", + messages=[{"role": "user", "content": prompt}], + max_tokens=1000 + ) + + return response.choices[0].message.content +``` + +Note, we're also defining a custom summary prompt to ensure the output is formatted as a summary with action items and decisions made. + +## Step 5: Deploy on Hugging Face Spaces + +To deploy, we'll need to create a `requirements.txt` file and a `app.py` file. + +`requirements.txt`: + +```txt +gradio +huggingface_hub +``` + +`app.py`: + +
+📋 Click to view the complete app.py file + +```python +import gradio as gr +from huggingface_hub import InferenceClient + + +def transcribe_audio(audio_file_path): + """Transcribe audio using fal.ai for speed""" + client = InferenceClient(provider="fal-ai") + + # Pass the file path directly - the client handles file reading + transcript = client.automatic_speech_recognition( + audio=audio_file_path, model="openai/whisper-large-v3" + ) + + return transcript.text + + +def generate_summary(transcript): + """Generate summary using Together AI""" + client = InferenceClient(provider="together") + + prompt = f""" + Analyze this meeting transcript and provide: + 1. A concise summary of key points + 2. Action items with responsible parties + 3. Important decisions made + + Transcript: {transcript} + + Format with clear sections: + ## Summary + ## Action Items + ## Decisions Made + """ + + response = client.chat.completions.create( + model="Qwen/Qwen3-235B-A22B-FP8", + messages=[{"role": "user", "content": prompt}], + max_tokens=1000, + ) + + return response.choices[0].message.content + + +def process_meeting_audio(audio_file): + """Main processing function""" + if audio_file is None: + return "Please upload an audio file.", "" + + try: + # Step 1: Transcribe + transcript = transcribe_audio(audio_file) + + # Step 2: Summarize + summary = generate_summary(transcript) + + return transcript, summary + + except Exception as e: + return f"Error processing audio: {str(e)}", "" + + +# Create Gradio interface +app = gr.Interface( + fn=process_meeting_audio, + inputs=gr.Audio(label="Upload Meeting Audio", type="filepath"), + outputs=[ + gr.Textbox(label="Transcript", lines=10), + gr.Textbox(label="Summary & Action Items", lines=8), + ], + title="🎤 AI Meeting Notes", + description="Upload audio to get instant transcripts and summaries.", +) + +if __name__ == "__main__": + app.launch() +``` + +
+ +To deploy, we'll need to create a new Space and upload our files. + +1. **Create a new Space**: Go to [huggingface.co/new-space](https://huggingface.co/new-space) +2. **Choose Gradio SDK** and make it public +3. **Upload your files**: Upload `app.py` and `requirements.txt` +4. **Add your token**: In Space settings, add `HF_TOKEN` as a secret (get it from [your settings](https://huggingface.co/settings/tokens)) +5. **Launch**: Your app will be live at `https://huggingface.co/spaces/your-username/your-space-name` + +> **Note**: While we used CLI authentication locally, Spaces requires the token as a secret for the deployment environment. + +## Next Steps + +Congratulations! You've created a production-ready AI application that: handles real-world tasks, provides a professional interface, scales automatically, and costs efficiently. If you want to explore more providers, you can check out the [Inference Providers](https://huggingface.co/inference-providers) page. Or here are some ideas for next steps: + +- **Improve your prompt**: Try different prompts to improve the quality for your use case +- **Try different models**: Experiment with various speech and text models +- **Compare performance**: Benchmark speed vs. accuracy across providers From 3a51d0b8a068c97ff735b958646d5a12a2b1b598 Mon Sep 17 00:00:00 2001 From: burtenshaw Date: Thu, 26 Jun 2025 15:41:34 +0200 Subject: [PATCH 05/15] update toc with guides pages --- docs/inference-providers/_toctree.yml | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/docs/inference-providers/_toctree.yml b/docs/inference-providers/_toctree.yml index e1a2c9591c..7b90931143 100644 --- a/docs/inference-providers/_toctree.yml +++ b/docs/inference-providers/_toctree.yml @@ -11,6 +11,13 @@ - local: security title: Security +- title: Guides + sections: + - local: guides/first-api-call + title: Your First API Call + - local: guides/building-first-app + title: Building Your First AI App + - title: Providers sections: - local: providers/cerebras From 20b53648cde77cdb508e59bef625212ed9769dbf Mon Sep 17 00:00:00 2001 From: burtenshaw Date: Mon, 30 Jun 2025 10:49:25 +0200 Subject: [PATCH 06/15] move pricing change to separate pr 1799 --- docs/inference-providers/pricing.md | 55 +++++++---------------------- 1 file changed, 12 insertions(+), 43 deletions(-) diff --git a/docs/inference-providers/pricing.md b/docs/inference-providers/pricing.md index bcc8533713..3199c74bc1 100644 --- a/docs/inference-providers/pricing.md +++ b/docs/inference-providers/pricing.md @@ -1,62 +1,31 @@ # Pricing and Billing -Access 200+ models from leading AI inference providers with centralized, transparent, pay-as-you-go pricing. No infrastructure management required—just pay for what you use, with no markup from Hugging Face. +Inference Providers is a production-ready service involving external partners and is therefore a paid product. However, as a Hugging Face user, you get monthly credits to run experiments. The amount of credits you get depends on your type of account: -## Free Credits to Get Started - -Every Hugging Face user receives monthly credits to experiment with Inference Providers: - -| Account Type | Monthly Credits | Value | -| ---------------------------- | --------------- | -------- | -| Free Users | Limited credits | ~$0.10 | -| PRO Users | Full access | **$2.00** | -| Enterprise Hub Organizations | Per-seat access | **$2.00 per seat** | - - - -Your monthly credits automatically apply when you route requests through Hugging Face. For Enterprise organizations, credits are shared among all members. - - - -## How Billing Works: Choose Your Approach - -Inference Providers offers flexibility in how you're billed. Understanding these options upfront helps you choose the best approach for your needs: - -| Feature | **Routed by Hugging Face** | **Custom Provider Key** | **Direct Calls** | -| :--- | :--- | :--- | :--- | -| **How it Works** | Your request routes through HF to the provider | You set a custom provider key in HF settings | You provide the provider key directly in your code | -| **Billing** | Pay-as-you-go on your HF account | Billed directly by the provider | Billed directly by the provider | -| **Monthly Credits** | **✅ Yes** - Credits apply to eligible providers | **❌ No** - Credits don't apply | **❌ No** - Credits don't apply | -| **Provider Account Needed** | **❌ No** - We handle everything | **✅ Yes** - You need provider accounts | **✅ Yes** - You need provider accounts | -| **Best For** | Simplicity, experimentation, consolidated billing | More billing control, using non-integrated providers | Full control, bypassing HF routing | -| **Integration** | SDKs, Playground, widgets, Data AI Studio | SDKs, Playground, widgets, Data AI Studio | SDKs only | - -### Which Option Should I Choose? - -- **Start with Routed by Hugging Face** if you want simplicity and to use your monthly credits -- **Use Custom Provider Key** if you need specific provider features or you're consistently using the same provider -- **Use Direct Calls** if you want to bypass Hugging Face routing entirely - -## Pay-as-you-Go Details +| Tier | Included monthly credits | +| ---------------------------- | ------------------------------------ | +| Free Users | subject to change, less than $0.10 | +| PRO Users | $2.00 | +| Enterprise Hub Organizations | $2.00 per seat, shared among members | To benefit from Enterprise Hub included credits, you need to explicitly specify the organization to be billed when performing the inference requests. See the [Organization Billing section](#organization-billing) below for more details. -**PRO users and Enterprise Hub organizations** can continue using the API after exhausting their monthly credits. This ensures uninterrupted access to models for production workloads. +## Pay-as-you-Go -**Current Status**: Pay-as-you-Go is available for providers integrated with our billing system. We're actively integrating remaining providers—those not yet integrated will be blocked once free credits are exhausted. +**PRO users and Enterprise Hub organizations** can continue using the API once their monthly included credits are exhausted. This billing model, known as "Pay-as-you-Go" (PAYG), is charged on top of the monthly subscription. PAYG is only available for providers that are integrated with our billing system. We're actively working to integrate all providers, but in the meantime, any providers that are not yet integrated will be blocked once the free-tier limit is reached. If you have remaining credits, we estimate costs for providers that aren’t fully integrated with our billing system. These estimates are usually higher than the actual cost to prevent abuse, which is why PAYG is currently disabled for those providers. +You can track your spending on your [billing page](https://huggingface.co/settings/billing). + Hugging Face charges you the same rates as the provider, with no additional fees. We just pass through the provider costs directly. -You can track your spending anytime on your [billing page](https://huggingface.co/settings/billing). - -## Routed Requests vs Direct Calls (Detailed Comparison) +## Routed requests vs direct calls The documentation above assumes you are making routed requests to external providers. In practice, there are 3 different ways to run inference, each with unique billing implications: @@ -82,7 +51,7 @@ For instance, a request to [black-forest-labs/FLUX.1-dev](https://huggingface.co The `"hf-inference"` provider is currently the default provider when working with the JavaScript and Python SDKs. Note that this default might change in the future. -## Billing forEnterprise Hub organizations +## Organization billing For Enterprise Hub organizations, it is possible to centralize billing for all of your users. Each user still uses their own User Access Token but the requests are billed to your organization. This can be done by passing `"X-HF-Bill-To: my-org-name"` as a header in your HTTP requests. From 186e741e44e86cae391346f7cfd6b1e6a2a0f13c Mon Sep 17 00:00:00 2001 From: burtenshaw Date: Mon, 30 Jun 2025 14:20:58 +0200 Subject: [PATCH 07/15] add js implementation to first app --- .../guides/building-first-app.md | 363 +++++++++++++++++- .../guides/first-api-call.md | 5 +- 2 files changed, 364 insertions(+), 4 deletions(-) diff --git a/docs/inference-providers/guides/building-first-app.md b/docs/inference-providers/guides/building-first-app.md index 5debbc2535..5489a9a63c 100644 --- a/docs/inference-providers/guides/building-first-app.md +++ b/docs/inference-providers/guides/building-first-app.md @@ -12,10 +12,26 @@ Our app will: 3. **Generate summaries** using a powerful language model 4. **Deploy to the web** for easy sharing + + + **Tech Stack**: Gradio (for the UI) + Inference Providers (for the AI) + + + +**Tech Stack**: HTML/JavaScript (for the UI) + Inference Providers (for the AI) + +We'll use HTML and JavaScript for the UI just to keep things simple and agnostic, but if you want to see more mature examples, you can check out the [Hugging Face JS spaces](https://huggingface.co/huggingfacejs/spaces) page. + + + + ## Step 1: Set Up Authentication + + + Before we start coding, authenticate with Hugging Face using the CLI: ```bash @@ -25,8 +41,34 @@ huggingface-cli login When prompted, paste your Hugging Face token. This handles authentication automatically for all your inference calls. You can generate one from [your settings page](https://huggingface.co/settings/tokens/new?ownUserPermissions=inference.serverless.write&tokenType=fineGrained). + + + +You'll need your Hugging Face token. Get one from [your settings page](https://huggingface.co/settings/tokens/new?ownUserPermissions=inference.serverless.write&tokenType=fineGrained). We can set it as an environment variable in our app. + +```bash +export HF_TOKEN="your_token_here" +``` + +```javascript +// Add your token at the top of your script +const HF_TOKEN = process.env.HF_TOKEN; +``` + + + +When we deploy our app to Hugging Face Spaces, we'll need to add our token as a secret. This is a secure way to handle the token and avoid exposing it in the code. + + + + + + ## Step 2: Build the User Interface + + + Now let's create a simple web interface using Gradio: ```python @@ -59,14 +101,93 @@ if __name__ == "__main__": Here we're using Gradio's `gr.Audio` component to either upload an audio file or use the microphone input. We're keeping things simple with two outputs: a transcript and a summary with action items. + + + +For JavaScript, we'll create a clean HTML interface with native file upload and a simple loading state: + +```html + +

🎤 AI Meeting Notes

+ +
+ +

Upload audio file

+ +
+ +
Processing...
+ +
+
+

📝 Transcript

+
+
+
+

📋 Summary

+
+
+
+ +``` + +This creates a clean drag-and-drop interface with styled results sections for the transcript and summary. + +Our application can then use the `InferenceClient` from `huggingface.js` to call the transcription and summarization functions. + +```javascript +import { InferenceClient } from 'https://esm.sh/@huggingface/inference'; + +// Access the token from Hugging Face Spaces secrets +const HF_TOKEN = window.huggingface?.variables?.HF_TOKEN; +// Or if you're running locally, you can set it as an environment variable +// const HF_TOKEN = process.env.HF_TOKEN; + +document.getElementById('file').onchange = async (e) => { + if (!e.target.files[0]) return; + + const file = e.target.files[0]; + + show(document.getElementById('loading')); + hide(document.getElementById('results'), document.getElementById('error')); + + try { + console.log('🎤 Starting transcription...'); + const transcript = await transcribe(file); + console.log('✅ Transcription completed:', transcript.substring(0, 100) + '...'); + + console.log('🖊️ Starting summarization...'); + const summary = await summarize(transcript); + console.log('✅ Summary completed:', summary.substring(0, 100) + '...'); + + document.getElementById('transcript').textContent = transcript; + document.getElementById('summary').textContent = summary; + + hide(document.getElementById('loading')); + show(document.getElementById('results')); + } catch (error) { + hide(document.getElementById('loading')); + showError(`Error: ${error.message}`); + } +}; +``` + +We'll also need to implement the `transcribe` and `summarize` functions. + + + + ## Step 3: Add Speech Transcription + + + Now let's implement the transcription using `fal.ai` and OpenAI's `whisper-large-v3` model for fast, reliable speech processing: ```python def transcribe_audio(audio_file_path): """Transcribe audio using fal.ai for speed""" - client = InferenceClient(provider="fal-ai") + client = InferenceClient(provider="auto") # Pass the file path directly - the client handles file reading transcript = client.automatic_speech_recognition( @@ -77,9 +198,40 @@ def transcribe_audio(audio_file_path): return transcript.text ``` +Using the `auto` provider will automatically select the best provider for the model we're using. + + + + +We'll use the Hugging Face Inference client with automatic provider selection: + +```javascript +import { InferenceClient } from 'https://esm.sh/@huggingface/inference'; + +async function transcribe(file) { + const client = new InferenceClient(HF_TOKEN); + + const output = await client.automaticSpeechRecognition({ + data: file, + model: "openai/whisper-large-v3-turbo", + provider: "auto" + }); + + return output.text || output || 'Transcription completed'; +} +``` + +Using the `auto` provider will automatically select the best provider for the model we're using. + + + + ## Step 4: Add AI Summarization -Next, we'll use a powerful language model like `Qwen/Qwen3-235B-A22B-FP8` from Qwen via Together AI for summarization: + + + +Next, we'll use a powerful language model like `deepseek-ai/DeepSeek-R1-0528` from Qwen via Together AI for summarization: ```python def generate_summary(transcript): @@ -101,7 +253,7 @@ def generate_summary(transcript): """ response = client.chat.completions.create( - model="Qwen/Qwen3-235B-A22B-FP8", + model="deepseek-ai/DeepSeek-R1-0528", messages=[{"role": "user", "content": prompt}], max_tokens=1000 ) @@ -111,8 +263,54 @@ def generate_summary(transcript): Note, we're also defining a custom summary prompt to ensure the output is formatted as a summary with action items and decisions made. + + + +We'll use the chat completion API with automatic provider selection again, and define a custom prompt to ensure the output is formatted as a summary with action items and decisions made: + +```javascript +async function summarize(transcript) { + const client = new InferenceClient(HF_TOKEN); + + const prompt = `Analyze this meeting transcript and provide: + 1. A concise summary of key points + 2. Action items with responsible parties + 3. Important decisions made + + Transcript: ${transcript} + + Format with clear sections: + ## Summary + ## Action Items + ## Decisions Made`; + + const response = await client.chatCompletion({ + model: "deepseek-ai/DeepSeek-R1-0528", + messages: [ + { + role: "user", + content: prompt + } + ], + max_tokens: 1000 + }, { + provider: "auto" + }); + + return response.choices?.[0]?.message?.content || response || 'No summary available'; +} +``` + +We're using automatic provider selection which will choose the best available provider for the model. + + + + ## Step 5: Deploy on Hugging Face Spaces + + + To deploy, we'll need to create a `requirements.txt` file and a `app.py` file. `requirements.txt`: @@ -217,6 +415,165 @@ To deploy, we'll need to create a new Space and upload our files. > **Note**: While we used CLI authentication locally, Spaces requires the token as a secret for the deployment environment. + + + +For JavaScript deployment, create a simple static HTML file: + +
+📋 Click to view the complete index.html file + +```html + + + + + 🎤 AI Meeting Notes + + + +

🎤 AI Meeting Notes

+ +
+ +

Upload audio file

+ +
+ +
Processing...
+
+ +
+
+

📝 Transcript

+
+
+
+

📋 Summary

+
+
+
+ + + + +``` + +
+ +To deploy: + +1. **Create a new Space**: Go to [huggingface.co/new-space](https://huggingface.co/new-space) +2. **Choose Static SDK** and make it public +3. **Upload your file**: Upload `index.html` +4. **Add your token as a secret**: In Space settings, add `HF_TOKEN` as a **Secret** +5. **Launch**: Your app will be live at `https://huggingface.co/spaces/your-username/your-space-name` + +> **Note**: The token is securely managed by Hugging Face Spaces and accessed via `window.huggingface.variables.HF_TOKEN`. + + + + ## Next Steps Congratulations! You've created a production-ready AI application that: handles real-world tasks, provides a professional interface, scales automatically, and costs efficiently. If you want to explore more providers, you can check out the [Inference Providers](https://huggingface.co/inference-providers) page. Or here are some ideas for next steps: diff --git a/docs/inference-providers/guides/first-api-call.md b/docs/inference-providers/guides/first-api-call.md index 3b3f5c5dac..bd0f1ff674 100644 --- a/docs/inference-providers/guides/first-api-call.md +++ b/docs/inference-providers/guides/first-api-call.md @@ -66,7 +66,7 @@ Install the required package: pip install huggingface_hub ``` -You can now use the code snippet to generate an image: +You can now use the code snippet to generate an image in your app. ```python import os @@ -94,6 +94,8 @@ Install the required package: npm install @huggingface/inference ``` +Then, you can use the code snippet to generate an image in your app. + ```typescript import { InferenceClient } from "@huggingface/inference"; @@ -107,6 +109,7 @@ const image = await client.textToImage({ }); /// Use the generated image (it's a Blob) ``` + From 4212ef4cd50232f1fc4512b971aa158e1e684386 Mon Sep 17 00:00:00 2001 From: burtenshaw Date: Mon, 30 Jun 2025 14:25:12 +0200 Subject: [PATCH 08/15] use auto not together --- .../guides/building-first-app.md | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/docs/inference-providers/guides/building-first-app.md b/docs/inference-providers/guides/building-first-app.md index 5489a9a63c..93774ba5e1 100644 --- a/docs/inference-providers/guides/building-first-app.md +++ b/docs/inference-providers/guides/building-first-app.md @@ -231,12 +231,18 @@ Using the `auto` provider will automatically select the best provider for the mo -Next, we'll use a powerful language model like `deepseek-ai/DeepSeek-R1-0528` from Qwen via Together AI for summarization: +Next, we'll use a powerful language model like `deepseek-ai/DeepSeek-R1-0528` from DeepSeek via an Inference Provider. + + + +We'll use the `auto` provider to automatically select the best provider for the model. You can define your own priority list of providers in the [Inference Providers](https://huggingface.co/settings/inference-providers) page. + + ```python def generate_summary(transcript): - """Generate summary using Together AI""" - client = InferenceClient(provider="together") + """Generate summary using an Inference Provider""" + client = InferenceClient(provider="auto") prompt = f""" Analyze this meeting transcript and provide: @@ -343,8 +349,8 @@ def transcribe_audio(audio_file_path): def generate_summary(transcript): - """Generate summary using Together AI""" - client = InferenceClient(provider="together") + """Generate summary using an Inference Provider""" + client = InferenceClient(provider="auto") prompt = f""" Analyze this meeting transcript and provide: From 381be97bd67ef14e92f08663196bfae9755c41b8 Mon Sep 17 00:00:00 2001 From: burtenshaw Date: Mon, 30 Jun 2025 14:27:27 +0200 Subject: [PATCH 09/15] use auto in full app code --- .../guides/building-first-app.md | 17 ++++------------- 1 file changed, 4 insertions(+), 13 deletions(-) diff --git a/docs/inference-providers/guides/building-first-app.md b/docs/inference-providers/guides/building-first-app.md index 93774ba5e1..25847c0135 100644 --- a/docs/inference-providers/guides/building-first-app.md +++ b/docs/inference-providers/guides/building-first-app.md @@ -317,16 +317,7 @@ We're using automatic provider selection which will choose the best available pr -To deploy, we'll need to create a `requirements.txt` file and a `app.py` file. - -`requirements.txt`: - -```txt -gradio -huggingface_hub -``` - -`app.py`: +To deploy, we'll need to create an `app.py` file and upload it to Hugging Face Spaces.
📋 Click to view the complete app.py file @@ -337,8 +328,8 @@ from huggingface_hub import InferenceClient def transcribe_audio(audio_file_path): - """Transcribe audio using fal.ai for speed""" - client = InferenceClient(provider="fal-ai") + """Transcribe audio using an Inference Provider""" + client = InferenceClient(provider="auto") # Pass the file path directly - the client handles file reading transcript = client.automatic_speech_recognition( @@ -367,7 +358,7 @@ def generate_summary(transcript): """ response = client.chat.completions.create( - model="Qwen/Qwen3-235B-A22B-FP8", + model="deepseek-ai/DeepSeek-R1-0528", messages=[{"role": "user", "content": prompt}], max_tokens=1000, ) From 4316d658882b052c5676d46139d0b9adbe0400db Mon Sep 17 00:00:00 2001 From: burtenshaw Date: Mon, 30 Jun 2025 14:32:49 +0200 Subject: [PATCH 10/15] add app screenshots --- docs/inference-providers/guides/building-first-app.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/docs/inference-providers/guides/building-first-app.md b/docs/inference-providers/guides/building-first-app.md index 25847c0135..d2f94e5235 100644 --- a/docs/inference-providers/guides/building-first-app.md +++ b/docs/inference-providers/guides/building-first-app.md @@ -400,6 +400,10 @@ if __name__ == "__main__": app.launch() ``` +Our app will run on port 7860 and look like this: + +![Gradio app](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/inference-providers-guides/gradio-app.png) +
To deploy, we'll need to create a new Space and upload our files. @@ -556,6 +560,10 @@ For JavaScript deployment, create a simple static HTML file: ``` +We can run our app locally by going to the file from our browser. + +![Local app](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/inference-providers-guides/js-app.png) + To deploy: From 2f56129d14f0a29c587c97f8201c9df1a419da88 Mon Sep 17 00:00:00 2001 From: burtenshaw Date: Mon, 30 Jun 2025 14:35:16 +0200 Subject: [PATCH 11/15] use auto in first api call --- docs/inference-providers/guides/first-api-call.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/inference-providers/guides/first-api-call.md b/docs/inference-providers/guides/first-api-call.md index bd0f1ff674..aaa2c3b1a6 100644 --- a/docs/inference-providers/guides/first-api-call.md +++ b/docs/inference-providers/guides/first-api-call.md @@ -14,7 +14,7 @@ This guide assumes you have a Hugging Face account. If you don't have one, you c ## Step 1: Find a Model on the Hub -Visit the [Hugging Face Hub](https://huggingface.co/models) and look for models with the "Inference Providers" filter, you can select the provider that you want. We'll go with `fal`. +Visit the [Hugging Face Hub](https://huggingface.co/models?pipeline_tag=text-to-image&inference_provider=fal-ai,hf-inference,nebius,nscale,replicate,together&sort=trending) and look for models with the "Inference Providers" filter, you can select the provider that you want. We'll go with `fal`. ![search image](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/inference-providers-guides/search.png) @@ -22,7 +22,7 @@ For this example, we'll use [FLUX.1-schnell](https://huggingface.co/black-forest ## Step 2: Try the Interactive Widget -Before writing any code, try the widget directly on the model page: +Before writing any code, try the widget directly on the [model page](https://huggingface.co/black-forest-labs/FLUX.1-dev?inference_provider=fal-ai): ![widget image](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/inference-providers-guides/widget.png) @@ -42,7 +42,7 @@ You'll need a Hugging Face account (free at [huggingface.co](https://huggingface ## Step 3: From Clicks to Code -Now let's replicate this with Python. Click the **"View Code Snippets"** button in the widget to see the generated code snippets. +Now let's replicate this with Python. Click the **"View Code Snippets"** button in the widget to see the [generated code snippets](https://huggingface.co/black-forest-labs/FLUX.1-dev?inference_api=true&language=python&inference_provider=auto). ![code snippets image](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/inference-providers-guides/code-snippets.png) @@ -73,7 +73,7 @@ import os from huggingface_hub import InferenceClient client = InferenceClient( - provider="fal-ai", + provider="auto", api_key=os.environ["HF_TOKEN"], ) @@ -102,7 +102,7 @@ import { InferenceClient } from "@huggingface/inference"; const client = new InferenceClient(process.env.HF_TOKEN); const image = await client.textToImage({ - provider: "fal-ai", + provider: "auto", model: "black-forest-labs/FLUX.1-schnell", inputs: "Astronaut riding a horse", parameters: { num_inference_steps: 5 }, From 9b7050b9eb5877f2d660abe3e5ad7125deacfd0b Mon Sep 17 00:00:00 2001 From: burtenshaw Date: Mon, 30 Jun 2025 14:40:13 +0200 Subject: [PATCH 12/15] simplify js logic --- docs/inference-providers/guides/building-first-app.md | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/docs/inference-providers/guides/building-first-app.md b/docs/inference-providers/guides/building-first-app.md index d2f94e5235..7b53060d11 100644 --- a/docs/inference-providers/guides/building-first-app.md +++ b/docs/inference-providers/guides/building-first-app.md @@ -152,14 +152,9 @@ document.getElementById('file').onchange = async (e) => { hide(document.getElementById('results'), document.getElementById('error')); try { - console.log('🎤 Starting transcription...'); const transcript = await transcribe(file); - console.log('✅ Transcription completed:', transcript.substring(0, 100) + '...'); - - console.log('🖊️ Starting summarization...'); const summary = await summarize(transcript); - console.log('✅ Summary completed:', summary.substring(0, 100) + '...'); - + document.getElementById('transcript').textContent = transcript; document.getElementById('summary').textContent = summary; From 49f164b8f4f5a43c0ef6d0bde2391c4d1801462f Mon Sep 17 00:00:00 2001 From: burtenshaw Date: Tue, 1 Jul 2025 10:33:16 +0200 Subject: [PATCH 13/15] Update docs/inference-providers/guides/building-first-app.md --- docs/inference-providers/guides/building-first-app.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/inference-providers/guides/building-first-app.md b/docs/inference-providers/guides/building-first-app.md index 7b53060d11..b4500a5ac6 100644 --- a/docs/inference-providers/guides/building-first-app.md +++ b/docs/inference-providers/guides/building-first-app.md @@ -405,7 +405,7 @@ To deploy, we'll need to create a new Space and upload our files. 1. **Create a new Space**: Go to [huggingface.co/new-space](https://huggingface.co/new-space) 2. **Choose Gradio SDK** and make it public -3. **Upload your files**: Upload `app.py` and `requirements.txt` +3. **Upload your files**: Upload `app.py` 4. **Add your token**: In Space settings, add `HF_TOKEN` as a secret (get it from [your settings](https://huggingface.co/settings/tokens)) 5. **Launch**: Your app will be live at `https://huggingface.co/spaces/your-username/your-space-name` From 26f90272b310fa0c7dccc69757481ce0198add20 Mon Sep 17 00:00:00 2001 From: burtenshaw Date: Tue, 1 Jul 2025 10:34:11 +0200 Subject: [PATCH 14/15] Apply suggestions from code review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: célina --- .../guides/building-first-app.md | 26 +++++++++---------- 1 file changed, 12 insertions(+), 14 deletions(-) diff --git a/docs/inference-providers/guides/building-first-app.md b/docs/inference-providers/guides/building-first-app.md index b4500a5ac6..b69d567ee2 100644 --- a/docs/inference-providers/guides/building-first-app.md +++ b/docs/inference-providers/guides/building-first-app.md @@ -177,7 +177,10 @@ We'll also need to implement the `transcribe` and `summarize` functions. -Now let's implement the transcription using `fal.ai` and OpenAI's `whisper-large-v3` model for fast, reliable speech processing: +Now let's implement the transcription using OpenAI's `whisper-large-v3` model for fast, reliable speech processing. + +We'll use the `auto` provider to automatically select the first available provider for the model. You can define your own priority list of providers in the [Inference Providers](https://huggingface.co/settings/inference-providers) page. + ```python def transcribe_audio(audio_file_path): @@ -193,12 +196,14 @@ def transcribe_audio(audio_file_path): return transcript.text ``` -Using the `auto` provider will automatically select the best provider for the model we're using. -We'll use the Hugging Face Inference client with automatic provider selection: +Now let's implement the transcription using OpenAI's `whisper-large-v3` model for fast, reliable speech processing. + +We'll use the `auto` provider to automatically select the first available provider for the model. You can define your own priority list of providers in the [Inference Providers](https://huggingface.co/settings/inference-providers) page. + ```javascript import { InferenceClient } from 'https://esm.sh/@huggingface/inference'; @@ -216,7 +221,6 @@ async function transcribe(file) { } ``` -Using the `auto` provider will automatically select the best provider for the model we're using. @@ -226,13 +230,8 @@ Using the `auto` provider will automatically select the best provider for the mo -Next, we'll use a powerful language model like `deepseek-ai/DeepSeek-R1-0528` from DeepSeek via an Inference Provider. - - - -We'll use the `auto` provider to automatically select the best provider for the model. You can define your own priority list of providers in the [Inference Providers](https://huggingface.co/settings/inference-providers) page. - - +Next, we'll use a powerful language model like `deepseek-ai/DeepSeek-R1-0528` from DeepSeek via an Inference Provider, and just like in the previous step, we'll use the `auto` provider to automatically select the first available provider for the model. +We will define a custom prompt to ensure the output is formatted as a summary with action items and decisions made: ```python def generate_summary(transcript): @@ -262,12 +261,12 @@ def generate_summary(transcript): return response.choices[0].message.content ``` -Note, we're also defining a custom summary prompt to ensure the output is formatted as a summary with action items and decisions made. -We'll use the chat completion API with automatic provider selection again, and define a custom prompt to ensure the output is formatted as a summary with action items and decisions made: +Next, we'll use a powerful language model like `deepseek-ai/DeepSeek-R1-0528` from DeepSeek via an Inference Provider, and just like in the previous step, we'll use the `auto` provider to automatically select the first available provider for the model. +We will define a custom prompt to ensure the output is formatted as a summary with action items and decisions made: ```javascript async function summarize(transcript) { @@ -302,7 +301,6 @@ async function summarize(transcript) { } ``` -We're using automatic provider selection which will choose the best available provider for the model. From cd49e24ec58bd834861caf0994e6e14ce4d65272 Mon Sep 17 00:00:00 2001 From: burtenshaw Date: Thu, 3 Jul 2025 09:50:03 +0200 Subject: [PATCH 15/15] add section on specify provider vs auto --- .../guides/first-api-call.md | 106 ++++++++++++++++++ 1 file changed, 106 insertions(+) diff --git a/docs/inference-providers/guides/first-api-call.md b/docs/inference-providers/guides/first-api-call.md index aaa2c3b1a6..24dd396499 100644 --- a/docs/inference-providers/guides/first-api-call.md +++ b/docs/inference-providers/guides/first-api-call.md @@ -124,6 +124,112 @@ Nice work! You've successfully used a production-grade AI model without any comp The model you just used runs on professional infrastructure, handling scaling, optimization, and reliability automatically. +## Dive Deeper: Provider Selection + +You might have noticed the `provider="auto"` parameter in the code examples above. This is a key feature of Inference Providers that gives you control over which infrastructure provider handles your request. + +`auto` is powerful because: + +1. It makes it easy to switch between providers, and to test different providers' performance for your use case. +2. It also gives a fallback mechanism in case a provider is unavailable. + +But if you want to be more specific, you can also specify a provider. Let's see how. + +### Understanding Provider Selection + +When you use `provider="auto"` (which is the default), the system automatically selects the first available provider for your chosen model based on your preference order in your [Inference Provider settings](https://hf.co/settings/inference-providers). This provides: + +- **Automatic failover**: If one provider is unavailable, the system tries the next one +- **Simplified setup**: No need to research which providers support your model +- **Optimal routing**: The system handles provider selection for you + +### Specifying a Specific Provider + +Alternatively, you can explicitly choose a provider if you have specific requirements: + + + + + +```python +import os +from huggingface_hub import InferenceClient + +client = InferenceClient(api_key=os.environ["HF_TOKEN"]) + +# Using automatic provider selection (default) +image_auto = client.text_to_image( + "Astronaut riding a horse", + model="black-forest-labs/FLUX.1-schnell", + provider="auto" # This is the default +) + +# Using a specific provider +image_fal = client.text_to_image( + "Astronaut riding a horse", + model="black-forest-labs/FLUX.1-schnell", + provider="fal-ai" # Explicitly use Fal AI +) + +# Using another specific provider +image_replicate = client.text_to_image( + "Astronaut riding a horse", + model="black-forest-labs/FLUX.1-schnell", + provider="replicate" # Explicitly use Replicate +) +``` + + + + + +```typescript +import { InferenceClient } from "@huggingface/inference"; + +const client = new InferenceClient(process.env.HF_TOKEN); + +// Using automatic provider selection (default) +const imageAuto = await client.textToImage({ + model: "black-forest-labs/FLUX.1-schnell", + inputs: "Astronaut riding a horse", + provider: "auto", // This is the default + parameters: { num_inference_steps: 5 }, +}); + +// Using a specific provider +const imageFal = await client.textToImage({ + model: "black-forest-labs/FLUX.1-schnell", + inputs: "Astronaut riding a horse", + provider: "fal-ai", // Explicitly use Fal AI + parameters: { num_inference_steps: 5 }, +}); + +// Using another specific provider +const imageReplicate = await client.textToImage({ + model: "black-forest-labs/FLUX.1-schnell", + inputs: "Astronaut riding a horse", + provider: "replicate", // Explicitly use Replicate + parameters: { num_inference_steps: 5 }, +}); +``` + + + + + +### When to Use Each Approach + +**Use `provider="auto"` when:** +- You're just getting started with Inference Providers +- You want the simplest setup and maximum reliability +- You don't have specific infrastructure requirements +- You want automatic failover if a provider is unavailable + +**Use a specific provider when:** +- You need consistent performance characteristics +- You have specific billing or cost requirements +- You want to test different providers' performance for your use case + ## Next Steps Now that you've seen how easy it is to use AI models, you might wonder: