Skip to content

Commit 0844ea4

Browse files
Fix model list output, indicate project is optional (#1613)
Description ----------- - Team/project is now optional, fix several mentions of it being required. - The chat completions output was appearing below the models list API. Move it to proper place and heading level, and add an example of model list API output. <!-- preview-links-comment --> 📄 **[View preview links for changed pages](#1613 (comment) --------- Co-authored-by: John Mulhausen <john.mulhausen@wandb.com>
1 parent 0997894 commit 0844ea4

3 files changed

Lines changed: 60 additions & 34 deletions

File tree

content/en/guides/inference/_index.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -22,11 +22,11 @@ import openai
2222
client = openai.OpenAI(
2323
# The custom base URL points to W&B Inference
2424
base_url='https://api.inference.wandb.ai/v1',
25-
25+
2626
# Get your API key from https://wandb.ai/authorize
2727
api_key="<your-api-key>",
28-
29-
# Team and project are required for usage tracking
28+
29+
# Optional: Team and project for usage tracking
3030
project="<your-team>/<your-project>",
3131
)
3232

@@ -53,10 +53,10 @@ print(response.choices[0].message.content)
5353
{{< alert title="Important" color="warning" >}}
5454
W&B Inference credits come with Free, Pro, and Academic plans for a limited time. Availability may vary for Enterprise accounts. When credits run out:
5555

56-
- **Free users** must upgrade to a paid plan to continue using Inference.
56+
- **Free users** must upgrade to a paid plan to continue using Inference.
5757
[Upgrade to Pro or Enterprise](https://wandb.ai/subscriptions)
5858
- **Pro users** are billed monthly for usage beyond free credits, up to a default cap of $6,000/month. See [Account tiers and default usage caps]({{< relref "usage-limits#account-tiers-and-default-usage-caps" >}})
5959
- **Enterprise usage** is capped at $700,000/year. Your account executive handles billing and limit increases. See [Account tiers and default usage caps]({{< relref "usage-limits#account-tiers-and-default-usage-caps" >}})
6060

6161
To learn more, visit the [pricing page](https://wandb.ai/site/pricing/) or see [model-specific costs](https://wandb.ai/site/pricing/inference).
62-
{{< /alert >}}
62+
{{< /alert >}}

content/en/guides/inference/api-reference.md

Lines changed: 52 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -24,9 +24,8 @@ https://api.inference.wandb.ai/v1
2424
To use this endpoint, you need:
2525
- A W&B account with Inference credits
2626
- A valid W&B API key
27-
- A W&B entity (team) and project
2827

29-
In code samples, these appear as `<your-team>/<your-project>`.
28+
If you belong to more than one team or want to attribute your usage to a project you will also need team and project IDs. In code samples, these appear as `<your-team>/<your-project>`. Your default entity and the project name `inference` will be used if unspecified.
3029
{{< /alert >}}
3130

3231
## Available methods
@@ -40,7 +39,7 @@ Create a chat completion using the `/chat/completions` endpoint. This endpoint f
4039
To create a chat completion, provide:
4140
- The Inference service base URL: `https://api.inference.wandb.ai/v1`
4241
- Your W&B API key: `<your-api-key>`
43-
- Your W&B entity and project: `<your-team>/<your-project>`
42+
- Optional: Your W&B team and project: `<your-team>/<your-project>`
4443
- A model ID from the [available models]({{< relref "models" >}})
4544

4645
{{< tabpane text=true >}}
@@ -74,7 +73,7 @@ client = openai.OpenAI(
7473
# Consider setting it in the environment as OPENAI_API_KEY instead for safety
7574
api_key="<your-api-key>",
7675

77-
# Team and project are required for usage tracking
76+
# Optional: Team and project for usage tracking
7877
project="<your-team>/<your-project>",
7978
)
8079

@@ -93,6 +92,34 @@ print(response.choices[0].message.content)
9392
{{% /tab %}}
9493
{{< /tabpane >}}
9594

95+
#### Response format
96+
97+
The API returns responses in OpenAI-compatible format:
98+
99+
```json
100+
{
101+
"id": "chatcmpl-...",
102+
"object": "chat.completion",
103+
"created": 1234567890,
104+
"model": "meta-llama/Llama-3.1-8B-Instruct",
105+
"choices": [
106+
{
107+
"index": 0,
108+
"message": {
109+
"role": "assistant",
110+
"content": "Here's a joke for you..."
111+
},
112+
"finish_reason": "stop"
113+
}
114+
],
115+
"usage": {
116+
"prompt_tokens": 25,
117+
"completion_tokens": 50,
118+
"total_tokens": 75
119+
}
120+
}
121+
```
122+
96123
### List supported models
97124

98125
Get all available models and their IDs. Use this to select models dynamically or check what's available.
@@ -104,7 +131,7 @@ Get all available models and their IDs. Use this to select models dynamically or
104131
curl https://api.inference.wandb.ai/v1/models \
105132
-H "Content-Type: application/json" \
106133
-H "Authorization: Bearer <your-api-key>" \
107-
-H "OpenAI-Project: <your-team>/<your-project>"
134+
-H "OpenAI-Project: <your-team>/<your-project>"
108135
```
109136

110137
{{% /tab %}}
@@ -116,7 +143,7 @@ import openai
116143
client = openai.OpenAI(
117144
base_url="https://api.inference.wandb.ai/v1",
118145
api_key="<your-api-key>",
119-
project="<your-team>/<your-project>"
146+
project="<your-team>/<your-project>" # Optional, for usage tracking
120147
)
121148

122149
response = client.models.list()
@@ -128,35 +155,34 @@ for model in response.data:
128155
{{% /tab %}}
129156
{{< /tabpane >}}
130157

131-
## Response format
158+
#### Response format
132159

133160
The API returns responses in OpenAI-compatible format:
134161

135162
```json
136163
{
137-
"id": "chatcmpl-...",
138-
"object": "chat.completion",
139-
"created": 1234567890,
140-
"model": "meta-llama/Llama-3.1-8B-Instruct",
141-
"choices": [
164+
"object": "list",
165+
"data": [
142166
{
143-
"index": 0,
144-
"message": {
145-
"role": "assistant",
146-
"content": "Here's a joke for you..."
147-
},
148-
"finish_reason": "stop"
149-
}
150-
],
151-
"usage": {
152-
"prompt_tokens": 25,
153-
"completion_tokens": 50,
154-
"total_tokens": 75
155-
}
167+
"id": "deepseek-ai/DeepSeek-V3.1",
168+
"object": "model",
169+
"created": 0,
170+
"owned_by": "system",
171+
"root": "deepseek-ai/DeepSeek-V3.1"
172+
},
173+
{
174+
"id": "openai/gpt-oss-20b",
175+
"object": "model",
176+
"created": 0,
177+
"owned_by": "system",
178+
"root": "openai/gpt-oss-20b"
179+
},
180+
...
181+
]
156182
}
157183
```
158184

159185
## Next steps
160186

161187
- Try the [usage examples]({{< relref "examples" >}}) to see the API in action
162-
- Explore models in the [UI]({{< relref "ui-guide" >}})
188+
- Explore models in the [UI]({{< relref "ui-guide" >}})

content/en/guides/inference/examples.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@ client = openai.OpenAI(
3737
# Get your API key from https://wandb.ai/authorize
3838
api_key="<your-api-key>",
3939

40-
# Required for W&B inference usage tracking
40+
# Optional: Team and project for usage tracking
4141
project="wandb/inference-demo",
4242
)
4343

@@ -97,7 +97,7 @@ class WBInferenceModel(weave.Model):
9797
base_url="https://api.inference.wandb.ai/v1",
9898
# Get your API key from https://wandb.ai/authorize
9999
api_key="<your-api-key>",
100-
# Required for W&B inference usage tracking
100+
# Optional: Team and project for usage tracking
101101
project="<your-team>/<your-project>",
102102
)
103103
resp = client.chat.completions.create(
@@ -153,4 +153,4 @@ After running this code, go to your W&B account at [https://wandb.ai/](https://w
153153
## Next steps
154154

155155
- Explore the [API reference]({{< relref "api-reference" >}}) for all available methods
156-
- Try models in the [UI]({{< relref "ui-guide" >}})
156+
- Try models in the [UI]({{< relref "ui-guide" >}})

0 commit comments

Comments
 (0)