Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions api-reference/inference-api/headers.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -403,6 +403,32 @@ const portkey = new Portkey({
</CodeGroup>
</Accordion>

### Fetch Integrated Models
<ResponseField name="x-portkey-fetch-integrated-models" type="boolean">
Applies to `GET /v1/models` only. When `true`, forces the endpoint to return Portkey's integrated (Model Catalog) models even when a provider, virtual key, or config is passed on the request. Without this header, any provider signal causes the endpoint to proxy to the upstream provider's `/v1/models`. You can also set this as `fetch_integrated_models: true` inside a Portkey config. ([Docs](/api-reference/inference-api/models/models))
</ResponseField>
<Accordion title="Example">
<CodeGroup>

```sh cURL {4}
curl https://api.portkey.ai/v1/models \
-H "x-portkey-api-key: $PORTKEY_API_KEY" \
-H "x-portkey-provider: @openai-prod" \
-H "x-portkey-fetch-integrated-models: true"
```

```json Config
{
"fetch_integrated_models": true,
"targets": [
{ "virtual_key": "openai-prod" },
{ "virtual_key": "anthropic-prod" }
]
}
```
</CodeGroup>
</Accordion>

## Custom Headers

You can pass any other headers your API expects by directly forwarding them without any processing by Portkey.<br />
Expand Down
159 changes: 159 additions & 0 deletions api-reference/inference-api/models/models.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,162 @@
title: Models
openapi: get /models
---

## Endpoint Behavior

The `GET /v1/models` endpoint behaves differently depending on whether your request carries provider routing signals (headers or config). Understanding this is important when you're deciding which model list your application should consume.

### Two response modes

<CardGroup cols={2}>
<Card title="Integrated (default when no provider)" icon="database">
Returns the models configured in your Portkey **Model Catalog** — the ones your workspace can actually route to, including any custom models you've added.
</Card>
<Card title="Provider-backed" icon="cloud">
Proxies the request to the upstream provider's own `/v1/models` endpoint (e.g. OpenAI, Anthropic, Bedrock) and returns their live model list.
</Card>
</CardGroup>

### How Portkey decides

Portkey inspects your request for any of the following **provider signals**:

- `x-portkey-provider` header
- `x-portkey-virtual-key` header
- `x-portkey-config` with `provider`, `virtual_key`, or `targets` set

| Provider signal present? | `fetch_integrated_models` set? | Response |
| :-- | :-- | :-- |
| No | — | Portkey **integrated models** (model catalog) |
| Yes | No | **Provider-backed** — proxied to the provider's `/v1/models` |
| Yes | Yes | Portkey **integrated models**, filtered to the providers referenced in your config |

<Note>
Because the default behavior with a provider signal is to hit the upstream, the response you see depends on which provider you're pointed at. Anthropic returns Claude models, OpenAI returns GPT models, and so on. If you want a single unified view, use the integrated mode described below.
</Note>

---

## Provider-Backed Mode

When you send a provider signal (e.g. `x-portkey-provider: @openai-prod`), Portkey calls that provider's `/v1/models` endpoint and returns the response.

### Response format

By default, Portkey normalizes the provider's response into the standard **OpenAI models list format** — the same shape as `openai.com/v1/models`:

```json
{
"object": "list",
"data": [
{
"id": "claude-3-5-sonnet-20241022",
"object": "model",
"created": 1729555200,
"owned_by": "anthropic"
}
],
"provider": "anthropic"
}
```

To receive the **raw provider response** without normalization, set `strict_open_ai_compliance` to `false`. See [Strict OpenAI Compliance](/product/ai-gateway/strict-open-ai-compliance) for details.

<CodeGroup>

```sh cURL
curl https://api.portkey.ai/v1/models \
-H "x-portkey-api-key: $PORTKEY_API_KEY" \
-H "x-portkey-provider: @anthropic-prod" \
-H "x-portkey-strict-open-ai-compliance: false"
```

```py Python
from portkey_ai import Portkey

portkey = Portkey(
api_key="PORTKEY_API_KEY",
provider="@anthropic-prod",
strict_open_ai_compliance=False
)

models = portkey.models.list()
```

```js JavaScript
import Portkey from 'portkey-ai';

const portkey = new Portkey({
apiKey: 'PORTKEY_API_KEY',
provider: '@anthropic-prod',
strictOpenAiCompliance: false
});

const models = await portkey.models.list();
```

</CodeGroup>

### Supported providers

Native `listModels` support (with OpenAI-format normalization) is available for:

- OpenAI
- Azure OpenAI
- Anthropic
- Google (Gemini)
- Google Vertex AI
- AWS Bedrock

For providers without a native `listModels` transform, the raw provider response is returned as-is.

---

## Integrated Mode Override

Even when a provider signal is present, you can force Portkey to return your **integrated (model catalog) models** — the ones set up in your workspace — by using one of the following:

<ResponseField name="x-portkey-fetch-integrated-models" type="boolean">
Header. Set to `true` to return integrated models even when a provider/virtual-key/config is passed on the request.
</ResponseField>

<ResponseField name="fetch_integrated_models" type="boolean">
Config field. Set to `true` on your Portkey config to achieve the same effect.
</ResponseField>

When active, Portkey filters the integrated model list to only the provider slugs referenced in your routing config (including virtual keys and `@provider/model` overrides). If any leaf in the config uses `passthrough: true`, all integrated models are returned unfiltered.

<CodeGroup>

```sh cURL {4}
curl https://api.portkey.ai/v1/models \
-H "x-portkey-api-key: $PORTKEY_API_KEY" \
-H "x-portkey-provider: @openai-prod" \
-H "x-portkey-fetch-integrated-models: true"
```

```json Config
{
"fetch_integrated_models": true,
"targets": [
{ "virtual_key": "openai-prod" },
{ "virtual_key": "anthropic-prod" }
]
}
```

</CodeGroup>

<Tip>
Use this when your application logic depends on your workspace's curated model list — for example, to show end users only the models your team has approved and configured — even though the request still needs to carry provider auth for downstream routing.
</Tip>

---

## Choosing a mode

| If you want to… | Use |
| :-- | :-- |
| Show the full list of models the provider currently offers | Provider-backed (default with a provider signal) |
| Show only the models available in your Portkey workspace | Send no provider signal, **or** set `x-portkey-fetch-integrated-models: true` |
| Get the untouched provider payload (extra fields, provider-specific metadata) | Provider-backed + `strict_open_ai_compliance: false` |
6 changes: 6 additions & 0 deletions changelog/enterprise.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,12 @@ Gen AI and guardrail OTel spans now carry Portkey tenant attributes — org ID,

[OpenTelemetry Documentation](/product/observability/opentelemetry)

### Unified `/v1/models` Handler

Provider-backed `GET /v1/models` requests now flow through a dedicated `listModels` handler that proxies to the upstream provider (OpenAI, Azure OpenAI, Anthropic, Google, Vertex AI, Bedrock) and normalizes the response to OpenAI's models list format. Setting `strict_open_ai_compliance: false` returns the raw provider payload.

[Models API Documentation](/api-reference/inference-api/models/models)

### Provider Updates

- **Anthropic**: `thinking.display` parameter is now passed through to the upstream API, enabling clients to control whether extended thinking content is returned in responses
Expand Down