Skip to content

models.mdx

Latest commit

 

History

History
163 lines (120 loc) · 5.26 KB

models.mdx

File metadata and controls

163 lines (120 loc) · 5.26 KB
Raw
title Models
openapi get /models

Endpoint Behavior

The GET /v1/models endpoint behaves differently depending on whether your request carries provider routing signals (headers or config). Understanding this is important when you're deciding which model list your application should consume.

Two response modes

Returns the models configured in your Portkey **Model Catalog** — the ones your workspace can actually route to, including any custom models you've added. Proxies the request to the upstream provider's own `/v1/models` endpoint (e.g. OpenAI, Anthropic, Bedrock) and returns their live model list.

How Portkey decides

Portkey inspects your request for any of the following provider signals:

  • x-portkey-provider header
  • x-portkey-virtual-key header
  • x-portkey-config with provider, virtual_key, or targets set
Provider signal present? fetch_integrated_models set? Response
No — Portkey integrated models (model catalog)
Yes No Provider-backed — proxied to the provider's /v1/models
Yes Yes Portkey integrated models, filtered to the providers referenced in your config
Because the default behavior with a provider signal is to hit the upstream, the response you see depends on which provider you're pointed at. Anthropic returns Claude models, OpenAI returns GPT models, and so on. If you want a single unified view, use the integrated mode described below.

Provider-Backed Mode

When you send a provider signal (e.g. x-portkey-provider: @openai-prod), Portkey calls that provider's /v1/models endpoint and returns the response.

Response format

By default, Portkey normalizes the provider's response into the standard OpenAI models list format — the same shape as openai.com/v1/models:

{
  "object": "list",
  "data": [
    {
      "id": "claude-3-5-sonnet-20241022",
      "object": "model",
      "created": 1729555200,
      "owned_by": "anthropic"
    }
  ],
  "provider": "anthropic"
}

To receive the raw provider response without normalization, set strict_open_ai_compliance to false. See Strict OpenAI Compliance for details.

curl https://api.portkey.ai/v1/models \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -H "x-portkey-provider: @anthropic-prod" \
  -H "x-portkey-strict-open-ai-compliance: false"
from portkey_ai import Portkey

portkey = Portkey(
    api_key="PORTKEY_API_KEY",
    provider="@anthropic-prod",
    strict_open_ai_compliance=False
)

models = portkey.models.list()
import Portkey from 'portkey-ai';

const portkey = new Portkey({
  apiKey: 'PORTKEY_API_KEY',
  provider: '@anthropic-prod',
  strictOpenAiCompliance: false
});

const models = await portkey.models.list();

Supported providers

Native listModels support (with OpenAI-format normalization) is available for:

  • OpenAI
  • Azure OpenAI
  • Anthropic
  • Google (Gemini)
  • Google Vertex AI
  • AWS Bedrock

For providers without a native listModels transform, the raw provider response is returned as-is.

Integrated Mode Override

Even when a provider signal is present, you can force Portkey to return your integrated (model catalog) models — the ones set up in your workspace — by using one of the following:

Header. Set to `true` to return integrated models even when a provider/virtual-key/config is passed on the request. Config field. Set to `true` on your Portkey config to achieve the same effect.

When active, Portkey filters the integrated model list to only the provider slugs referenced in your routing config (including virtual keys and @provider/model overrides). If any leaf in the config uses passthrough: true, all integrated models are returned unfiltered.

curl https://api.portkey.ai/v1/models \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -H "x-portkey-provider: @openai-prod" \
  -H "x-portkey-fetch-integrated-models: true"
{
  "fetch_integrated_models": true,
  "targets": [
    { "virtual_key": "openai-prod" },
    { "virtual_key": "anthropic-prod" }
  ]
}
Use this when your application logic depends on your workspace's curated model list — for example, to show end users only the models your team has approved and configured — even though the request still needs to carry provider auth for downstream routing.

Choosing a mode

If you want to… Use
Show the full list of models the provider currently offers Provider-backed (default with a provider signal)
Show only the models available in your Portkey workspace Send no provider signal, or set x-portkey-fetch-integrated-models: true
Get the untouched provider payload (extra fields, provider-specific metadata) Provider-backed + strict_open_ai_compliance: false