apisix/docs/en/latest/plugins/ai-request-rewrite.md at master · apache/apisix

title

ai-request-rewrite

keywords

Apache APISIX

AI Gateway

Plugin

ai-request-rewrite

description

The ai-request-rewrite plugin forwards client requests to LLM services for processing before sending them upstream, enabling AI-driven redaction, enrichment, and reformatting.

import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem';

Description

The ai-request-rewrite Plugin processes client requests by forwarding them to LLM services for transformation before relaying them to Upstream services. This enables LLM-powered modifications such as data redaction, content enrichment, or reformatting. The Plugin supports integration with OpenAI, DeepSeek, Gemini, Vertex AI, Anthropic, OpenRouter, and other OpenAI-compatible APIs.

Plugin Attributes

Name	Type	Required	Default	Valid values	Description
`prompt`	string	True			The prompt to send to the LLM service for rewriting the client request.
`provider`	string	True		[openai, deepseek, azure-openai, aimlapi, gemini, vertex-ai, anthropic, openrouter, openai-compatible]	LLM service provider. When set to `aimlapi`, the Plugin uses the OpenAI-compatible driver and proxies the request to `https://api.aimlapi.com/v1/chat/completions`. When set to `openai-compatible`, the Plugin proxies requests to the custom endpoint configured in `override`. When set to `azure-openai`, the Plugin also proxies requests to the custom endpoint configured in `override` and additionally omits the `model` parameter from the request body sent to Azure OpenAI.
`auth`	object	True			Authentication configurations.
`auth.header`	object	False			Authentication headers. Key must match pattern `^[a-zA-Z0-9._-]+$`. At least one of `header` and `query` should be configured.
`auth.query`	object	False			Authentication query parameters. Key must match pattern `^[a-zA-Z0-9._-]+$`. At least one of `header` and `query` should be configured.
`options`	object	False			Model configurations. In addition to `model`, you can configure additional parameters and they will be forwarded to the upstream LLM service in the request body. For instance, if you are working with OpenAI, you can configure additional parameters such as `temperature`, `top_p`, and `stream`. See your LLM provider's API documentation for more available options.
`options.model`	string	False			Name of the LLM model, such as `gpt-4` or `gpt-3.5`. See your LLM provider's API documentation for more available models.
`override`	object	False			Override setting.
`override.endpoint`	string	False			LLM provider endpoint. Required when `provider` is `openai-compatible`.
`timeout`	integer	False	30000	1 - 60000	Request timeout in milliseconds when requesting the LLM service.
`keepalive`	boolean	False	true		If true, keep the connection alive when requesting the LLM service.
`keepalive_timeout`	integer	False	60000	≥ 1000	Keepalive timeout in milliseconds for requests to the LLM service.
`keepalive_pool`	integer	False	30	≥ 1	Keepalive pool size for connections to the LLM service.
`ssl_verify`	boolean	False	true		If true, verify the LLM service's SSL certificate.

How It Works

Examples

The examples below demonstrate how you can configure ai-request-rewrite for different scenarios.

The examples use OpenAI as the LLM service. To follow along, obtain an OpenAI API key and save it to an environment variable:

export OPENAI_API_KEY=<your-api-key>

:::note

You can fetch the admin_key from config.yaml and save to an environment variable with the following command:

admin_key=$(yq '.deployment.admin.admin_key[0].key' conf/config.yaml | sed 's/"//g')

:::

Redact Sensitive Information

The following example demonstrates how to use the ai-request-rewrite Plugin to redact sensitive information before the request reaches the Upstream service.

Create a Route and configure the ai-request-rewrite Plugin. The provider is set to openai, the OpenAI API key is passed in the Authorization header, and the prompt instructs the LLM to identify and mask sensitive information:

curl "http://127.0.0.1:9180/apisix/admin/routes/1" -X PUT \
  -H "X-API-KEY: ${admin_key}" \
  -d '{
    "uri": "/anything",
    "methods": ["POST"],
    "plugins": {
      "ai-request-rewrite": {
        "provider": "openai",
        "auth": {
          "header": {
            "Authorization": "Bearer '"$OPENAI_API_KEY"'"
          }
        },
        "options":{
          "model": "gpt-4"
        },
        "prompt": "Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver'\''s license numbers). Replace detected sensitive values with a masked format (e.g., \"*** **** **** 1234\") for credit card numbers. Ensure the JSON structure remains unchanged."
      }
    },
    "upstream": {
      "type": "roundrobin",
      "nodes": {
        "httpbin.org:80": 1
      }
    }
  }'

Create a Route with the ai-request-rewrite Plugin:

services:
  - name: ai-request-rewrite-service
    routes:
      - name: ai-request-rewrite-route
        uris:
          - /anything
        methods:
          - POST
        plugins:
          ai-request-rewrite:
            provider: openai
            auth:
              header:
                Authorization: "Bearer ${OPENAI_API_KEY}"
            options:
              model: gpt-4
            prompt: "Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., \"*** **** **** 1234\") for credit card numbers. Ensure the JSON structure remains unchanged."
    upstream:
      type: roundrobin
      nodes:
        - host: httpbin.org
          port: 80
          weight: 1

Synchronize the configuration to the gateway:

adc sync -f adc.yaml

apiVersion: v1
kind: Service
metadata:
  namespace: aic
  name: httpbin-external-domain
spec:
  type: ExternalName
  externalName: httpbin.org
---
apiVersion: apisix.apache.org/v1alpha1
kind: PluginConfig
metadata:
  namespace: aic
  name: ai-request-rewrite-plugin-config
spec:
  plugins:
    - name: ai-request-rewrite
      config:
        provider: openai
        auth:
          header:
            Authorization: "Bearer your-api-key"
        options:
          model: gpt-4
        prompt: "Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., \"*** **** **** 1234\") for credit card numbers. Ensure the JSON structure remains unchanged."
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  namespace: aic
  name: ai-request-rewrite-route
spec:
  parentRefs:
    - name: apisix
  rules:
    - matches:
        - path:
            type: Exact
            value: /anything
          method: POST
      filters:
        - type: ExtensionRef
          extensionRef:
            group: apisix.apache.org
            kind: PluginConfig
            name: ai-request-rewrite-plugin-config
      backendRefs:
        - name: httpbin-external-domain
          port: 80

apiVersion: apisix.apache.org/v2
kind: ApisixUpstream
metadata:
  namespace: aic
  name: httpbin-external-domain
spec:
  ingressClassName: apisix
  externalNodes:
  - type: Domain
    name: httpbin.org
---
apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
  namespace: aic
  name: ai-request-rewrite-route
spec:
  ingressClassName: apisix
  http:
    - name: ai-request-rewrite-route
      match:
        paths:
          - /anything
        methods:
          - POST
      upstreams:
      - name: httpbin-external-domain
      plugins:
        - name: ai-request-rewrite
          enable: true
          config:
            provider: openai
            auth:
              header:
                Authorization: "Bearer your-api-key"
            options:
              model: gpt-4
            prompt: "Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., \"*** **** **** 1234\") for credit card numbers. Ensure the JSON structure remains unchanged."

Apply the configuration to your cluster:

kubectl apply -f ai-request-rewrite-ic.yaml

Send a POST request to the Route with some personally identifiable information:

curl "http://127.0.0.1:9080/anything" -X POST \
  -H "Content-Type: application/json" \
  -d '{
    "content": "John said his debit card number is 4111 1111 1111 1111 and SIN is 123-45-6789."
  }'

You should receive a response similar to the following:

{
  "args": {},
  "data": "{\"content\": \"John said his debit card number is **** **** **** 1111 and SIN is ***-**-***.\"}",
  ...,
  "json": {
    "content": "John said his debit card number is **** **** **** 1111 and SIN is ***-**-***."
  },
  "method": "POST",
  "origin": "192.168.97.1, 103.97.2.170",
  "url": "http://127.0.0.1/anything"
}

Reformat Data

The following example demonstrates how to use the ai-request-rewrite Plugin to reformat data before the request reaches the Upstream service.

Create a Route and configure the ai-request-rewrite Plugin. The prompt instructs the LLM to convert natural language queries into structured JSON:

curl "http://127.0.0.1:9180/apisix/admin/routes/1" -X PUT \
  -H "X-API-KEY: ${admin_key}" \
  -d '{
    "uri": "/anything",
    "methods": ["POST"],
    "plugins": {
      "ai-request-rewrite": {
        "provider": "openai",
        "auth": {
          "header": {
            "Authorization": "Bearer '"$OPENAI_API_KEY"'"
          }
        },
        "options":{
          "model": "gpt-4"
        },
        "prompt": "Convert natural language queries into structured JSON format with intent and extracted parameters."
      }
    },
    "upstream": {
      "type": "roundrobin",
      "nodes": {
        "httpbin.org:80": 1
      }
    }
  }'

Create a Route with the ai-request-rewrite Plugin:

services:
  - name: ai-request-rewrite-service
    routes:
      - name: ai-request-rewrite-route
        uris:
          - /anything
        methods:
          - POST
        plugins:
          ai-request-rewrite:
            provider: openai
            auth:
              header:
                Authorization: "Bearer ${OPENAI_API_KEY}"
            options:
              model: gpt-4
            prompt: "Convert natural language queries into structured JSON format with intent and extracted parameters."
    upstream:
      type: roundrobin
      nodes:
        - host: httpbin.org
          port: 80
          weight: 1

Synchronize the configuration to the gateway:

adc sync -f adc.yaml

apiVersion: v1
kind: Service
metadata:
  namespace: aic
  name: httpbin-external-domain
spec:
  type: ExternalName
  externalName: httpbin.org
---
apiVersion: apisix.apache.org/v1alpha1
kind: PluginConfig
metadata:
  namespace: aic
  name: ai-request-rewrite-plugin-config
spec:
  plugins:
    - name: ai-request-rewrite
      config:
        provider: openai
        auth:
          header:
            Authorization: "Bearer your-api-key"
        options:
          model: gpt-4
        prompt: "Convert natural language queries into structured JSON format with intent and extracted parameters."
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  namespace: aic
  name: ai-request-rewrite-route
spec:
  parentRefs:
    - name: apisix
  rules:
    - matches:
        - path:
            type: Exact
            value: /anything
          method: POST
      filters:
        - type: ExtensionRef
          extensionRef:
            group: apisix.apache.org
            kind: PluginConfig
            name: ai-request-rewrite-plugin-config
      backendRefs:
        - name: httpbin-external-domain
          port: 80

apiVersion: apisix.apache.org/v2
kind: ApisixUpstream
metadata:
  namespace: aic
  name: httpbin-external-domain
spec:
  ingressClassName: apisix
  externalNodes:
  - type: Domain
    name: httpbin.org
---
apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
  namespace: aic
  name: ai-request-rewrite-route
spec:
  ingressClassName: apisix
  http:
    - name: ai-request-rewrite-route
      match:
        paths:
          - /anything
        methods:
          - POST
      upstreams:
      - name: httpbin-external-domain
      plugins:
        - name: ai-request-rewrite
          enable: true
          config:
            provider: openai
            auth:
              header:
                Authorization: "Bearer your-api-key"
            options:
              model: gpt-4
            prompt: "Convert natural language queries into structured JSON format with intent and extracted parameters."

Apply the configuration to your cluster:

kubectl apply -f ai-request-rewrite-ic.yaml

Send a POST request to the Route:

curl "http://127.0.0.1:9080/anything" -X POST \
  -H "Content-Type: application/json" \
  -d '{
    "content": "Book a flight from NYC to LA on April 10, 2022."
  }'

You should receive a response similar to the following:

{
  "args": {},
  "data": "{\n  \"intent\": \"BookFlight\",\n  \"parameters\": {\n    \"origin\": \"NYC\",\n    \"destination\": \"LA\",\n    \"date\": \"2022-04-10\"\n  }\n}",
  ...,
  "json": {
    "intent": "BookFlight",
    "parameters": {
      "date": "2022-04-10",
      "destination": "LA",
      "origin": "NYC"
    }
  },
  "method": "POST",
  "origin": "192.168.97.1, 103.97.2.167",
  "url": "http://127.0.0.1/anything"
}

Summarize Information

The following example demonstrates how to use the ai-request-rewrite Plugin to summarize information before the request reaches the Upstream service.

Create a Route and configure the ai-request-rewrite Plugin. The prompt instructs the LLM to summarize lengthy input while preserving key details:

curl "http://127.0.0.1:9180/apisix/admin/routes/1" -X PUT \
  -H "X-API-KEY: ${admin_key}" \
  -d '{
    "uri": "/anything",
    "methods": ["POST"],
    "plugins": {
      "ai-request-rewrite": {
        "provider": "openai",
        "auth": {
          "header": {
            "Authorization": "Bearer '"$OPENAI_API_KEY"'"
          }
        },
        "options":{
          "model": "gpt-4"
        },
        "prompt": "Summarize lengthy input while preserving key details. Ensure the summary remains concise and informative."
      }
    },
    "upstream": {
      "type": "roundrobin",
      "nodes": {
        "httpbin.org:80": 1
      }
    }
  }'

Create a Route with the ai-request-rewrite Plugin:

services:
  - name: ai-request-rewrite-service
    routes:
      - name: ai-request-rewrite-route
        uris:
          - /anything
        methods:
          - POST
        plugins:
          ai-request-rewrite:
            provider: openai
            auth:
              header:
                Authorization: "Bearer ${OPENAI_API_KEY}"
            options:
              model: gpt-4
            prompt: "Summarize lengthy input while preserving key details. Ensure the summary remains concise and informative."
    upstream:
      type: roundrobin
      nodes:
        - host: httpbin.org
          port: 80
          weight: 1

Synchronize the configuration to the gateway:

adc sync -f adc.yaml

apiVersion: v1
kind: Service
metadata:
  namespace: aic
  name: httpbin-external-domain
spec:
  type: ExternalName
  externalName: httpbin.org
---
apiVersion: apisix.apache.org/v1alpha1
kind: PluginConfig
metadata:
  namespace: aic
  name: ai-request-rewrite-plugin-config
spec:
  plugins:
    - name: ai-request-rewrite
      config:
        provider: openai
        auth:
          header:
            Authorization: "Bearer your-api-key"
        options:
          model: gpt-4
        prompt: "Summarize lengthy input while preserving key details. Ensure the summary remains concise and informative."
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  namespace: aic
  name: ai-request-rewrite-route
spec:
  parentRefs:
    - name: apisix
  rules:
    - matches:
        - path:
            type: Exact
            value: /anything
          method: POST
      filters:
        - type: ExtensionRef
          extensionRef:
            group: apisix.apache.org
            kind: PluginConfig
            name: ai-request-rewrite-plugin-config
      backendRefs:
        - name: httpbin-external-domain
          port: 80

apiVersion: apisix.apache.org/v2
kind: ApisixUpstream
metadata:
  namespace: aic
  name: httpbin-external-domain
spec:
  ingressClassName: apisix
  externalNodes:
  - type: Domain
    name: httpbin.org
---
apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
  namespace: aic
  name: ai-request-rewrite-route
spec:
  ingressClassName: apisix
  http:
    - name: ai-request-rewrite-route
      match:
        paths:
          - /anything
        methods:
          - POST
      upstreams:
      - name: httpbin-external-domain
      plugins:
        - name: ai-request-rewrite
          enable: true
          config:
            provider: openai
            auth:
              header:
                Authorization: "Bearer your-api-key"
            options:
              model: gpt-4
            prompt: "Summarize lengthy input while preserving key details. Ensure the summary remains concise and informative."

Apply the configuration to your cluster:

kubectl apply -f ai-request-rewrite-ic.yaml

Send a POST request to the Route with lengthy content:

curl "http://127.0.0.1:9080/anything" -X POST \
  -H "Content-Type: application/json" \
  -d '{
    "content": "Hey! So, I'\''m planning a trip to Japan next spring for about three weeks, and I want to visit Tokyo, Kyoto, and Osaka, but I'\''m not sure how to split my time between them. I really love history and cultural sites, so temples and shrines are a must. I'\''m also a big foodie, especially into ramen and sushi, so I'\''d love recommendations on the best spots. I prefer quieter areas for accommodation, but I don'\''t mind traveling into busy areas for sightseeing. Oh, and I'\''d also like to do a day trip somewhere outside these cities—maybe Hakone or Nara? I heard the cherry blossoms might still be in bloom in early April, so I'\''d love to catch them if possible. Also, what'\''s the best way to get around—should I get a JR Pass, or would individual tickets be better? Thanks!"
  }'

You should receive a response similar to the following:

{
  "args": {},
  "data": "The individual is planning a three-week trip to Japan in the spring, looking to visit Tokyo, Kyoto, and Osaka. They are interested in history, culture, temples, and shrines. They love ramen and sushi, so are seeking food recommendations. Accommodation should be in quieter areas, but they are open to busy sites for sightseeing. Along with these cities, they plan to make a day trip to either Hakone or Nara, hoping to see the cherry blossoms in early April. The best transport method between buying the JR Pass or individual tickets is also a query.",
  ...,
  "method": "POST",
  "origin": "192.168.97.1, 103.97.2.171",
  "url": "http://127.0.0.1/anything"
}

Send Request to an OpenAI-Compatible LLM

The following example demonstrates how to use the ai-request-rewrite Plugin with an OpenAI-compatible LLM provider by setting provider to openai-compatible and configuring the custom endpoint in override.endpoint.

Create a Route and configure the ai-request-rewrite Plugin:

curl "http://127.0.0.1:9180/apisix/admin/routes/1" -X PUT \
  -H "X-API-KEY: ${admin_key}" \
  -d '{
    "uri": "/anything",
    "plugins": {
      "ai-request-rewrite": {
        "prompt": "Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver'\''s license numbers). Replace detected sensitive values with a masked format (e.g., \"*** **** **** 1234\") for credit card numbers. Ensure the JSON structure remains unchanged.",
        "provider": "openai-compatible",
        "auth": {
          "header": {
            "Authorization": "Bearer <your-api-key>"
          }
        },
        "options": {
          "model": "qwen-plus",
          "max_tokens": 1024,
          "temperature": 1
        },
        "override": {
          "endpoint": "https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions"
        }
      }
    },
    "upstream": {
      "type": "roundrobin",
      "nodes": {
        "httpbin.org:80": 1
      }
    }
  }'

Create a Route with the ai-request-rewrite Plugin:

services:
  - name: ai-request-rewrite-service
    routes:
      - name: ai-request-rewrite-route
        uris:
          - /anything
        plugins:
          ai-request-rewrite:
            prompt: "Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., \"*** **** **** 1234\") for credit card numbers. Ensure the JSON structure remains unchanged."
            provider: openai-compatible
            auth:
              header:
                Authorization: "Bearer <your-api-key>"
            options:
              model: qwen-plus
              max_tokens: 1024
              temperature: 1
            override:
              endpoint: "https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions"
    upstream:
      type: roundrobin
      nodes:
        - host: httpbin.org
          port: 80
          weight: 1

Synchronize the configuration to the gateway:

adc sync -f adc.yaml

apiVersion: v1
kind: Service
metadata:
  namespace: aic
  name: httpbin-external-domain
spec:
  type: ExternalName
  externalName: httpbin.org
---
apiVersion: apisix.apache.org/v1alpha1
kind: PluginConfig
metadata:
  namespace: aic
  name: ai-request-rewrite-plugin-config
spec:
  plugins:
    - name: ai-request-rewrite
      config:
        prompt: "Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., \"*** **** **** 1234\") for credit card numbers. Ensure the JSON structure remains unchanged."
        provider: openai-compatible
        auth:
          header:
            Authorization: "Bearer your-api-key"
        options:
          model: qwen-plus
          max_tokens: 1024
          temperature: 1
        override:
          endpoint: "https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions"
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  namespace: aic
  name: ai-request-rewrite-route
spec:
  parentRefs:
    - name: apisix
  rules:
    - matches:
        - path:
            type: Exact
            value: /anything
      filters:
        - type: ExtensionRef
          extensionRef:
            group: apisix.apache.org
            kind: PluginConfig
            name: ai-request-rewrite-plugin-config
      backendRefs:
        - name: httpbin-external-domain
          port: 80

apiVersion: apisix.apache.org/v2
kind: ApisixUpstream
metadata:
  namespace: aic
  name: httpbin-external-domain
spec:
  ingressClassName: apisix
  externalNodes:
  - type: Domain
    name: httpbin.org
---
apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
  namespace: aic
  name: ai-request-rewrite-route
spec:
  ingressClassName: apisix
  http:
    - name: ai-request-rewrite-route
      match:
        paths:
          - /anything
      upstreams:
      - name: httpbin-external-domain
      plugins:
        - name: ai-request-rewrite
          enable: true
          config:
            prompt: "Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., \"*** **** **** 1234\") for credit card numbers. Ensure the JSON structure remains unchanged."
            provider: openai-compatible
            auth:
              header:
                Authorization: "Bearer your-api-key"
            options:
              model: qwen-plus
              max_tokens: 1024
              temperature: 1
            override:
              endpoint: "https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions"

Apply the configuration to your cluster:

kubectl apply -f ai-request-rewrite-ic.yaml

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Description

Plugin Attributes

How It Works

Examples

Redact Sensitive Information

Reformat Data

Summarize Information

Send Request to an OpenAI-Compatible LLM

FilesExpand file tree

ai-request-rewrite.md

Latest commit

History

ai-request-rewrite.md

File metadata and controls

Description

Plugin Attributes

How It Works

Examples

Redact Sensitive Information

Reformat Data

Summarize Information

Send Request to an OpenAI-Compatible LLM