DP to CP flow for AI workspace and AI Gateway

[thivindu]

Problem

The AI Gateway currently lacks the support for pushing artifacts created in the gateway to the AI Workspace: LLM Provider Templates, LLM providers, LLM Proxies and MCP proxies. Therefore the AI workspace is unaware of the artifacts directly created in the AI Gateway using the Gateway controller REST APIs.

Solution

Provide the ability to push the artifacts created in the Gateway to the AI workspace. The artifact created in the gateway will be the single source of truth and the artifacts will appear in the AI Workspace in a read only mode.

• Artifact creation - The artifact will be pushed to the AI workspace.
• Artifact updation - The changes to the artifact will be pushed to the AI workspace.
• Artifact deletion - The artifact is deleted from the AI gateway but will still be displayed in the AI workspace in an undeployed state.

---

Design

An endpoint already exists in the Platform API to support the DP to CP flow of REST APIs.

POST /api/internal/v1/apis/{apiId}/gateway-deployments

apiId - ID of the API that is created in the Gateway

Request body -

id:
    description: Unique identifier for the api artifact
configuration:
    $ref: "#APIConfiguration"
status:
    description: Status of the artifact (deployed, pending, failed)
createdAt:
    description: Timestamp when API was created
updatedAt:
    description: Timestamp when API was last updated
deployedAt:
    description: Timestamp when API was deployed to the gateway
deployedVersion:
    description: Version of the API that was deployed
projectIdentifier:
    description: Project identifier associated with the API

This endpoint will be generalized to support:

• LLM Provider Template
• LLM provider
• LLM Proxy
• MCP proxy
• API Types (REST, SOAP, GraphQL, .....)

Updated REST API :-

POST /api/internal/v1/artifacts/{artifactId}/gateway-deployments

artifactId - ID of the Artifact that is created in the Gateway

Request body -

id:
    description: Unique identifier for the api artifact

configuration:
    $ref: "#ArtifactConfiguration"
status:
    description: Status of the artifact (deployed, pending, failed, undeployed)
createdAt:
    description: Timestamp when artifact was created
updatedAt:
    description: Timestamp when artifact was last updated
deployedAt:
    description: Timestamp when artifact was deployed to the gateway
deployedVersion:
    description: Version of the artifact that was deployed
projectIdentifier:
    description: Project identifier associated with the artifact
origin:
    description: Where the artifact was created from (CP or DP)

Origin field to keeps track of where the artifact was created and triggers the readonly mode in the CP.

ArtifactConfiguration -

apiVersion:
    description: Artifact specification version
kind:
    description: Artifact type
metadata:
    description: Artifact metadata
spec:
    description: Artifact spec

Response -

201:
{
    message: Artifact deployment created successfully
}

The artifacts will be persisted in the relevant tables in the AI Workspace (Platform API) based on the kind in ArtifactConfiguration.

---

Discussion points

• Managing the CP artifact when the same artifact is deployed in multiple gateways
• Recreating the same artifact after deletion of the artifact from the gateway
• Creating LLM proxies from the CP from a gateway origin LLM provider

[malinthaprasan]

Shall we revisit the resource path (POST /api/internal/v1/artifacts/{artifactId}/gateway-deployments) to a alternative one - I find it a bit confusing compared to what I actually does, we are actually creating an API Artifact itself, not really a deployment. The deployment is already created in the gateway. We are only doing a CP side artifact creation and mapping it with what we already have in the gateway.

Shall we go with something like :

URL:

POST /api/internal/v1/artifacts/import-gateway-artifact

Request Body:

configuration:        # Gateway Artifact Yaml config 
  apiVersion: ...
  kind: ...           # LLMProvider | LLMProviderTemplate | LLMProxy | MCPProxy | REST | ...
  metadata:
    name: ...         # used for uniqueness / upsert
  spec: ...
<any other properties which is necessay>: ...  # optional. Even if we don't have any, lets wrap the "Gateway Artifact Yaml config" with "configuration" field so we can add any properties if necessary later in the future.

Response Body

200:
  id: ...              # Platform-API-side id (returned so the gateway can correlate if necessary)
  status: deployed | pending | failed | undeployed
  origin: DP
  createdAt: ...
  updatedAt: ...
  deployedAt: ...
  deployedVersion: ...

I think, in the original request payload:

• origin - not needed, this will always be "Gateway" as indicated in the request path.
• projectId - This should be part of the GatewayArtifiactConfiguration as an annotation. I think we already use this. If the project id annotation is not present, the request should be rejected.
• deployedVersion - Isn't this part of the configuration which has the version field?
• deployedAt - Unless we really need to keep track on the original time the api was deployed, we wouldn't need this.
• other properties such are createdAt, updatedAt etc - not needed as they should be only part of the response.

We will also need to consider how the Choreo/Bijira specific components should be created when doing the bottom up approach. Because, here the gateway only connect to Platform API and inform that the artifact is created, and now Platform API has to cordinate with the rest of the Control Plane components (including Choreo/Bijira) to create the other required artifacts.

[slahirucd7]

In Gateway with On-Prem APIM mode, bottom up API deployments directly calls the publisher portal's import API resource and creates the API in the control plane. With the below PR it will be changed to consider the internal REST API available for the platform API in the on-prem APIM.
https://github.com/wso2-support/carbon-apimgt/pull/8241

When considering above two descriptions I found below two approaches.

• Directly applying changes to the control plane with the AI gateway REST API calls (Ex: Create LLM provider, if GW is connected it will reflect it in the control plane. If not soon after connecting to the control plane it will reflect it in the control plane.)
• Create LLM providers, Proxies in the gateway and push artefacts later to the control plane. This is the scenario explained in the above section.

In both scenarios we need to consider below,

To create LLM providers,

• Organization should be there (this is valid since gw is connecting to the CP)
• Project Should be there (existing project vs new project creation).
• LLM provider creation (LLM provider creation conflicts will have to consider)

To create LLM proxies

• Organization should be there (this is valid since gw is connecting to the CP)
• Project Should be there (existing project vs new project creation).
• LLM provider should be there (GW having provider vs CP not having LLM providers)
• LLM proxy conflicts should also handled.

[thivindu]

The resource path of REST endpoint in the Platform API will be :

URL

POST /api/internal/v1/artifacts/import-gateway-artifact

Request body:

id:
    description: Unique identifier for the api artifact. This preserves the artifact uuid in both the gateway and platform API.
configuration:
    apiVersion: ...
    kind: ...           # LLMProvider | LLMProviderTemplate | LLMProxy | MCPProxy | REST | ...
    metadata:
       name: ...
       annotations: 
           project: ...
    spec: ...
status:
    description: Status of the artifact (deployed, pending, failed, undeployed)
properties:        # optional. To add any properties if necessary later in the future.

Response Body

200:
  id: ...
  status: deployed | pending | failed | undeployed
  origin: DP
  createdAt: ...
  updatedAt: ...
  deployedAt: ...
  deployedVersion: ...

Artifacts will be pushed to the CP from the Gateway upon reconnection and when a new artifact is created in the Gateway and the CP is connected.

The artifact that is pushed from the DP is marked as read only in the CP and only updates that will not affect the gateway runtime artifact is allowed, such as adding documentation.

Deleting an artifact from the gateway will remove the deployment, but the artifact will remain in the control plane as an undeployed entity, requiring a separate, manual deletion action in the control plane to remove it entirely. The Control Plane will not support redeploying an API that was deleted from the gateway. A DP initiated artifact cannot be deleted from the CP.

Distinguishing between top-down and bottom-up APIs will be determined by adding a database field origin so that such artifacts will not be recreated in the control plane when they are deployed in the gateway and pushed to the CP.

The information in the deployment artifact is split in to metadata and gateway specific information so that only one gateway will be configured to synchronize metadata with the control plane's working copy, while other gateways will synchronize only deployment-specific configuration information. This will be managed via a synchronization flag sync=true within the gateway configuration. If more than one gateway is configured to synchronize metadata, a last-in wins strategy is used to decide the control plane's working copy of the artifact.

The callback from the gateway is used to identify if the API was actually deployed in an gateway.

[thivindu]

Update -

Move the origin column from the artifact table to the specific llm_provider_templates, llm_providers, llm_proxies, mcp_proxies and other artifact tables in the Platform API.

Remove the syncMetadata field from the gateway registration. A last-in wins strategy is used to decide the control plane's working copy of the artifact. Send the deployment time from the gateway to the CP and decide the last deployment on that.

Implement a lock to solve concurrent updates in the CP to the same artifact.

Implement a retry mechanism for the artifact pushing.

Change the POST /api/internal/v1/artifacts/import-gateway-artifact to POST /api/internal/v1/artifacts/import-gateway-artifacts and support bulk imports.

Deploy and push artifacts to the CP when it is created from the immutable gateway.

Introduce a new configuration DeploymentSyncEnabled for the CP -> DP and DP -> CP artifact syncing and refactor the sync implementation.