feat(frontend): Add AI Gateway integration for dynamic provider and model selection#2175
Merged
Conversation
…odel selection Add comprehensive AI Gateway integration to Console UI with the following features: - **Gateway Selection**: Fetch and display available AI Gateways from AI Gateway API - Auto-select first available gateway when creating agents - Show gateway ID in agent configuration details - **Dynamic Provider Loading**: Fetch LLM providers from AI Gateway ListModelProviders API - Replace hardcoded provider list with dynamic data - Display provider logos from API response - **Dynamic Model Loading**: Fetch models from AI Gateway ListModels API - Filter models by selected provider - Auto-select first model when provider changes - Show model metadata (context window, description) - **Dual-Mode Support**: - AI Gateway mode: Use gateway for auth, dynamic providers/models, hide API key field - Legacy mode: Use API key, hardcoded providers/models (backward compatible) - **Feature Flag**: enable-api-key-configuration-agent - false (default): AI Gateway mode with dynamic data - true: Legacy mode with hardcoded providers and API key requirement - **API Integration**: - Add buf.build/redpandadata/ai-gateway dependency - Generate TypeScript types for Gateway, ModelProviders, and Models services - Create React Query hooks with proper caching and authentication - Configure dev server proxy for AI Gateway API (/.redpanda/api/redpanda.api.aigateway.v1) - **Validation**: - API key optional when gateway is configured (proto updated with ignore = IGNORE_IF_ZERO_VALUE) - Gateway required when available - Conditional validation based on mode - **UX Improvements**: - No layout shift: gateway field always visible (disabled when unavailable) - Loading states for gateway/provider/model dropdowns - No validation errors during loading transitions - Icons displayed for providers in both selected state and dropdown Technical details: - Proto generation includes AI Gateway v1 API from buf.build - Proxy configuration in both Console UI and Cloud UI to handle CORS - Custom transport for AI Gateway with bearer token authentication - React Query caching with staleTime to prevent excessive refetching Co-Authored-By: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com>
alenkacz
force-pushed
the
av/ai-gateway-integration-ui
branch
from
87fce9c to
12894ca
Compare
Contributor
|
The latest Buf updates on your PR. Results from workflow Buf CI / validate (pull_request).
|
Comment on lines
+56
to
+80
| proxy: [ | ||
| // AI Gateway API - proxy to separate AI Gateway service | ||
| // Matches: /.redpanda/api/redpanda.api.aigateway.v1.* | ||
| // Proto package is: redpanda.api.aigateway.v1 (includes .api) | ||
| // AI Gateway now expects the full path with .api | ||
| ...(process.env.AI_GATEWAY_URL | ||
| ? [ | ||
| { | ||
| context: ['/.redpanda/api/redpanda.api.aigateway.v1'], | ||
| target: process.env.AI_GATEWAY_URL, | ||
| changeOrigin: true, | ||
| secure: false, | ||
| logLevel: 'debug', | ||
| // No pathRewrite - AI Gateway expects full path with .api | ||
| }, | ||
| ] | ||
| : []), | ||
| // All other APIs - proxy to Console backend | ||
| { | ||
| context: ['/api', '/redpanda.api', '/auth', '/logout'], | ||
| target: process.env.PROXY_TARGET || 'http://localhost:9090', | ||
| changeOrigin: !!process.env.PROXY_TARGET, | ||
| secure: process.env.PROXY_TARGET ? false : undefined, | ||
| }, | ||
| ], |
Contributor
There was a problem hiding this comment.
Shouldn't we be able to proxy both Console backend and the new Gateway API? I think currently if AI_GATEWAY_URL environment variable is set, then there is no proxy established for Console backend. Shouldn't we allow both at the same time?
Contributor
Author
There was a problem hiding this comment.
I think both works now 🤔 definitely works when I run it from local
| // Gateway detection and list query (using v1 API from ai-gateway module) | ||
| // Only fetch when NOT in legacy mode | ||
| const { data: gatewaysData, isLoading: isLoadingGateways } = useListGatewaysQuery( | ||
| { pageSize: 1000 }, // Get all gateways (max 1000) |
Contributor
There was a problem hiding this comment.
is it really 1000? usually it's either 100 or 500
| @@ -72,6 +74,38 @@ export const AIAgentCreatePage = () => { | |||
| skipInvalidation: true, | |||
| }); | |||
|
|
|||
| // Feature flag: when true, use legacy API key mode (hardcoded providers) | |||
| const useLegacyApiKeyMode = isFeatureFlagEnabled('enableApiKeyConfigurationAgent'); | |||
Contributor
There was a problem hiding this comment.
Let's refrain from using use prefix for non-hooks - let's call it isLegacyApiKeyMode or similar.
Comment on lines
+49
to
+50
| staleTime: 60000, // 1 minute - prevent excessive refetching | ||
| gcTime: 300000, // 5 minutes cache |
Contributor
There was a problem hiding this comment.
You don't need this. staleTime, gcTime etc. would be handled by the query client on a global level.
Contributor
|
CI is failing due to |
…d in initialValues - Add isLoadingGateways to LLMConfigSectionProps interface - Remove duplicate gatewayId property from initialValues object Fixes TypeScript compilation errors.
- Run Biome formatter to fix import ordering and code style - Change pageSize from 1000 to 100 for gateway queries - Rename useLegacyApiKeyMode to isLegacyApiKeyMode (non-hook variable) - Clarify proxy configuration supports both Console backend and AI Gateway - Update useEffect dependencies for exhaustive deps check
…gateway display Add comprehensive AI Gateway support across agent lifecycle: - Display gateway info on agent details page (read-only field) - Load providers and models from AI Gateway API on edit screen - Hide API token field when agent uses gateway (create and edit) - Strip API-specific prefixes from provider and model names - Auto-select first enabled provider and model in create mode - Configure console dev server on port 3004 for module federation Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Remove staleTime, gcTime, refetchOnWindowFocus, refetchOnMount, and refetchOnReconnect options from AI Gateway query hooks as these should be configured at the global query client level per code review feedback.
When enableApiKeyConfigurationAgent feature flag is enabled, the edit screen will now use legacy behavior (hardcoded providers/models, no AI Gateway API calls) even if the agent was originally created with a gateway. This ensures consistent behavior across create and edit screens when the feature flag is enabled.
- Add isLoadingGateways to gateway Select disabled state and placeholder - Remove unused detectProvider import - Add non-null assertions for aiAgent in useMemo and handleSave - Fix provider type checking with explicit null check - Convert selectedProvider to boolean with !! in hasNoModels Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
These auto-generated protobuf files were importing from the wrong package (go.panda.dev/redpanda-aigw instead of local console imports), causing golangci-lint to fail. These files belong to the redpanda-aigw repository, not the console repository. The frontend AI Gateway integration uses TypeScript protobuf files generated separately and doesn't need these backend Go files. Fixes golangci-lint error: no export data for "go.panda.dev/redpanda-aigw/protos/gen/redpanda/api/aigateway/v1" Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
| }, | ||
| }, | ||
| server: { | ||
| port: 3004, |
Contributor
There was a problem hiding this comment.
this wouldn't always be the case - you should either run bun run start for enterprise or bun run start2 --port=3004
Comment on lines
+91
to
+92
| filter: input?.filter ?? '', | ||
| orderBy: input?.orderBy ?? '', |
Contributor
There was a problem hiding this comment.
If there is nothing to filter by, we shouldn't need to provide it
Contributor
There was a problem hiding this comment.
Can we do something like
const listModelsFilter = create(ListModelsRequestSchema_Filter, { input?.filter ? ...input?.filter : undefined })
Comment on lines
+122
to
+135
| if (isUsingGateway && modelsData?.models) { | ||
| // Map gateway models to our format (already filtered for enabled at API level) | ||
| return modelsData.models.map((model) => { | ||
| // Strip "models/provider/" prefix from model name | ||
| // e.g., "models/openai/gpt-4o-mini" -> "gpt-4o-mini" | ||
| const modelName = model.name.split('/').pop() || model.name; | ||
|
|
||
| return { | ||
| value: modelName, | ||
| name: model.displayName || modelName, | ||
| description: model.description || '', | ||
| }; | ||
| }); | ||
| } |
Contributor
There was a problem hiding this comment.
[major] We could add transform option to our useQuery directly, that way we wouldn't need to handle the data parsing in the component layer. Can you please use the transform property from connect/tanstack query framework to fetch data and parse it into the expected format directly in the useListModelsQuery hook?
| // Map gateway providers to our format (already filtered for enabled at API level) | ||
| return providersData.modelProviders.map((provider) => { | ||
| // Strip "model_providers/" prefix from provider name | ||
| const providerName = provider.name.replace(/^model_providers\//, ''); |
Contributor
There was a problem hiding this comment.
We may want to create a const PROVIDER_REGEX = new RegExp(/^model_providers\//g); or similar.
| const { data: modelsData, isLoading: isLoadingModels } = useListModelsQuery( | ||
| { | ||
| filter: selectedProvider | ||
| ? `provider = "${selectedProvider}" AND enabled = "true"` |
Contributor
There was a problem hiding this comment.
Do we need any special query parsing for the filters?
malinskibeniamin
approved these changes
Jan 26, 2026
Addresses code review comments from @malinskibeniamin: 1. [MAJOR] Add select/transform to query hooks - Add transformModelProviders with PROVIDER_PREFIX_REGEX to strip "model_providers/" prefix - Add transformModels with MODEL_PREFIX_REGEX to strip "models/provider/" prefix - Move data parsing from component layer to query hooks using select option - Update return types to use ReturnType<typeof transform> for type safety 2. Make filter parameter optional - Use conditional spread: ...(input?.filter && { filter: input.filter }) - Don't provide filter/orderBy if not specified - Applies to both listModelProviders and listModels requests 3. Simplify component data mapping - Remove duplicate prefix stripping logic from llm-config-section.tsx - Use transformed data directly from query hooks - Cleaner separation of concerns: queries handle data transformation, components handle presentation Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit addresses all code review comments on PR #2175: 1. Revert rsbuild port configuration (frontend/rsbuild.config.ts:51) - Removed hardcoded port: 3004 - Use `bun run start` for enterprise or `bun run start2 --port=3004` for Module Federation 2. Split ai-gateway.tsx into separate files (frontend/src/react-query/api/ai-gateway.tsx:1) - Created ai-gateway/model-providers.tsx for useListModelProvidersQuery - Created ai-gateway/models.tsx for useListModelsQuery - Kept ai-gateway.tsx for useListGatewaysQuery only - Added documentation about AI Gateway transport requirement at top of each file 3. Remove pageSize override in component (frontend/src/components/pages/agents/create/ai-agent-create-page.tsx:84) - Removed pageSize: 100 from useListGatewaysQuery call - Now uses AI_GATEWAY_DEFAULT_PAGE_SIZE (50) from hook 4. Update imports in consuming components - Updated llm-config-section.tsx to import from separate files - Updated ai-agent-configuration-tab.tsx to import from separate files Note: The [nit] comment about nested ternary operators (line 326) is acknowledged and will be addressed in a subsequent PR as suggested. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
When switching providers (e.g., OpenAI to Anthropic), the model dropdown briefly showed a blank value while new models were loading. This happened because the old model value remained in the form field but didn't match any models in the empty filteredModels array during loading. Fix: Clear the model field when provider changes and models are loading or unavailable. This ensures the "Loading models..." placeholder is visible instead of a blank dropdown, providing better UX feedback during provider switching. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Biome formatting fixes: - Fix line breaks for long ternary expressions - Convert double quotes to single quotes for consistency - Format multi-line object property access UX improvements: - Clear model field when provider changes to show "Loading models..." placeholder - Use undefined instead of empty string for Select value to properly trigger placeholder - Prevents blank dropdown when switching providers during model loading Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
| const routeApi = getRouteApi('/agents/$id'); | ||
|
|
||
| import { CLOUD_MANAGED_TAG_KEYS, isCloudManagedTagKey } from 'components/constants'; | ||
| import { isFeatureFlagEnabled } from 'config'; |
Contributor
There was a problem hiding this comment.
[Biome] reported by reviewdog 🐶
Suggested change
| import { isFeatureFlagEnabled } from 'config'; |
|
|
||
| // Get gateway display name | ||
| const gatewayDisplayName = useMemo(() => { | ||
| if (!aiAgentData?.aiAgent?.gateway?.virtualGatewayId || !gatewaysData?.gateways) { |
Contributor
There was a problem hiding this comment.
[Biome] reported by reviewdog 🐶
Suggested change
| if (!aiAgentData?.aiAgent?.gateway?.virtualGatewayId || !gatewaysData?.gateways) { | |
| if (!(aiAgentData?.aiAgent?.gateway?.virtualGatewayId && gatewaysData?.gateways)) { |
| }, [aiAgentData, mcpServersData]); | ||
|
|
||
| if (!aiAgentData?.aiAgent) { | ||
| if (!aiAgentData?.aiAgent || !displayData) { |
Contributor
There was a problem hiding this comment.
[Biome] reported by reviewdog 🐶
Suggested change
| if (!aiAgentData?.aiAgent || !displayData) { | |
| if (!(aiAgentData?.aiAgent && displayData)) { |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds comprehensive AI Gateway integration to Console UI, enabling dynamic LLM provider and model selection from the AI Gateway API instead of hardcoded lists.
Key Features
Gateway Selection: Fetch and display available AI Gateways from AI Gateway API
Dynamic Provider Loading: Replace hardcoded provider list with API data
ListModelProvidersAPIDynamic Model Loading: Fetch models based on selected provider
provider = "{providerId}"queryDual-Mode Support:
gatewayIdFeature Flag:
enableApiKeyConfigurationAgentfalse(default): AI Gateway mode with dynamic datatrue: Legacy API key mode with hardcoded providersTechnical Implementation
API Integration:
buf.build/redpandadata/ai-gatewaydependency tobuf.yamluseListGatewaysQuery,useListModelProvidersQuery,useListModelsQueryProxy Configuration:
/.redpanda/api/redpanda.api.aigateway.v1→ AI Gateway serviceValidation Updates:
ignore = IGNORE_IF_ZERO_VALUE)UX Improvements:
Files Changed
New Files:
frontend/src/react-query/api/ai-gateway.tsx- API integration hooksfrontend/src/hooks/use-ai-gateway-transport.ts- Custom transport for AI Gatewayfrontend/src/protogen/redpanda/api/aigateway/v1/*- Generated TypeScript types (120+ files)backend/pkg/protogen/redpanda/api/aigateway/v1/*- Generated Go types (100+ files)Modified Files:
frontend/src/components/ui/ai-agent/llm-config-section.tsx- Gateway/provider/model UIfrontend/src/components/pages/agents/create/ai-agent-create-page.tsx- Gateway detection & API callsfrontend/src/components/pages/agents/create/schemas.ts- Validation updatesfrontend/src/components/pages/agents/details/ai-agent-configuration-tab.tsx- Edit screen dual-mode supportfrontend/src/components/constants.ts- Feature flag definitionfrontend/rsbuild.config.ts- Proxy configurationbuf.yaml- AI Gateway dependencytaskfiles/proto.yaml- Proto generation for AI GatewayTesting Checklist
Dependencies
Requires backend to be deployed with proto change from commit
817620329551a2fff2264b766ab4c88b2166c5d6whereapi_keyfield hasignore = IGNORE_IF_ZERO_VALUEto allow empty API key when gateway is configured.🤖 Generated with Claude Code