Feature Summary

Add helm chart support for deploying LiteLLM proxy as part of the Texera Kubernetes deployment. This enables AI agent functionality by providing a unified LLM API gateway that can route requests to various LLM providers (Anthropic, OpenAI, etc.).

The LiteLLM proxy acts as a central point for:

• Managing API keys for multiple LLM providers
• Providing a unified OpenAI-compatible API for all models
• Tracking usage and spend across different models
• Enabling the access control service to interact with LLM models for agent features

Proposed Solution or Design

The implementation includes:

1. New Helm Templates:

   • litellm-config.yaml - ConfigMap for LiteLLM configuration
   • litellm-deployment.yaml - Deployment for the LiteLLM proxy container
   • litellm-secret.yaml - Secret for API keys (Anthropic, OpenAI) and database credentials
   • litellm-service.yaml - ClusterIP service for internal access
   • postgresql-litellm-persistence.yaml - PV/PVC for LiteLLM's PostgreSQL database

2. Chart Dependencies:

   • Add postgresql-litellm as an aliased postgresql dependency for LiteLLM's persistence

3. Service Integration:

   • Update access-control-service deployment to include LITELLM_MASTER_KEY and LITELLM_BASE_URL environment variables
   • Add ExternalName services for cross-namespace access
   • Add ingress paths for /api/models and /api/chat endpoints

4. Configuration:

   • Add litellm section in values.yaml with configurable image, resources, API keys, and persistence settings
   • Add postgresql-litellm section for the dedicated LiteLLM database

Impact / Priority

(P2)Medium – useful enhancement

Affected Area

• Deployment / Infrastructure