[Improvement]: Support configuring backend connection timeout for LLM Providers

[menakaj]

Please select the area the issue is related to

Area/AIGateway (AI Gateway runtime/control plane)

Please select the aspect the issue is related to

Aspect/API (API backends, definitions, contracts, interfaces, OpenAPI)

Suggested Improvement

Summary

The LlmProvider kind does not support configuring a backend connection timeout. LLM backends typically have high and variable latency — without a configurable timeout, the gateway falls back to the global router default, which is
unsuitable for providers that require longer-running connections (e.g. large model inference, streaming responses).

Current Behaviour

The upstream field on LlmProvider accepts url, ref, and auth. There is no timeout field.

  spec:
    upstream:
      url: https://api.openai.com/v1
      auth:
        type: api-key
        header: Authorization
        value: Bearer sk-...
    # ↑ no way to set a connection timeout

The UpstreamTimeout schema and its supporting infrastructure (parsing, createCluster() wiring) already exist and are used by UpstreamDefinition for regular REST APIs — but this path is not reachable from LlmProvider.

Root Cause

The gap exists at three layers:

1. Schema — The Upstream schema (management-openapi.yaml) used by LlmProvider has no timeout field. UpstreamTimeout is only referenced from UpstreamDefinition (the reusable upstream pool used by REST APIs).
2. Transformer — transformProvider() in llm_transformer.go only maps Url from the upstream; there is no timeout to forward because the struct doesn't carry it.
3. xDS translator — resolveUpstreamCluster() in translator.go returns nil timeout for direct-URL upstreams (the url path). The timeout extraction only runs for ref-based upstreams that resolve to an UpstreamDefinition.

Suggested Approach

1. Add timeout to the Upstream schema (management-openapi.yaml)

  Upstream:
    properties:
      url: ...
      ref: ...
      hostRewrite: ...
      timeout:                                        # add
        $ref: "#/components/schemas/UpstreamTimeout"

This flows into both Upstream (RestAPI) and LLMProviderConfigData_Upstream (LLM) after codegen.

2. Forward timeout in the LLM transformer (llm_transformer.go)

  spec.Upstream.Main = api.Upstream{
      Url:     provider.Spec.Upstream.Url,
      Timeout: provider.Spec.Upstream.Timeout,   // add
  }

4. Return timeout from the direct-URL path in the xDS translator (translator.go)
   resolveUpstreamCluster() currently returns nil timeout when upstream.Url != nil. Add a helper mirroring resolveTimeoutFromDefinition and call it there:

  func resolveTimeoutFromUpstream(upstream *api.Upstream) *resolvedTimeout {
      if upstream.Timeout == nil {
          return nil
      }
      t := &resolvedTimeout{}
      if upstream.Timeout.Connect != nil {
          if d, err := parseTimeout(*upstream.Timeout.Connect); err == nil {
              t.Connect = &d
          }
      }
      return t
  }

No changes needed to createCluster() — it already accepts and applies connectTimeout.

Related Issues

LLM Service Providers: provide timeout override option