feat(api-proxy): add middle-power model fallback with stale-cache recovery#3607
Conversation
✅ Coverage Check PassedOverall Coverage
📁 Per-file Coverage Changes (1 files)
Coverage comparison generated by |
Smoke Test ResultsGitHub MCP: ✅ Retrieved PR #3602: "api-proxy: fallback unavailable GitHub.com Connectivity: ❌ Pre-step data not provided (placeholder not substituted) File Write/Read: ❌ File Overall Status: FAIL (2/3 tests could not be verified) cc
|
Smoke Test Results✅ GitHub API: 2 PR entries verified Status: PASS
|
📡 OTel Tracing Validation ResultsAll scenarios passed ✅
Implementation Summary
✅ OTEL tracing integration is production-ready.
|
Smoke Test Results\n- GitHub MCP Testing: ❌ (mcpscripts not found)\n- GitHub.com Connectivity: ❌ (Status 000)\n- File Writing Testing: ✅\n- Bash Tool Testing: ✅\n\nOverall status: FAILWarning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "localhost"See Network Configuration for more information.
|
|
✅ Merged PRs: api-proxy: fallback unavailable Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "registry.npmjs.org"See Network Configuration for more information.
|
Chroot Runtime Version Test Results
Overall Result: ❌ Tests FAILED The chroot environment is not using the same runtime versions as the host. Python and Node.js versions differ between the host runner and the chroot environment, which may indicate the selective bind mounts are not properly exposing the host binaries.
|
🔥 Smoke Test: Services Connectivity - FAIL❌ Redis: connection timeout Result: FAIL - All service connectivity checks failed. Services may not be running or
|
There was a problem hiding this comment.
Pull request overview
Adds a configurable “middle-power” model fallback policy to the API proxy to avoid repeated upstream 4xx/retry loops when model alias resolution fails or cached availability is stale, and exposes this policy through config + /reflect.
Changes:
- Introduces
apiProxy.modelFallbackconfig (schema + mapping) and passes it to the api-proxy sidecar viaAWF_MODEL_FALLBACK. - Updates api-proxy model resolution to support a deterministic middle-tier fallback and performs an on-demand models refresh when resolution fails / fallback activates.
- Makes body transforms async-capable (composition + proxy request path) and adds reflect/test coverage for the new behavior.
Show a summary per file
| File | Description |
|---|---|
| src/types/api-proxy-options.ts | Adds modelFallback option type to wrapper config surface. |
| src/services/api-proxy-service.ts | Wires AWF_MODEL_FALLBACK env var into sidecar container env. |
| src/services/api-proxy-service-rate-limit.test.ts | Adds compose/env assertion test for AWF_MODEL_FALLBACK. |
| src/config-file.ts | Adds config typing + maps apiProxy.modelFallback into CLI options. |
| src/config-file.test.ts | Adds schema validation and mapping tests for modelFallback. |
| src/commands/build-config.ts | Plumbs options.modelFallback into constructed WrapperConfig. |
| src/awf-config-schema.json | Adds JSON schema for apiProxy.modelFallback. |
| docs/awf-config.schema.json | Mirrors schema change for published docs schema. |
| containers/api-proxy/server.network.test.js | Asserts /reflect includes model_fallback. |
| containers/api-proxy/server.models.test.js | Adds tests for stale-cache refresh, fallback logs, and async transform composition. |
| containers/api-proxy/server.js | Implements fallback config parsing, on-demand model refresh, and fallback observability logs. |
| containers/api-proxy/proxy-utils.js | Extends composeBodyTransforms to support async transforms. |
| containers/api-proxy/proxy-request.js | Awaits (possibly async) body transforms when proxying requests. |
| containers/api-proxy/model-resolver.test.js | Adds coverage for extended alias parsing + fallback selection behavior. |
| containers/api-proxy/model-resolver.js | Implements middle-power fallback selection + extended alias support. |
| containers/api-proxy/model-discovery.js | Adds provider-aware tier ranking and tier-sorted model helper. |
| containers/api-proxy/management.js | Exposes effective fallback config via reflect output. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comments suppressed due to low confidence (1)
containers/api-proxy/model-resolver.js:34
parseModelAliasesnow accepts extended alias entries ({ patterns: string[], fallback?: boolean }), but the JSDoc still claims it returnsRecord<string, string[]>, and the wrapper config surfaces still only allowapiProxy.modelsvalues to bestring[](seesrc/awf-config-schema.json/docs/awf-config.schema.jsonapiProxy.modelsschema andsrc/config-file.tstype). As a result, the new extended alias shape can’t be expressed via the AWF config file/TS types despite being advertised. Update the schema/types/docs to permit the extended shape (or adjust the implementation/docs to match what’s actually configurable).
/**
* Parse model aliases configuration from a raw JSON string.
*
* @param {string|null|undefined} rawConfig - JSON string from AWF_MODEL_ALIASES env var
* @returns {{ models: Record<string, string[]> } | null} Parsed config or null if invalid/absent
*/
- Files reviewed: 17/17 changed files
- Comments generated: 3
| const transformed = await bodyTransform(body); | ||
| if (transformed) body = transformed; |
|
|
||
| function parseModelFallbackConfig(rawConfig) { | ||
| if (!rawConfig) return { ...DEFAULT_MODEL_FALLBACK }; | ||
| try { | ||
| const parsed = JSON.parse(rawConfig); | ||
| if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) return { ...DEFAULT_MODEL_FALLBACK }; | ||
| const enabled = parsed.enabled === undefined ? true : Boolean(parsed.enabled); | ||
| const strategy = typeof parsed.strategy === 'string' && parsed.strategy.trim() | ||
| ? parsed.strategy.trim() | ||
| : DEFAULT_MODEL_FALLBACK.strategy; |
| return async (body) => { | ||
| let result = rewriteModelInBody(body, provider, MODEL_ALIASES.models, cachedModels, MODEL_FALLBACK); | ||
| if (!result || (result.fallback && result.fallback.activated)) { | ||
| await refreshProviderModelsForResolution(provider); | ||
| result = rewriteModelInBody(body, provider, MODEL_ALIASES.models, cachedModels, MODEL_FALLBACK); | ||
| } |
🏗️ Build Test Suite Results
Overall: 8/8 ecosystems passed — ✅ PASS All build and test operations completed successfully across all 8 ecosystems (Bun, C++, Deno, .NET, Go, Java, Node.js, Rust).
|
Smoke Test: Copilot BYOK Mode ✅PR #3607: feat(api-proxy): add middle-power model fallback with stale-cache recovery Results
Note: Running in BYOK offline mode ( Overall: PASS (core BYOK functionality verified)
|
Model resolution in
api-proxycould fail when aliases miss, requested models are unavailable, or cache is stale, causing upstream 4xx loops and repeated retries. This change adds a default safety-net fallback that deterministically selects an available “middle-power” model and makes fallback behavior configurable via AWF config.Model resolution fallback policy
opus > sonnet > haikugpt-5.x > gpt-4.x > gpt-3.5"alias": ["provider/pattern"]"alias": { "patterns": [...], "fallback": false }Stale cache / availability recovery
Structured fallback observability
model_fallback_activated(warn)model_fallback_candidates(debug)model_fallback_skipped(info)Config + reflection wiring
apiProxy.modelFallback.enabled(defaulttrue)apiProxy.modelFallback.strategy(middle_power)AWF_MODEL_FALLBACK./reflectasmodel_fallback.Illustrative behavior
{ "apiProxy": { "models": { "sonnet": { "patterns": ["copilot/*sonnet*"], "fallback": true } }, "modelFallback": { "enabled": true, "strategy": "middle_power" } } }When
sonnetcannot be resolved from current alias candidates, api-proxy now selects a provider-available median-tier model and emitsmodel_fallback_activatedwith candidate/tier context.