feat: base integration for circuit breaker#1163
Merged
Merged
Conversation
Contributor
|
DescriptionSummary By MatterAI
🔄 What ChangedThis pull request introduces the base integration for a circuit breaker mechanism within the
🔍 Impact of the ChangeThis change significantly enhances the reliability and resilience of the routing mechanism by preventing requests from being sent to unhealthy or overloaded targets. By integrating circuit breaker patterns, the system can gracefully degrade and recover, improving overall stability and user experience. The 📁 Total Files Changed
🧪 Test AddedNo specific tests were detailed in the PR description. However, for a feature of this nature, the following tests would be crucial:
🔒Security VulnerabilitiesNo direct security vulnerabilities were detected in the provided patch. The changes primarily focus on reliability and routing logic. However, ensuring that the MotivationThis PR aims to integrate a base circuit breaker mechanism to improve the resilience and reliability of the routing system by preventing requests from being sent to unhealthy targets. Type of Change
How Has This Been Tested?
Screenshots (if applicable)N/A Checklist
Related IssuesN/A Tip Quality Recommendations
Sequence DiagramsequenceDiagram
participant Client
participant GatewayAPI as API Endpoint
participant HandlerUtils as src/handlers/handlerUtils.ts
participant Context as c (Hono Context)
participant TargetConfig as currentTarget
participant InheritedConfig as currentInheritedConfig
Client->>GatewayAPI: HTTP Request (e.g., POST /v1/chat/completions)
GatewayAPI->>HandlerUtils: Call tryTargetsRecursively(c, target, request, requestHeaders, fn, method, currentJsonPath, inheritedConfig)
HandlerUtils->>InheritedConfig: Merge inheritedConfig with currentTarget.overrideParams
HandlerUtils->>InheritedConfig: Set currentInheritedConfig.id = inheritedConfig.id || currentTarget.id
alt isHandlingCircuitBreaker (currentInheritedConfig.id is present)
HandlerUtils->>TargetConfig: Filter currentTarget.targets to get healthyTargets (where !t.isOpen)
HandlerUtils->>TargetConfig: Update currentTarget.targets with healthyTargets
end
HandlerUtils->>HandlerUtils: Determine strategyMode (FALLBACK, WEIGHTED, CONDITIONAL, SINGLE)
alt StrategyModes.FALLBACK
loop for each target in currentTarget.targets
HandlerUtils->>HandlerUtils: Calculate originalIndex = target.originalIndex || index
HandlerUtils->>HandlerUtils: Recursively call tryTargetsRecursively(c, target, request, requestHeaders, fn, method, `${currentJsonPath}.targets[${originalIndex}]`, currentInheritedConfig)
HandlerUtils-->>Client: Return response if successful
end
else StrategyModes.WEIGHTED
HandlerUtils->>HandlerUtils: Select provider based on weight
HandlerUtils->>HandlerUtils: Calculate originalIndex = provider.originalIndex || index
HandlerUtils->>HandlerUtils: Recursively call tryTargetsRecursively(c, provider, request, requestHeaders, fn, method, `${currentJsonPath}.targets[${originalIndex}]`, currentInheritedConfig)
HandlerUtils-->>Client: Return response
else StrategyModes.CONDITIONAL
HandlerUtils->>HandlerUtils: Determine finalTarget based on conditions
HandlerUtils->>HandlerUtils: Calculate originalIndex = finalTarget.originalIndex || finalTarget.index
HandlerUtils->>HandlerUtils: Recursively call tryTargetsRecursively(c, finalTarget, request, requestHeaders, fn, method, `${currentJsonPath}.targets[${originalIndex}]`, currentInheritedConfig)
HandlerUtils-->>Client: Return response
else StrategyModes.SINGLE
HandlerUtils->>HandlerUtils: Calculate originalIndex = currentTarget.targets[0].originalIndex || 0
HandlerUtils->>HandlerUtils: Recursively call tryTargetsRecursively(c, currentTarget.targets[0], request, requestHeaders, fn, method, `${currentJsonPath}.targets[${originalIndex}]`, currentInheritedConfig)
HandlerUtils-->>Client: Return response
end
HandlerUtils->>HandlerUtils: Call tryPost(c, currentTarget, request, requestHeaders, fn, method)
alt tryPost successful
HandlerUtils-->>Client: Return response
alt isHandlingCircuitBreaker
HandlerUtils->>Context: Call c.get('handleCircuitBreakerResponse')(response, currentInheritedConfig.id, currentTarget.cbConfig, currentJsonPath, c)
end
else tryPost fails (error)
HandlerUtils-->>Client: Return error response
alt isHandlingCircuitBreaker
HandlerUtils->>Context: Call c.get('recordCircuitBreakerFailure')(env(c), currentInheritedConfig.id, currentTarget.cbConfig, currentJsonPath, response.status)
end
end
HandlerUtils->>HandlerUtils: constructConfigFromRequestHeaders(headers)
HandlerUtils->>HandlerUtils: Include 'cb_config' in extracted headers
HandlerUtils-->>Client: Final Response
|
![matterai-app[bot]](https://avatars.githubusercontent.com/in/474549?s=60&v=4){kind=small}
Contributor
|
Important PR Review SkippedPR review skipped as per the configuration setting. Run a manually review by commenting /matter review 💡Tips to use Matter AICommand List
|
Contributor
|
Important PR Review SkippedPR review skipped as per the configuration setting. Run a manually review by commenting /matter review 💡Tips to use Matter AICommand List
|
Contributor
|
Important PR Review SkippedPR review skipped as per the configuration setting. Run a manually review by commenting /matter review 💡Tips to use Matter AICommand List
|
Contributor
|
Important PR Review SkippedPR review skipped as per the configuration setting. Run a manually review by commenting /matter review 💡Tips to use Matter AICommand List
|
VisargD
reviewed
Jun 25, 2025
Contributor
|
Important PR Review SkippedPR review skipped as per the configuration setting. Run a manually review by commenting /matter review 💡Tips to use Matter AICommand List
|
VisargD
approved these changes
Jun 25, 2025
Contributor
|
Important PR Review SkippedPR review skipped as per the configuration setting. Run a manually review by commenting /matter review 💡Tips to use Matter AICommand List
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Motivation
Type of Change
How Has This Been Tested?
Screenshots (if applicable)
Checklist
Related Issues