client routes: add exclusive-proxy config to control fallback behavior#881
Open
nikagra wants to merge 2 commits intoscylladb:scylla-4.xfrom
Open
client routes: add exclusive-proxy config to control fallback behavior#881nikagra wants to merge 2 commits intoscylladb:scylla-4.xfrom
nikagra wants to merge 2 commits intoscylladb:scylla-4.xfrom
Conversation
b870f1d to
90b8684
Compare
|
@nikagra , I would prefere to have an option to enable it and handle case when host switches from direct connection to client routes, you can make host connection pool either softly drop all the connections that used to directly, check if it won't blow the PR. |
1d9717b to
805ec7f
Compare
There was a problem hiding this comment.
Pull request overview
Adds an advanced.client-routes.exclusive-proxy configuration flag to control whether the driver may fall back to a node’s broadcast address when no entry exists in system.client_routes.
Changes:
- Introduces
CLIENT_ROUTES_EXCLUSIVE_PROXYas a new built-in driver option (typed + defaulted). - Updates client-routes endpoint resolution to either fall back (default) or throw when a route is missing (exclusive mode).
- Updates reference configuration documentation and expands tests to cover both exclusive/non-exclusive behaviors.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| core/src/main/java/com/datastax/oss/driver/internal/core/metadata/ClientRoutesTopologyMonitor.java | Reads the new flag and passes fallback endpoint as null in exclusive mode; adjusts endpoint construction behavior. |
| core/src/main/java/com/datastax/oss/driver/internal/core/metadata/ClientRoutesEndPoint.java | Adds exclusive-proxy behavior in resolve() (fallback vs WARN+throw) and updates constructor contract. |
| core/src/main/java/com/datastax/oss/driver/api/core/config/DefaultDriverOption.java | Adds CLIENT_ROUTES_EXCLUSIVE_PROXY option path. |
| core/src/main/java/com/datastax/oss/driver/api/core/config/TypedDriverOption.java | Adds typed Boolean option constant for exclusive-proxy. |
| core/src/main/java/com/datastax/oss/driver/api/core/config/OptionsMap.java | Registers default value (false) for the new typed option. |
| core/src/main/resources/reference.conf | Documents advanced.client-routes.exclusive-proxy with default false. |
| core/src/test/java/com/datastax/oss/driver/internal/core/metadata/ClientRoutesTopologyMonitorTest.java | Adds/updates tests for exclusive vs non-exclusive fallback behavior and host_id-null handling. |
| core/src/test/java/com/datastax/oss/driver/internal/core/metadata/ClientRoutesEndPointTest.java | Updates constructor calls and adds tests for exclusive mode throwing vs fallback mode. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…ssing When a node has client-routes configured but no matching entry is found, fall back to the broadcast address by default (backward-compatible). When advanced.client-routes.exclusive-proxy=true, throw instead of falling back so the node stays DOWN until a CLIENT_ROUTES_CHANGE event provides the route. The constructor of ClientRoutesEndPoint now enforces that fallbackEndPoint is non-null when exclusiveProxy=false, making the invariant explicit at construction time rather than silently failing at resolve() time. The WARN log on null host_id no longer mentions exclusive-proxy behaviour since that path fires regardless of the flag; the message now focuses on the actual cause (corrupted system tables).
…cement Introduces advanced.client-routes.exclusive-proxy (default: false). When false (default), the driver falls back to the node broadcast address when no client-route entry exists — preserving backward compatibility with mixed proxy/direct topologies. When true, no fallback is attempted: the node stays DOWN until a CLIENT_ROUTES_CHANGE event publishes the route. This prevents silent bypass of the proxy infrastructure during node additions.
805ec7f to
d341fef
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
advanced.client-routes.exclusive-proxy(boolean, defaultfalse) to control whether the driver falls back to a node's broadcast address when no route entry exists insystem.client_routesfalse(default): nodes without a route entry are reached directly via their broadcast address — backward-compatible, supports mixed proxy/direct topologiestrue: any node without a route is treated as unreachable;resolve()throwsIllegalStateExceptionwith a WARN log, the node stays DOWN, and the reconnection loop retries untilCLIENT_ROUTES_CHANGEpopulates the routeMotivation
Two requirements needed to coexist:
Mixed proxy/direct topologies (original design): some nodes are behind the private endpoint, others are not — the driver connects directly to unrouted nodes. The default
exclusive-proxy = falsepreserves this behavior.Strict proxy enforcement (new): when a new node is added there is a race window before its
system.client_routesentry is published. Withexclusive-proxy = truethe driver refuses to bypass the proxy during this window, matching the behavior of the Rust driver fix (failing address translation on missing entries). The node goes DOWN and reconnects once the route is available.Changes
DefaultDriverOption: newCLIENT_ROUTES_EXCLUSIVE_PROXYenum constantTypedDriverOption: new typedBooleanconstantreference.conf:advanced.client-routes.exclusive-proxy = falsewith documentationClientRoutesEndPoint: restoredfallbackEndPointfield;resolve()uses the fallback whenexclusive-proxy = false, throwsIllegalStateException+ WARN whenexclusive-proxy = trueClientRoutesTopologyMonitor: reads config flag in constructor; passes fallback endpoint (nullwhen exclusive-proxy is true) and flag toClientRoutesEndPointcreateHandlerWithExclusiveProxy()helper in the topology monitor testNot in this PR