Problem description
@grpc/grpc-js-xds can crash when an xDS route uses weighted_clusters, omits the deprecated total_weight field, and the cluster weights do not add up to 100.
For example, a route containing one cluster with weight 1 fails with:
TypeError: Cannot read properties of undefined (reading 'ref')
at Object.invoke (.../node_modules/@grpc/grpc-js-xds/build/src/resolver-xds.js:343:36)
The same xDS route works with grpcurl/gRPC Go.
The resolver currently defaults an absent total_weight to 100:
routeAction = new WeightedClusterRouteAction(
weightedClusters,
route.route!.weighted_clusters!.total_weight?.value ?? 100,
// ...
);
WeightedClusterRouteAction therefore generates a random value in [0, 100). With one cluster of weight 1, only values in [0, 1) select that cluster. The remaining ~99% return an empty cluster name.
The resolver then looks up that empty name and dereferences undefined:
const clusterResult = action.getCluster();
const clusterRef = this.clusterRefs.get(clusterResult.name)!;
clusterRef.ref();
The generated Envoy API documentation says that total_weight is deprecated and clients should use the sum of all cluster weights.
Relevant code:
Reproduction steps
- Configure an RDS route similar to:
route:
weighted_clusters:
clusters:
- name: test-cluster
weight:
value: 1
# total_weight intentionally omitted
- Register
@grpc/grpc-js-xds and create an xDS client:
const grpc = require('@grpc/grpc-js');
require('@grpc/grpc-js-xds').register();
const client = new grpc.Client(
'xds:///test-service',
grpc.credentials.createInsecure()
);
-
Make repeated RPCs through the route.
-
Most calls fail with:
TypeError: Cannot read properties of undefined (reading 'ref')
Setting the cluster weight to 100, explicitly setting total_weight: 1, or using a non-weighted single-cluster route avoids the failure.
Expected behavior
When total_weight is omitted, the client should use the sum of the supplied cluster weights.
For one cluster with weight 1, that cluster should receive 100% of requests.
The resolver should also return a controlled error instead of throwing if cluster selection somehow produces an unknown or empty cluster name.
Environment
- OS: Linux x86_64, Kubernetes
- Node version: [fill in]
@grpc/grpc-js version: [fill in]
@grpc/grpc-js-xds version: [fill in]
- Installation method: npm
Suggested fix
Calculate the denominator from the cluster weights:
const totalWeight = weightedClusters.reduce(
(sum, cluster) => sum + cluster.weight,
0
);
Use that value when creating WeightedClusterRouteAction, rather than defaulting an omitted total_weight to 100.
It would also be helpful to guard the clusterRefs.get() result so an invalid selection produces a gRPC status error instead of an uncaught TypeError.
Additional context
A configuration with weights 1 and 99 works because the weights happen to total 100. The failure occurs when the supplied weights have a nonzero sum other than 100 and total_weight is omitted.
Problem description
@grpc/grpc-js-xdscan crash when an xDS route usesweighted_clusters, omits the deprecatedtotal_weightfield, and the cluster weights do not add up to 100.For example, a route containing one cluster with weight
1fails with:The same xDS route works with
grpcurl/gRPC Go.The resolver currently defaults an absent
total_weightto100:WeightedClusterRouteActiontherefore generates a random value in[0, 100). With one cluster of weight1, only values in[0, 1)select that cluster. The remaining ~99% return an empty cluster name.The resolver then looks up that empty name and dereferences
undefined:The generated Envoy API documentation says that
total_weightis deprecated and clients should use the sum of all cluster weights.Relevant code:
Reproduction steps
@grpc/grpc-js-xdsand create an xDS client:Make repeated RPCs through the route.
Most calls fail with:
Setting the cluster weight to
100, explicitly settingtotal_weight: 1, or using a non-weighted single-cluster route avoids the failure.Expected behavior
When
total_weightis omitted, the client should use the sum of the supplied cluster weights.For one cluster with weight
1, that cluster should receive 100% of requests.The resolver should also return a controlled error instead of throwing if cluster selection somehow produces an unknown or empty cluster name.
Environment
@grpc/grpc-jsversion: [fill in]@grpc/grpc-js-xdsversion: [fill in]Suggested fix
Calculate the denominator from the cluster weights:
Use that value when creating
WeightedClusterRouteAction, rather than defaulting an omittedtotal_weightto100.It would also be helpful to guard the
clusterRefs.get()result so an invalid selection produces a gRPC status error instead of an uncaughtTypeError.Additional context
A configuration with weights
1and99works because the weights happen to total 100. The failure occurs when the supplied weights have a nonzero sum other than 100 andtotal_weightis omitted.