Skip to content

Commit b86eeb4

Browse files
b1tamarahoffmaen
authored and
tas-runtime-bot
committed
Document hash-based routing
Co-authored-by: Clemens Hoffmann <clemens.hoffmann@sap.com>
1 parent 2ad225a commit b86eeb4

File tree

3 files changed

+177
-46
lines changed

3 files changed

+177
-46
lines changed

‎custom-per-route-options.html.md.erb‎

Lines changed: 111 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -8,24 +8,26 @@ By default, communication between Gorouter and backends is configured through th
88
This topic describes how to specify per-route Gorouter options scoped at the application level.
99
This greater granularity lets developers tailor optimal routing behavior for applications' unique load profiles or other requirements.
1010

11-
Gorouter supports the following per-route option, described in the section below:
11+
Gorouter supports the following per-route options, described in the sections below:
1212

1313
- `loadbalancing`: Configures the load balancing algorithm used by Gorouter for this particular route. <%= vars.per_route_lb_version %>
14-
- Settings: `round-robin`, `least-connection`.
14+
- Settings: `round-robin`, `least-connection`, `hash`.
15+
- `hash_header`: Defines the header Gorouter uses for routing decisions on this route. Required when `loadbalancing` is set to `hash`. Cannot be used with other load balancing algorithms. <%= vars.hash_routing_version %>
16+
- `hash_balance`: Sets the float number for the balance factor used by Gorouter to manage load imbalance applying the hash-based routing for this route. Optional when `loadbalancing` is `hash`. Cannot be used with other algorithms. <%= vars.hash_routing_version %>
1517

1618
## <a id="loadbalancing"></a> Configure Gorouter's Load Balancing Algorithm
1719

1820
<%= vars.per_route_lb_version %>
1921

2022
The per-route option `loadbalancing` allows configuring the load balancing algorithm, which defines how the load is distributed between Gorouters and backends.
2123

22-
This option supports two settings for load balancing:
24+
This option supports three settings for load balancing:
2325

2426
- `round-robin` distributes the load evenly across all available backends
2527
- `least-connection` directs traffic to the backend with the fewest active connections at any given time, optimizing resource utilization
28+
- `hash` distributes requests based on a hash of a specific HTTP header, ensuring requests with the same header are consistently directed to the same backend. <%= vars.hash_routing_version %>
2629

27-
28-
### <a id="lb-set-manifest"></a> Configure Load Balancing in an App Manifest
30+
### <a id="lb-set-manifest"></a> Configure Load Balancing using an App Manifest
2931

3032
To configure per-route load balancing for an application that has not yet been pushed:
3133

@@ -49,91 +51,155 @@ To configure per-route load balancing for an application that has not yet been p
4951
cf push -f manifest.yml
5052
```
5153

52-
### <a id="lb-update-route"></a> Change Load Balancing Algorithm of an Existing Route
54+
### <a id="lb-create-route"></a> Create a Route with a Specific Load Balancing Algorithm Using the CF CLI
55+
56+
To create a route with a per-route `loadbalancing` option, you can use the CLI command `create-route`.
57+
For example:
58+
59+
```console
60+
cf create-route EXAMPLE.COM --hostname MY-HOST --option loadbalancing=round-robin
61+
```
62+
63+
### <a id="lb-map-route"></a> Map a Route to an Existing App with a Specific Load Balancing Algorithm Using the CF CLI
64+
65+
To create and map a new route to an existing application with the per-route `loadbalancing` option, you can use the CLI command `map-route`.
66+
67+
For example:
68+
69+
```console
70+
cf map-route MY-APP EXAMPLE.COM --hostname MY-HOST --option loadbalancing=round-robin
71+
```
72+
73+
<p class="note">
74+
The command <code>map-route</code> supports the <code>--option</code> flag only for new routes.
75+
To update an existing route, use the command <code>update-route</code> described below.</p>
76+
77+
78+
### <a id="lb-update-route"></a> Update the Load Balancing Algorithm of an Existing Route Using the CF CLI
5379

54-
To change the per-route `loadbalancing` option of an existing route, you can use the cli command, `update-route`.
80+
To change the per-route `loadbalancing` option of an existing route, you can use the CLI command `update-route`.
5581

5682
For example, to change an app route's algorithm from `least-connection` to `round-robin`, you can run the `update-route` command:
5783

5884
```console
5985
cf update-route EXAMPLE.COM --hostname MY-HOST --option loadbalancing=round-robin
6086
```
87+
### <a id="lb-update-route"></a> Remove the Specific Load Balancing Algorithm Using the CF CLI
6188

62-
Alternatively, it is also possible to update the per-route load balancing option via the `/v3/routes` API.
63-
64-
Run the `PATCH` request to the targeted API endpoint:
89+
To remove the `loadbalancing` option from an existing route, run:
6590

6691
```console
67-
cf curl /v3/routes/GUID -X PATCH -H "Content-type: application/json" \
68-
-d '{
69-
"options": {
70-
"loadbalancing": "round-robin"
71-
}
72-
}'
92+
cf update-route EXAMPLE.COM --hostname MY-HOST -r loadbalancing
7393
```
74-
Where `GUID` is the unique identifier for the route.
7594

76-
### <a id="lb-create-route"></a> Create a Route with a specific Load Balancing Algorithm
95+
## <a id="hash-based-routing"></a> Configure Hash-Based Routing
7796

78-
To create a route with a per-route `loadbalancing` option, you can use the cli command `create-route`.
79-
For example:
97+
<%= vars.hash_routing_version %>
98+
99+
Hash-Based Routing is a load-balancing method that routes incoming requests to application instances using a hash of a specific HTTP header value. This ensures consistent routing, so requests with the same header value always go to the same instance. The per-route hash options offer detailed control over hash-based load balancing for individual routes.
100+
101+
### <a id="options-hash-set-manifest"></a> Configure Hash-Based Routing with an App Manifest
102+
1. In the application manifest, include a `route` definition with the following `options` attributes:
103+
- `loadbalancing` set to `hash`
104+
- `hash_header` set to the HTTP header name used for routing decisions
105+
- optionally, `hash_balance` set to a float number for the balance factor used by Gorouter to [manage load imbalance](hash-based-routing.html#handling-imbalance-loads) for this particular route.
106+
107+
```yaml
108+
---
109+
applications:
110+
- name: MY-APP
111+
routes:
112+
- route: MY-HOST.EXAMPLE.COM
113+
options:
114+
loadbalancing: hash
115+
hash_header: HASH-HEADER-NAME
116+
hash_balance: 1.2
117+
```
118+
119+
Where `MY-APP` is the name of your app, `MY-HOST.EXAMPLE.COM` is the route you want to map to your app and `HASH-HEADER-NAME` is the HTTP header name.
120+
121+
1. Push the app with the manifest:
122+
123+
```console
124+
cf push -f manifest.yml
125+
```
126+
127+
### <a id="options-hash-create-route"></a> Create a Route with Hash-Based Options Using the CF CLI
128+
129+
To create a route with hash-specific options, you can use the CLI command `create-route`. For example:
80130

81131
```console
82-
cf create-route EXAMPLE.COM --hostname MY-HOST --option loadbalancing=round-robin
132+
cf create-route EXAMPLE.COM --hostname MY-HOST --option loadbalancing=hash --option hash_header=HASH-HEADER-NAME --option hash_balance=1.2
83133
```
84134

85-
Alternatively, it is also possible to create a route with a per-route load balancing option via the `/v3/routes` API:
135+
Alternatively, you can use the shorthand `-o` for `--option`. Since `hash_balance` is optional, you can omit it:
86136

87137
```console
88-
cf curl /v3/routes -X POST -H "Content-type: application/json" \
89-
-d '{
90-
"host": "MY-HOST",
91-
"path": "MY-PATH",
92-
...
93-
"options": {
94-
"loadbalancing": "round-robin"
95-
}
96-
}'
138+
cf create-route EXAMPLE.COM --hostname MY-HOST -o loadbalancing=hash -o hash_header=HASH-HEADER-NAME
97139
```
98140

99-
### <a id="lb-map-route"></a> Map a Route to an Existing App with specific Load Balancing Algorithm
141+
### <a id="options-hash-map-route"></a> Map a Route with Hash Options to an Existing App Using the CF CLI
100142

101-
To create and map a new route to an existing application with the per-route `loadbalancing` option, you can use the cli command `map-route`.
143+
To create a new route suitable for hash-based routing and map it to an existing application, you can use the CLI command `map-route`.
102144

103145
For example:
104146

105147
```console
106-
cf map-route MY-APP EXAMPLE.COM --hostname MY-HOST --option loadbalancing=round-robin
148+
cf map-route MY-APP EXAMPLE.COM --hostname MY-HOST -o loadbalancing=hash -o hash_header=HASH-HEADER-NAME -o hash_balance=1.2
107149
```
108150

109151
<p class="note">
110152
The command <code>map-route</code> supports the <code>--option</code> flag only for new routes.
111-
To update an existing route, the command <code>update-route</code> must be used as described before.</p>
153+
To update an existing route, see the instructions for <code>update-route</code> below.
112154

113-
### <a id="lb-retrieve-route-options"></a> Retrieve Route Options
155+
### <a id="options-hash-update-route"></a> Update an Existing Route with Hash Options Using the CF CLI
114156

115-
To read route options, you can query the route using the `route` command:
157+
You can change an existing route that uses the default load balancing algorithm to the hash load balancing algorithm.
158+
159+
For example, to change an app route's algorithm from default `round-robin` to `hash` and set `hash_header` to HASH-HEADER-NAME without a balance factor, you can run the `update-route` command:
116160

117161
```console
118-
cf route EXAMPLE.COM --hostname MY-HOST
162+
cf update-route EXAMPLE.COM --hostname MY-HOST -o loadbalancing=hash -o hash_header=HASH-HEADER-NAME
119163
```
120164

121-
The response lists the chosen `loadbalancing` algorithm option, e.g. `least-connection`:
165+
To add a balance factor along with the previous settings, you can later run the `update-route` command, for example, with the `hash_balance` option set to `1.5`:
122166

123167
```console
124-
options: {loadbalancing=least-connection}
168+
cf update-route EXAMPLE.COM --hostname MY-HOST -o hash_balance=1.5
125169
```
126170

127-
Alternatively, you can query the `routes` API endpoint for a route:
171+
To unset the balance factor, run the `update-route` command with `hash_balance` set to 0.
128172

129173
```console
130-
cf curl /v3/routes/?hosts=MY-HOST
174+
cf update-route EXAMPLE.COM --hostname MY-HOST -o hash_balance=0
131175
```
132176

133-
Where `MY-HOST` is the host attribute of the route. The response lists the chosen `loadbalancing` algorithm option as well:
177+
Setting the balance factor to 0 indicates to Gorouter that load imbalance is accepted and all requests for a particular hash should be routed to the same instance as long as it's healthy, without redirecting to other predetermined instances.
178+
179+
### <a id="options-hash-revert"></a> Revert Hash Options Using the CF CLI
180+
181+
Running the `update-route` command with the `-r` flag for the option `loadbalancing` removes all hash options from the route, returning to the default load balancing algorithm:
182+
183+
```console
184+
cf update-route EXAMPLE.COM --hostname MY-HOST -r loadbalancing
185+
```
186+
187+
## <a id="lb-retrieve-route-options"></a> Retrieve Route Options
188+
189+
To view route options, you can query the route using the `route` command:
134190

135191
```console
136-
"options": {"loadbalancing": "least-connection"}
192+
cf route EXAMPLE.COM --hostname MY-HOST
137193
```
138194

139-
To retrieve all the routes with the corresponding options in a space of an organization, you can use the `routes` command.
195+
The response lists the chosen `loadbalancing` algorithm option, e.g. `least-connection`:
196+
197+
```console
198+
options: {loadbalancing=least-connection}
199+
```
200+
201+
Or `hash` with its related options:
202+
203+
```console
204+
options: {hash_balance=1.2, hash_header=HASH-HEADER-NAME, loadbalancing=hash}
205+
```

‎deploy-apps/manifest-attributes.html.md.erb‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -583,7 +583,9 @@ Under each route, you can optionally include an `options` attribute to configure
583583

584584
Available options are:
585585

586-
- `loadbalancing` - defines how Gorouter distributes requests across the application backends. Valid values are `round-robin` and `least-connection`.
586+
- `loadbalancing` - defines how Gorouter distributes requests across the application backends. Valid values are `round-robin`, `least-connection` and `hash`.
587+
- `hash_header` - defines the header used for hash-based routing decisions. Required when `loadbalancing` is set to `hash`.
588+
- `hash_balance` - defines the balance factor used to manage load imbalance for hash-based routing. Optional when `loadbalancing` is set to `hash`. Values in the 1.1-2.0 range provide the best balance of even distribution and performance. Omitting `hash_balance` or setting it explicitly to 0 indicates that the load situation will not be considered.
587589

588590
For example:
589591

@@ -597,6 +599,11 @@ For example:
597599
- route: example2.com
598600
options:
599601
loadbalancing: least-connection
602+
- route: example3.com
603+
options:
604+
loadbalancing: hash
605+
hash_header: Hash-Relevant-Header
606+
hash_balance: 1.25
600607
```
601608

602609
#### <a id='manifest-attibutes-2'></a> Manifest attributes

‎hash-based-routing.html.md.erb‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
---
2+
title: Hash-Based Routing
3+
owner: CF for VMs Networking
4+
---
5+
6+
## <a id="purpose"></a> Purpose
7+
8+
Hash-Based Routing is a load-balancing algorithm that distributes incoming requests to application instances based on a hash of a specific HTTP header value, e.g., `X-Resource-ID` or `Tenant-ID`. This ensures consistent routing behavior where requests containing the same header value are always directed to the same instance.
9+
10+
## <a id="prerequisites"></a> Prerequisites
11+
12+
To use Hash-Based Routing, ensure that
13+
14+
- your <%= vars.app_runtime_abbr %> deployment is running version <%= vars.hash_routing_version %> or later.
15+
- platform operators activate the feature so that the CF feature flag `hash_based_routing` is set to `true`. See [Feature Flags](https://docs.cloudfoundry.org/adminguide/listing-feature-flags.html#flags) for more details.
16+
17+
## <a id="key-features"></a> Key Features
18+
19+
- **Configurable via Per-Route Options**: Hash-Based load-balancing setup through application route options
20+
- **Configurable Hash Header**: The HTTP header to consider for Hash-Based Routing is configurable for each route
21+
- **Session Affinity Precedence**: Session affinity (sticky sessions) is prioritized over Hash-Based Routing
22+
- **No availability zones preference**: The global properties `locallyOptimistic` and `localAvailabilityZone` are ignored when using Hash-Based Routing
23+
- **Consistent Hashing**: Implementation of the Maglev Algorithm (see [Maglev: A Fast and Reliable Software Network Load Balancer](https://storage.googleapis.com/gweb-research2023-media/pubtools/2904.pdf) for details)
24+
- **Minimal Rehashing**: Usage of the Maglev Algorithm to map application instances by hash ensures that hash positions are shifted to other instances as little as possible when application instances are added or removed
25+
- **Handling imbalanced loads**: Detection and mitigation of imbalanced load on single instances prevents overloading while keeping instances for a particular hash at a minimum
26+
27+
Hash-Based Routing implements a clear precedence hierarchy:
28+
29+
1. **Sticky Sessions**: First checks if a sticky session is used and the sticky session endpoint is available
30+
2. **Hash-Based Routing**: Calculates the hash of the specified HTTP header value and routes the requests to a pre-determined application instance
31+
3. **Default Load Balancing**: Falls back to default load balancing if the header configured in the application route for Hash-Based Routing is absent in the request
32+
33+
34+
## <a id="hash-based-vs-session-affinity"></a> Hash-Based Routing vs. Session Affinity
35+
36+
While Session Affinity provides a way to keep session consistency, it works at the session level and can cause scalability issues. Heavy users might be routed to the same instance and stay pinned to it through session affinity, which could lead to overload. Session Affinity usually depends on session cookies (see more details in the [Session Affinity](https://docs.cloudfoundry.org/concepts/http-routing.html#-session-affinity) documentation), which requires implementation effort for web applications.
37+
38+
In contrast, Hash-Based Routing offers a more scalable and flexible approach by consistently distributing requests based on a hash of any configurable HTTP header value, not just session identifiers. This allows for load distribution based on tenant IDs, resource IDs, or other business-relevant identifiers. Configurable load imbalance handling enables spillover to other instances (see [Handling imbalanced loads](#handling-imbalance-loads)).
39+
40+
Unlike Session Affinity, Hash-Based Routing requires no code changes to the application.
41+
42+
## <a id="handling-imbalance-loads"></a> Handling imbalanced loads
43+
44+
Hash-Based Routing includes mechanisms to detect imbalanced load across application instances. An imbalanced load occurs when certain hashes receive more traffic, such as when a specific tenant generates many requests, resulting in heavier use of their mapped instances than others’. Additionally, multiple high-traffic targets might be assigned to the same instance.
45+
46+
To prevent overloading specific instances while others remain underutilized, the acceptable threshold for load imbalance can be configured using the `hash_balance` property.
47+
This setting determines whether an instance is handling more traffic than its fair share based on the average load across all instances, measured by the number of in-flight requests. For example, with a balance factor of 1.25, no single instance should handle more than 125% of the average number of in-flight requests across all instances managed by the current router. When this threshold is exceeded, the router redirects subsequent requests to other, less-loaded instances.
48+
Values of the balance factor in the 1.1-2.0 range offer a good balance between even distribution and performance. However, optimal values depend on the application's traffic patterns and load characteristics.
49+
50+
This approach ensures that a minimum number of instances process requests for a particular hash while preventing any single instance from becoming overloaded.
51+
52+
## <a id="minimal-rehashing"></a> Minimal Rehashing
53+
54+
The Maglev algorithm used in Hash-Based Routing minimizes rehashing when application instances are added or removed. When a new instance is added, only a small subset of hashes is remapped to it, while most continue to route to their original instances. Similarly, when an instance is removed, only the hashes mapped to that instance are reassigned. This design minimizes disruption and maintains consistent routing behavior as the application scales up or down.
55+
56+
## <a id="retries-in-hash-based"></a> Retries in Hash-Based Routing
57+
58+
For idempotent requests, Hash-Based Routing supports a retry mechanism. If a request fails due to a network error or a 5xx response from the application instance, the router retries the request with a different, predetermined application instance. The next entry in the Maglev lookup table determines this instance. This approach aligns with the approach to handling imbalances. It ensures that the retry mechanism adhere to the principles of Hash-Based Routing while providing resilience against transient failures.

0 commit comments

Comments
 (0)