Skip to content

Commit 2395a45

Browse files
dzane17kolchfa-aws
dzane17
and
kolchfa-aws
authored
Workload management group settings documentation (#12444)
* Workload management group settings documentation Signed-off-by: David Zane <davizane@amazon.com> * Tech review comment fixes Signed-off-by: David Zane <davizane@amazon.com> * Doc review Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> --------- Signed-off-by: David Zane <davizane@amazon.com> Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> Co-authored-by: Fanit Kolchina <kolchfa@amazon.com>
1 parent 6133ba8 commit 2395a45

6 files changed

Lines changed: 205 additions & 2 deletions

File tree

‎.github/vale/styles/OpenSearch/SubstitutionsError.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,8 @@ swap:
99
'allowlist': allow list
1010
'autoscaling': auto scaling
1111
'command-line interface': command line interface
12+
'coordinator node': coordinating node
13+
'coordinator nodes': coordinating nodes
1214
'data are': data is
1315
'data set': dataset
1416
'for information on': for information about

‎.github/vale/styles/OpenSearch/SubstitutionsSuggestion.yml‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,17 +8,35 @@ swap:
88
'app server': application server
99
'as well as': and
1010
'bring up': start or launch
11+
'brings up': starts or launches
12+
'bringing up': starting or launching
1113
'build out': expand or implement
14+
'builds out': expands or implements
15+
'building out': expanding or implementing
1216
'catch up': synchronize with
17+
'catches up': synchronizes with
18+
'catching up': synchronizing with
1319
'clean up': remove or normalize
1420
'cleans up': removes or normalizes
1521
'cleaning up': removing or normalizing
1622
'deal with': process or resolve
23+
'deals with': processes or resolves
24+
'dealing with': processing or resolving
1725
'due to': because of
1826
'fall back': revert
27+
'fall back to': default to
28+
'falls back': reverts
29+
'falls back to': defaults to
30+
'falling back': reverting
1931
'figure out': determine or identify
32+
'figures out': determines or identifies
33+
'figuring out': determining or identifying
2034
'fire up': start or launch
35+
'fires up': starts or launches
36+
'firing up': starting or launching
2137
'flesh out': elaborate or define
38+
'fleshes out': elaborates or defines
39+
'fleshing out': elaborating or defining
2240
'from scratch': manually, from the beginning, or without a template
2341
'get': receive, obtain, or retrieve
2442
'gets': receives, returns, or retrieves
@@ -28,47 +46,100 @@ swap:
2846
'handling': managing, processing, or resolving
2947
'it is recommended': we recommend
3048
'kick off': start or initiate
49+
'kicks off': starts or initiates
50+
'kicking off': starting or initiating
51+
'know about': detect or recognize
52+
'knows about': detects or recognizes
3153
'leverage': use
3254
'life cycle': lifecycle
55+
'live in': are located in
3356
'lives': is located
57+
'lives in': is located in
3458
'look into': investigate
59+
'looks into': investigates
60+
'looking into': investigating
3561
'looks like this': appears as follows
3662
'navigate in': navigate to
3763
'one-off': single-use or ad-hoc
3864
'out of the box': by default, natively, or is built-in
3965
'out-of-the-box': default, native, or built-in
66+
'pick up': load or detect
4067
'picks up': loads or detects
68+
'picking up': loading or detecting
4169
'plug in': integrate or use
70+
'plugs in': integrates or uses
71+
'plugging in': integrating or using
4272
'plug into': integrate with
73+
'plugs into': integrates with
4374
'plumbing': configuration or infrastructure
4475
'point to': reference
76+
'points to': references
77+
'pointing to': referencing
4578
'pull in': import or retrieve
79+
'pulls in': imports or retrieves
80+
'pulling in': importing or retrieving
4681
'push back': postpone
82+
'pushes back': postpones
83+
'pushing back': postponing
4784
'read in': ingest or parse
85+
'reads in': ingests or parses
86+
'reading in': ingesting or parsing
4887
'roll out': deploy or release
88+
'rolls out': deploys or releases
89+
'rolling out': deploying or releasing
4990
'scale up': scale or increase capacity
91+
'scales up': scales or increases capacity
92+
'scaling up': scaling or increasing capacity
5093
'see': view
94+
'sees': detects or finds
5195
'set up': configure or initialize
5296
'sets up': configures or initializes
5397
'setting up': configuring or initializing
5498
'show up': appear
99+
'shows up': appears
100+
'showing up': appearing
55101
'shut down': stop or terminate
102+
'shuts down': stops or terminates
103+
'shutting down': stopping or terminating
56104
'slow down': delay, reduce throughput, or increase latency
57105
'slows down': delays or reduces throughput
106+
'slowing down': delaying or reducing throughput
58107
'speed up': accelerate
108+
'speeds up': accelerates
109+
'speeding up': accelerating
59110
'spin up': provision or start
60111
'spins up': provisions or starts
112+
'spinning up': provisioning or starting
61113
'split up': divide or partition
114+
'splits up': divides or partitions
115+
'splitting up': dividing or partitioning
62116
'start here': getting started
63117
'stitching together': chaining or combining
118+
'talk to': communicate with or connect to
119+
'talks to': communicates with or connects to
120+
'talking to': communicating with or connecting to
121+
'touching': modifying or changing
64122
'tear down': remove or decommission
123+
'tears down': removes or decommissions
124+
'tearing down': removing or decommissioning
65125
'teardown': removal or decommission
126+
'tell': instruct or configure
127+
'tells': instructs or configures
128+
'telling': instructing or configuring
129+
'think': evaluate or determine
130+
'thinks': evaluates or determines
131+
'thinking': evaluating or determining
66132
'under the hood': internally
67133
'walks you through': describes or explains
134+
'want': require or expect
135+
'wants': requires or expects
68136
'wire up': connect or configure
69137
'wires up': connects or configures
138+
'wiring up': connecting or configuring
70139
'wiring': configuration or networking
71140
'wish|desire': want
72141
'work around': mitigate or bypass
142+
'works around': mitigates or bypasses
143+
'working around': mitigating or bypassing
73144
'workaround': mitigation or alternative approach
74145
'works with': supports, integrates with, or is compatible with

‎_tuning-your-cluster/availability-and-recovery/workload-management/wlm-feature-overview.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ Then restart your cluster. For more information, see [Installing plugins]({{site
2929

3030
## Workload groups
3131

32-
A _workload group_ is a logical grouping of tasks with defined resource limits. System administrators can dynamically manage workload groups using the Workload Management APIs. These workload groups can be used to create search requests with resource limits. For more information, see [Workload groups]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/workload-management/workload-groups/).
32+
A _workload group_ is a logical grouping of tasks with defined resource limits. System administrators can dynamically manage workload groups using the Workload Management APIs. These workload groups can be used to create search requests with resource limits. You can also define group-specific settings that are applied automatically to every request routed to the group. For more information, see [Workload groups]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/workload-management/workload-groups/) and [Workload group settings]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/workload-management/workload-group-settings/).
3333

3434
The following example request adds a workload group named `analytics`:
3535

‎_tuning-your-cluster/availability-and-recovery/workload-management/workload-group-rules.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
layout: default
33
title: Workload group rules
4-
nav_order: 30
4+
nav_order: 25
55
parent: Workload management
66
grand_parent: Availability and recovery
77
redirect_from:

‎_tuning-your-cluster/availability-and-recovery/workload-management/workload-group-settings.md‎

Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,124 @@
1+
---
2+
layout: default
3+
title: Workload group settings
4+
nav_order: 30
5+
parent: Workload management
6+
grand_parent: Availability and recovery
7+
---
8+
9+
# Workload group settings
10+
**Introduced 3.7**
11+
{: .label .label-purple }
12+
13+
OpenSearch operation is normally controlled by cluster-wide defaults and per-request parameters. In a multi-tenant cluster, you may need to apply different limits to different tenants. Without workload group settings, you can either lower the defaults for everyone, which restricts tenants that operate within limits, or trust every client to send the correct request parameters, which is difficult to enforce.
14+
15+
Workload group settings solve this problem by letting you attach group-specific configuration directly to a [workload group]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/workload-management/workload-groups/). When a request is routed to a group, the group's settings are applied automatically, letting you define guardrails per tenant:
16+
17+
- Apply stricter limits to resource-intensive or unverified tenants while keeping generous defaults for others, all without modifying cluster settings.
18+
- Limits are bound to the workload group, so they apply to every request routed to the group regardless of which client sent it. No client-side configuration is required.
19+
- A workload group can optionally take precedence over lenient request-level values, protecting the cluster from uncontrolled queries without rejecting them entirely.
20+
- All guardrails for a tenant are located in one place alongside the group's `resource_limits` and `resiliency_mode`.
21+
22+
## Supported settings
23+
24+
You can configure settings in the `settings` object of a workload group. All settings are optional. Only the settings you explicitly define on a workload group are enforced; any setting you omit defaults to the corresponding request parameter or cluster default. Each workload group setting accepts the same value range as the underlying request parameter or cluster setting it maps to.
25+
26+
The following table lists the supported workload group settings.
27+
28+
| Setting | Type | Description | Equivalent request parameter | Equivalent cluster setting |
29+
| :--- | :--- | :--- | :--- | :--- |
30+
| `search.default_search_timeout` | Time unit | The maximum amount of time a shard can spend on query execution. When a shard exceeds this timeout, it stops collecting hits and returns its current results to the coordinating node, which may produce partial results. | [`timeout`]({{site.url}}{{site.baseurl}}/api-reference/search-apis/search/) | [`search.default_search_timeout`]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/search-settings/) |
31+
| `search.cancel_after_time_interval` | Time unit | The maximum amount of time the entire search request can run at the coordinating node level. When the interval is reached, the request and all associated tasks are canceled and the client receives an error rather than partial results. | [`cancel_after_time_interval`]({{site.url}}{{site.baseurl}}/api-reference/search-apis/search/) | [`search.cancel_after_time_interval`]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/search-settings/) |
32+
| `search.max_concurrent_shard_requests` | Integer | The maximum number of concurrent shard-level requests a single search may issue per node. Limits search fan-out. | [`max_concurrent_shard_requests`]({{site.url}}{{site.baseurl}}/api-reference/search-apis/search/) | -- |
33+
| `search.batched_reduce_size` | Integer | The number of shard results combined into one batch on the coordinating node before the final reduction step. Lower values reduce coordinator memory usage when a search spans many shards. | [`batched_reduce_size`]({{site.url}}{{site.baseurl}}/api-reference/search-apis/search/) | -- |
34+
| `search.max_buckets` | Integer | The maximum number of aggregation buckets allowed in a single response. Guards against excessive memory use from large aggregations. | -- | [`search.max_buckets`]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/search-settings/) |
35+
| `override_request_values` | Boolean | Whether the workload group's settings take precedence over values supplied on the request. Default is `false`. See [Setting precedence](#setting-precedence). | -- | -- |
36+
37+
## Setting precedence
38+
39+
When a setting is defined on a workload group, OpenSearch resolves the effective value at request time using the following precedence rules:
40+
41+
- A workload group setting always takes precedence over the corresponding cluster setting when both are defined.
42+
- By default, an explicit value supplied on a request takes precedence over the workload group's setting. You can reverse this behavior by setting `override_request_values` to `true`.
43+
44+
The following table summarizes how the effective value is resolved.
45+
46+
| `override_request_values` | Precedence (highest to lowest) |
47+
| :--- | :--- |
48+
| `false` (Default) | Request parameter > Workload group setting > Cluster setting |
49+
| `true` | Workload group setting > Request parameter > Cluster setting |
50+
51+
## Creating a workload group containing settings
52+
53+
Add a `settings` object alongside the existing workload group fields:
54+
55+
```json
56+
PUT _wlm/workload_group
57+
{
58+
"name": "analytics",
59+
"resiliency_mode": "enforced",
60+
"resource_limits": {
61+
"cpu": 0.4,
62+
"memory": 0.2
63+
},
64+
"settings": {
65+
"search.default_search_timeout": "30s",
66+
"search.cancel_after_time_interval": "1m",
67+
"search.max_concurrent_shard_requests": 5,
68+
"search.batched_reduce_size": 512,
69+
"search.max_buckets": 10000
70+
}
71+
}
72+
```
73+
{% include copy-curl.html %}
74+
75+
## Updating workload group settings
76+
77+
You can update individual settings without affecting the other settings.
78+
79+
For example, to change only the search timeout for the `analytics` workload group:
80+
81+
```json
82+
PUT _wlm/workload_group/analytics
83+
{
84+
"settings": {
85+
"search.default_search_timeout": "1m"
86+
}
87+
}
88+
```
89+
{% include copy-curl.html %}
90+
91+
To remove a single setting, set its value to `null`:
92+
93+
```json
94+
PUT _wlm/workload_group/analytics
95+
{
96+
"settings": {
97+
"search.batched_reduce_size": null
98+
}
99+
}
100+
```
101+
{% include copy-curl.html %}
102+
103+
To clear all settings, send an empty `settings` object:
104+
105+
```json
106+
PUT _wlm/workload_group/analytics
107+
{
108+
"settings": {}
109+
}
110+
```
111+
{% include copy-curl.html %}
112+
113+
## Retrieving workload group settings
114+
115+
To retrieve workload group settings, use the [Workload Group API]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/workload-management/workload-groups/#retrieving-a-workload-group):
116+
117+
```json
118+
GET _wlm/workload_group/analytics
119+
```
120+
{% include copy-curl.html %}
121+
122+
## Deleting workload group settings
123+
124+
Settings are removed when the workload group is deleted. To remove individual settings without deleting the group, see [Updating workload group settings](#updating-workload-group-settings).

‎_tuning-your-cluster/availability-and-recovery/workload-management/workload-groups.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,7 @@ When creating or updating a workload group, you can specify the following parame
5858
| `name` | Create | The name of the workload group. |
5959
| `resiliency_mode` | Create or update | The resiliency mode of the workload group. Valid values are:<br>- `enforced` (queries are rejected if thresholds are exceeded). <br>- `soft` (queries can exceed thresholds if resources are available). <br>- `monitor` (queries are monitored but not canceled or rejected). <br> **Note**: These settings take effect only if the cluster-level `wlm.workload_group.mode` setting is `enabled`. See [Operating modes]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/workload-management/wlm-feature-overview/#operating-modes). |
6060
| `resource_limits` | Create or update | The resource limits for query requests in the workload group. Valid resources are `cpu` and `memory`. When creating a workload group, make sure that the sum of the resource limits for a single resource, either `cpu` or `memory`, does not exceed 1. |
61+
| `settings` | Create or update | Group-specific settings that are applied automatically to requests routed to the workload group. For supported settings and update behavior, see [Workload group settings]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/workload-management/workload-group-settings/). |
6162

6263
## Updating a workload group
6364

@@ -70,11 +71,16 @@ PUT _wlm/workload_group/analytics
7071
"resource_limits": {
7172
"cpu": 0.41,
7273
"memory": 0.21
74+
},
75+
"settings": {
76+
"search.default_search_timeout": "1m"
7377
}
7478
}
7579
```
7680
{% include copy-curl.html %}
7781

82+
For more information about the `settings` field, including supported settings and update behavior, see [Workload group settings]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/workload-management/workload-group-settings/).
83+
7884
## Retrieving a workload group
7985

8086
To retrieve all workload groups, use the following request:

0 commit comments

Comments
 (0)