Commit 62d617d
committed
Bridge live cluster settings into UnifiedQueryContext for PPL_REX_MAX_MATCH_LIMIT
The previous commit defaulted `PPL_REX_MAX_MATCH_LIMIT=10` in
`UnifiedQueryContext.Builder.settings` to fix the NPE in
`AstBuilder.visitRexCommand` on the unified path. The default is correct, but
it doesn't respect mid-run cluster overrides — every key in the static map
returns its hardcoded value regardless of `_cluster/settings` updates. This
breaks `CalciteRexCommandIT.testRexMaxMatchConfigurableLimit`, which
explicitly sets the cluster-side limit to 5 and asserts that `max_match=0`
caps at 5; on the unified path it stayed at 10.
This change introduces a `Builder.liveSettings(Settings)` hook that the REST
handler can use to inject the cluster's live `OpenSearchSettings` instance.
At `build()` time the Builder snapshots the live value of
`PPL_REX_MAX_MATCH_LIMIT` (only — see scoping note below) into the static
map, overriding the hardcoded default when the operator has set a cluster
value. Snapshot-at-build matches the per-HTTP-request lifecycle of
`UnifiedQueryContext` and avoids per-call lookup overhead.
## Why scoped to PPL_REX_MAX_MATCH_LIMIT only
The same architectural gap exists for every key in the static map
(`QUERY_SIZE_LIMIT`, `PPL_SUBSEARCH_MAXOUT`, `PPL_JOIN_SUBSEARCH_MAXOUT`,
`CALCITE_ENGINE_ENABLED`). For three of those, the static defaults are fine
in practice (no test overrides them mid-run; `head N` covers `QUERY_SIZE_LIMIT`
per-query). `CALCITE_ENGINE_ENABLED` is intentionally pinned to `true` for
the unified path — a cluster override toggling it off would defeat the point
of routing here. So this PR widens only the one key that demonstrably needs
it; widening the snapshot to the rest is a future scope decision tied to
whichever new IT first depends on it.
## Wire-up
`RestUnifiedQueryAction` gains a `pluginSettings` field (the same
`OpenSearchSettings` instance bound in the Guice module) and forwards it to
the Builder in both `buildContext` (per-request execution path) and
`buildParsingContext` (analytics-routing index name probe). Both
construction sites — `SQLPlugin.createSqlAnalyticsRouter` and
`TransportPPLQueryAction.<init>` — are updated to pass the existing
plugin-side `Settings` instance.
`buildParsingContext` had been `static` because it didn't need any instance
state; it's now an instance method since it reads `pluginSettings`.
## Test results
CalciteRexCommandIT through the analytics-engine route (every PPL query
forced through `/_analytics/ppl` via `tests.analytics.force_routing=true`):
* Before this change: 17/18 — `testRexMaxMatchConfigurableLimit` fails with
`expected:<5> but was:<10>` (cluster override doesn't reach the unified
path).
* After this change: 18/18 — all `testRexMaxMatch*` variants honor the
cluster setting.
## Companion PR
opensearch-project/OpenSearch#21550 — onboards PPL `rex` to DataFusion via
the analytics-engine path. The 17/18 baseline reported in that PR's
description was measured against the previous commit on this branch; with
this change the route hits 18/18.
Signed-off-by: Jialiang Liang <jiallian@amazon.com>1 parent 0f1c8ce commit 62d617d
4 files changed
Lines changed: 44 additions & 7 deletions
File tree
- api/src/main/java/org/opensearch/sql/api
- plugin/src/main/java/org/opensearch/sql/plugin
- rest
- transport
Lines changed: 32 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
113 | 113 | | |
114 | 114 | | |
115 | 115 | | |
| 116 | + | |
116 | 117 | | |
117 | 118 | | |
118 | 119 | | |
| |||
215 | 216 | | |
216 | 217 | | |
217 | 218 | | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
218 | 238 | | |
219 | 239 | | |
220 | 240 | | |
| |||
223 | 243 | | |
224 | 244 | | |
225 | 245 | | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
226 | 258 | | |
227 | 259 | | |
228 | 260 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
208 | 208 | | |
209 | 209 | | |
210 | 210 | | |
211 | | - | |
| 211 | + | |
212 | 212 | | |
213 | 213 | | |
214 | 214 | | |
| |||
Lines changed: 7 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
59 | 59 | | |
60 | 60 | | |
61 | 61 | | |
| 62 | + | |
62 | 63 | | |
63 | 64 | | |
64 | 65 | | |
65 | 66 | | |
66 | | - | |
| 67 | + | |
| 68 | + | |
67 | 69 | | |
68 | 70 | | |
69 | 71 | | |
| 72 | + | |
70 | 73 | | |
71 | 74 | | |
72 | 75 | | |
| |||
154 | 157 | | |
155 | 158 | | |
156 | 159 | | |
157 | | - | |
158 | | - | |
| 160 | + | |
| 161 | + | |
159 | 162 | | |
160 | 163 | | |
161 | 164 | | |
162 | 165 | | |
163 | 166 | | |
| 167 | + | |
164 | 168 | | |
165 | 169 | | |
166 | 170 | | |
| |||
Lines changed: 4 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
78 | 78 | | |
79 | 79 | | |
80 | 80 | | |
| 81 | + | |
| 82 | + | |
81 | 83 | | |
82 | 84 | | |
83 | 85 | | |
84 | | - | |
85 | | - | |
| 86 | + | |
86 | 87 | | |
87 | 88 | | |
88 | 89 | | |
89 | 90 | | |
90 | 91 | | |
91 | | - | |
| 92 | + | |
92 | 93 | | |
93 | 94 | | |
94 | 95 | | |
| |||
0 commit comments