Commit 968e781
committed
Bridge live cluster settings into UnifiedQueryContext for PPL_REX_MAX_MATCH_LIMIT
The previous commit defaulted `PPL_REX_MAX_MATCH_LIMIT=10` in
`UnifiedQueryContext.Builder.settings` to fix the NPE in
`AstBuilder.visitRexCommand` on the unified path. The default is correct, but
it doesn't respect mid-run cluster overrides — every key in the static map
returns its hardcoded value regardless of `_cluster/settings` updates. This
breaks `CalciteRexCommandIT.testRexMaxMatchConfigurableLimit`, which
explicitly sets the cluster-side limit to 5 and asserts that `max_match=0`
caps at 5; on the unified path it stayed at 10.
This change introduces a `Builder.liveSettings(Settings)` hook that the REST
handler can use to inject the cluster's live `OpenSearchSettings` instance.
At `build()` time the Builder snapshots the live value of
`PPL_REX_MAX_MATCH_LIMIT` (only — see scoping note below) into the static
map, overriding the hardcoded default when the operator has set a cluster
value. Snapshot-at-build matches the per-HTTP-request lifecycle of
`UnifiedQueryContext` and avoids per-call lookup overhead.
The same architectural gap exists for every key in the static map
(`QUERY_SIZE_LIMIT`, `PPL_SUBSEARCH_MAXOUT`, `PPL_JOIN_SUBSEARCH_MAXOUT`,
`CALCITE_ENGINE_ENABLED`). For three of those, the static defaults are fine
in practice (no test overrides them mid-run; `head N` covers `QUERY_SIZE_LIMIT`
per-query). `CALCITE_ENGINE_ENABLED` is intentionally pinned to `true` for
the unified path — a cluster override toggling it off would defeat the point
of routing here. So this PR widens only the one key that demonstrably needs
it; widening the snapshot to the rest is a future scope decision tied to
whichever new IT first depends on it.
`RestUnifiedQueryAction` gains a `pluginSettings` field (the same
`OpenSearchSettings` instance bound in the Guice module) and forwards it to
the Builder in both `buildContext` (per-request execution path) and
`buildParsingContext` (analytics-routing index name probe). Both
construction sites — `SQLPlugin.createSqlAnalyticsRouter` and
`TransportPPLQueryAction.<init>` — are updated to pass the existing
plugin-side `Settings` instance.
`buildParsingContext` had been `static` because it didn't need any instance
state; it's now an instance method since it reads `pluginSettings`.
CalciteRexCommandIT through the analytics-engine route (every PPL query
forced through `/_analytics/ppl` via `tests.analytics.force_routing=true`):
* Before this change: 17/18 — `testRexMaxMatchConfigurableLimit` fails with
`expected:<5> but was:<10>` (cluster override doesn't reach the unified
path).
* After this change: 18/18 — all `testRexMaxMatch*` variants honor the
cluster setting.
opensearch-project/OpenSearch#21550 — onboards PPL `rex` to DataFusion via
the analytics-engine path. The 17/18 baseline reported in that PR's
description was measured against the previous commit on this branch; with
this change the route hits 18/18.
Signed-off-by: Jialiang Liang <jiallian@amazon.com>1 parent 3763d17 commit 968e781
4 files changed
Lines changed: 47 additions & 7 deletions
File tree
- api/src/main/java/org/opensearch/sql/api
- plugin/src/main/java/org/opensearch/sql/plugin
- rest
- transport
Lines changed: 33 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
117 | 117 | | |
118 | 118 | | |
119 | 119 | | |
| 120 | + | |
120 | 121 | | |
121 | 122 | | |
122 | 123 | | |
| |||
219 | 220 | | |
220 | 221 | | |
221 | 222 | | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
222 | 242 | | |
223 | 243 | | |
224 | 244 | | |
| |||
232 | 252 | | |
233 | 253 | | |
234 | 254 | | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
235 | 268 | | |
236 | 269 | | |
237 | 270 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
208 | 208 | | |
209 | 209 | | |
210 | 210 | | |
211 | | - | |
| 211 | + | |
212 | 212 | | |
213 | 213 | | |
214 | 214 | | |
| |||
Lines changed: 7 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
59 | 59 | | |
60 | 60 | | |
61 | 61 | | |
| 62 | + | |
62 | 63 | | |
63 | 64 | | |
64 | 65 | | |
65 | 66 | | |
66 | | - | |
| 67 | + | |
| 68 | + | |
67 | 69 | | |
68 | 70 | | |
69 | 71 | | |
| 72 | + | |
70 | 73 | | |
71 | 74 | | |
72 | 75 | | |
| |||
154 | 157 | | |
155 | 158 | | |
156 | 159 | | |
157 | | - | |
158 | | - | |
| 160 | + | |
| 161 | + | |
159 | 162 | | |
160 | 163 | | |
161 | 164 | | |
162 | 165 | | |
163 | 166 | | |
| 167 | + | |
164 | 168 | | |
165 | 169 | | |
166 | 170 | | |
| |||
Lines changed: 6 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
67 | 67 | | |
68 | 68 | | |
69 | 69 | | |
| 70 | + | |
70 | 71 | | |
71 | 72 | | |
72 | 73 | | |
| |||
82 | 83 | | |
83 | 84 | | |
84 | 85 | | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
85 | 89 | | |
86 | 90 | | |
87 | 91 | | |
88 | | - | |
89 | | - | |
| 92 | + | |
90 | 93 | | |
91 | 94 | | |
92 | 95 | | |
| |||
105 | 108 | | |
106 | 109 | | |
107 | 110 | | |
108 | | - | |
| 111 | + | |
109 | 112 | | |
110 | 113 | | |
111 | 114 | | |
| |||
0 commit comments