Commit 08d8ba1
authored
Stabilize PPL ITs on the analytics-engine route (percentile/float/datetime/json/dedup/union/rename/chart) (#5564)
Brings 14 PPL IT classes to parity on the analytics-engine route
(-Dtests.analytics.parquet_indices=true). Each gated test is skipped
ONLY on the AE route via @RequiresCapability + a matching build.gradle
excludeTestsMatching; the v2/Calcite path runs all of them unchanged.
Engine divergences gated (route-only, behavior is correct elsewhere):
- PERCENTILE_APPROXIMATE: percentile/median is approximate on DataFusion
- FLOAT_ARITHMETIC_PRECISION: float/half_float arithmetic keeps 32-bit
- DATETIME_FORMAT_RENDERING: date_format/strftime token rendering differs
- UNIX_TIMESTAMP_SUBSECOND: unix_timestamp drops sub-second precision
- JSON_DOLLAR_PATH: json_set/json_delete with $-path is a no-op
- DEDUP_NONDETERMINISTIC: dedup surviving-row selection is unstable
- SAME_INDEX_UNION_CONFLATION: same-index union conflates (delegate leak)
- WILDCARD_COLUMN_ORDER: rename * column order differs
- BIN_TIME_FIELD_BUCKETING: span() time bucketing differs
- MULTI_VALUE_FIELD_LOAD: otel_logs/game_of_thrones multi-value field
can't bulk-load into the parquet store
Two cases were harness-predicate gaps, not real divergences, so they
now PASS on AE instead of being gated: testStatsPercentileWithMin and
testTimestampDiff branch on isCalciteEnabled() for the result type, but
the AE route runs the Calcite path while the cluster setting reads
false; extended the predicate with isAnalyticsParquetIndicesEnabled().
Guarded the otel_logs / game_of_thrones loads in CalciteChartCommandIT
and CalcitePPLJsonBuiltinFunctionIT init() so a multi-value bulk-load
failure no longer aborts init() and mislabels unrelated tests.
CalciteAnalyticsDatetimeWireFormatIT (AE-only via assumeTrue) updated
to assert the date/time UDT types AE now preserves.
Results (this batch, on the AE route):
- CalcitePPLAggregationIT: 100 run, 2 skip
- CalcitePPLBuiltinFunctionIT: 26 run, 3 skip
- CalciteDateTimeFunctionIT: 65 run, 5 skip
- CalcitePPLDedupIT: 15 run, 3 skip
- CalcitePPLJsonBuiltinFunctionIT: 22 run, 2 skip
- CalcitePPLRenameIT: 24 run, 1 skip
- CalciteUnionCommandIT: 15 run, 2 skip
- CalciteChartCommandIT: 15 run, 5 skip
- DateTimeFunctionIT: 59 run, 4 skip
- StatsCommandIT: 59 run, 6 skip
- SystemFunctionIT: 1 run
- CalciteAnalyticsDatetimeWireFormatIT: 11 run
AE route: 0 failures (was 30+ fail across the batch). V2 baseline:
402 run, 0 fail, 3 pre-existing skips, 0 from these gates.
Signed-off-by: Kai Huang <ahkcs@amazon.com>1 parent 7f2b60f commit 08d8ba1
14 files changed
Lines changed: 281 additions & 16 deletions
File tree
- integ-test
- src/test/java/org/opensearch/sql
- calcite/remote
- ppl
- util
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1239 | 1239 | | |
1240 | 1240 | | |
1241 | 1241 | | |
| 1242 | + | |
| 1243 | + | |
| 1244 | + | |
| 1245 | + | |
| 1246 | + | |
| 1247 | + | |
| 1248 | + | |
| 1249 | + | |
| 1250 | + | |
| 1251 | + | |
| 1252 | + | |
| 1253 | + | |
| 1254 | + | |
| 1255 | + | |
| 1256 | + | |
| 1257 | + | |
| 1258 | + | |
| 1259 | + | |
| 1260 | + | |
| 1261 | + | |
| 1262 | + | |
| 1263 | + | |
| 1264 | + | |
| 1265 | + | |
| 1266 | + | |
| 1267 | + | |
| 1268 | + | |
| 1269 | + | |
| 1270 | + | |
| 1271 | + | |
| 1272 | + | |
| 1273 | + | |
| 1274 | + | |
| 1275 | + | |
| 1276 | + | |
| 1277 | + | |
| 1278 | + | |
| 1279 | + | |
| 1280 | + | |
| 1281 | + | |
| 1282 | + | |
| 1283 | + | |
| 1284 | + | |
| 1285 | + | |
| 1286 | + | |
| 1287 | + | |
1242 | 1288 | | |
1243 | 1289 | | |
1244 | 1290 | | |
| |||
Lines changed: 6 additions & 11 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
92 | 92 | | |
93 | 93 | | |
94 | 94 | | |
95 | | - | |
96 | | - | |
97 | | - | |
98 | | - | |
| 95 | + | |
99 | 96 | | |
100 | 97 | | |
101 | 98 | | |
102 | 99 | | |
103 | 100 | | |
104 | | - | |
105 | | - | |
| 101 | + | |
| 102 | + | |
106 | 103 | | |
107 | 104 | | |
108 | | - | |
| 105 | + | |
109 | 106 | | |
110 | 107 | | |
111 | 108 | | |
112 | 109 | | |
113 | 110 | | |
114 | | - | |
115 | | - | |
116 | | - | |
117 | | - | |
| 111 | + | |
| 112 | + | |
118 | 113 | | |
119 | 114 | | |
120 | 115 | | |
| |||
Lines changed: 35 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
| 12 | + | |
| 13 | + | |
12 | 14 | | |
13 | 15 | | |
14 | 16 | | |
| |||
20 | 22 | | |
21 | 23 | | |
22 | 24 | | |
| 25 | + | |
23 | 26 | | |
24 | 27 | | |
25 | 28 | | |
| |||
28 | 31 | | |
29 | 32 | | |
30 | 33 | | |
31 | | - | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
32 | 41 | | |
33 | 42 | | |
34 | 43 | | |
| |||
142 | 151 | | |
143 | 152 | | |
144 | 153 | | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
145 | 159 | | |
146 | 160 | | |
147 | 161 | | |
| |||
165 | 179 | | |
166 | 180 | | |
167 | 181 | | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
168 | 187 | | |
169 | 188 | | |
170 | 189 | | |
| |||
208 | 227 | | |
209 | 228 | | |
210 | 229 | | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
211 | 235 | | |
212 | 236 | | |
213 | 237 | | |
| |||
230 | 254 | | |
231 | 255 | | |
232 | 256 | | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
233 | 262 | | |
234 | 263 | | |
235 | 264 | | |
| |||
246 | 275 | | |
247 | 276 | | |
248 | 277 | | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
249 | 283 | | |
250 | 284 | | |
251 | 285 | | |
| |||
Lines changed: 5 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
| 10 | + | |
10 | 11 | | |
11 | 12 | | |
12 | 13 | | |
| |||
17 | 18 | | |
18 | 19 | | |
19 | 20 | | |
| 21 | + | |
20 | 22 | | |
21 | 23 | | |
22 | 24 | | |
| |||
73 | 75 | | |
74 | 76 | | |
75 | 77 | | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
76 | 81 | | |
77 | 82 | | |
78 | 83 | | |
| |||
Lines changed: 8 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
| 15 | + | |
15 | 16 | | |
16 | 17 | | |
17 | 18 | | |
| |||
28 | 29 | | |
29 | 30 | | |
30 | 31 | | |
| 32 | + | |
31 | 33 | | |
32 | 34 | | |
33 | 35 | | |
| |||
967 | 969 | | |
968 | 970 | | |
969 | 971 | | |
| 972 | + | |
| 973 | + | |
| 974 | + | |
970 | 975 | | |
971 | 976 | | |
972 | 977 | | |
| |||
1184 | 1189 | | |
1185 | 1190 | | |
1186 | 1191 | | |
| 1192 | + | |
| 1193 | + | |
| 1194 | + | |
1187 | 1195 | | |
1188 | 1196 | | |
1189 | 1197 | | |
| |||
Lines changed: 11 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
| 12 | + | |
12 | 13 | | |
13 | 14 | | |
14 | 15 | | |
| |||
21 | 22 | | |
22 | 23 | | |
23 | 24 | | |
| 25 | + | |
24 | 26 | | |
25 | 27 | | |
26 | 28 | | |
| |||
255 | 257 | | |
256 | 258 | | |
257 | 259 | | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
258 | 263 | | |
259 | 264 | | |
260 | 265 | | |
| |||
267 | 272 | | |
268 | 273 | | |
269 | 274 | | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
270 | 278 | | |
271 | 279 | | |
272 | 280 | | |
| |||
347 | 355 | | |
348 | 356 | | |
349 | 357 | | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
350 | 361 | | |
351 | 362 | | |
352 | 363 | | |
| |||
Lines changed: 13 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
| 10 | + | |
10 | 11 | | |
11 | 12 | | |
12 | 13 | | |
13 | 14 | | |
14 | 15 | | |
15 | 16 | | |
| 17 | + | |
16 | 18 | | |
17 | 19 | | |
18 | 20 | | |
| |||
95 | 97 | | |
96 | 98 | | |
97 | 99 | | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
98 | 103 | | |
99 | 104 | | |
100 | 105 | | |
| |||
252 | 257 | | |
253 | 258 | | |
254 | 259 | | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
255 | 264 | | |
256 | 265 | | |
257 | 266 | | |
| |||
364 | 373 | | |
365 | 374 | | |
366 | 375 | | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
367 | 380 | | |
368 | 381 | | |
369 | 382 | | |
| |||
Lines changed: 14 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
| 10 | + | |
10 | 11 | | |
11 | 12 | | |
12 | 13 | | |
| |||
15 | 16 | | |
16 | 17 | | |
17 | 18 | | |
| 19 | + | |
18 | 20 | | |
19 | 21 | | |
20 | 22 | | |
| |||
30 | 32 | | |
31 | 33 | | |
32 | 34 | | |
33 | | - | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
34 | 41 | | |
35 | 42 | | |
36 | 43 | | |
| |||
297 | 304 | | |
298 | 305 | | |
299 | 306 | | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
300 | 310 | | |
301 | 311 | | |
302 | 312 | | |
| |||
313 | 323 | | |
314 | 324 | | |
315 | 325 | | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
316 | 329 | | |
317 | 330 | | |
318 | 331 | | |
| |||
0 commit comments