Commit 7da02d5
authored
Carry CalciteFieldFormatCommandIT through the helper-managed index path (#5417)
Same shape as #5407 for CalciteEvalCommandIT. The IT's init() previously
created the in-test test_eval index via direct `PUT /test_eval/_doc/N`
requests, relying on dynamic mapping. Two problems:
1. The doc PUTs auto-create the index with whatever settings the cluster
defaults to. The analytics-engine compatibility path (force-routing on;
tests.analytics.parquet_indices=true) needs parquet-backed indices,
which TestUtils.createIndexByRestClient applies via
TestUtils.makeParquetBacked when the system property is set. Direct
PUTs sidestep that helper, so test_eval lands as Lucene-backed and
the analytics planner rejects it with "No backend can scan all
requested fields on index [test_eval]". All four working tests fail
at execution.
2. init() runs before every @test method. The doc PUTs are doc-level
idempotent, so re-running was wasteful but not failing. Once we
switch to createIndexByRestClient, the index-level PUT is no longer
idempotent and re-running throws "resource_already_exists_exception".
Both addressed in one change:
- test_eval is created via TestUtils.createIndexByRestClient with an
explicit mapping (name/title=keyword, age=long). The helper honours
tests.analytics.parquet_indices=true and produces a parquet-backed
index for the analytics-engine sweep; on the v2 path the helper is a
no-op around the index PUT, so behaviour is unchanged.
- The whole init body is guarded by TestUtils.isIndexExist — same
idempotency idiom that loadIndex uses for predefined fixtures. First
@test method provisions; subsequent methods skip.
Also pins the projection order on testFieldFormatStringConcatenation. The
original query (`source=test_eval | fieldformat greeting = 'Hello ' +
name`) had no `| fields` clause and relied on the implicit projection's
column order — v2 returns Lucene-source insertion order, analytics returns
parquet-storage order (alphabetical), so the assertion only matched on v2
by coincidence. Adding `| fields name, title, age, greeting` makes the
assertion deterministic across paths; the existing expected rows
(`rows("Alice", "Engineer", 25, "Hello Alice")`) already match this
order, so v2 behaviour is preserved.
The other four tests already had explicit `| fields ...` clauses, so no
change there.
No semantic change for the v2 path: the explicit mapping types (keyword,
long, keyword) resolve to the same PPL types ("string", "bigint", "string")
that dynamic mapping inferred, and fieldformat reads from _source either
way.
Analytics-route compatibility goes from 1/5 to 4/5 (verified locally
against a runTask cluster with analytics-engine + opensearch-sql-plugin).
The remaining `testFieldFormatStringConcatenationWithNullFieldToString`
needs a `tostring()` UDF on the analytics path — a multi-mode UDF (binary
/ hex / commas / duration) tracked separately as out of scope.
Test plan:
- ./gradlew :integ-test:integTest --tests
'org.opensearch.sql.calcite.remote.CalciteFieldFormatCommandIT'
-> 5/5 green (v2 path, no regression).
- ./gradlew :integ-test:analyticsCompatibilityTest --tests
'org.opensearch.sql.calcite.remote.CalciteFieldFormatCommandIT'
-> 4/5 pass; the 5th fails on `tostring`'s missing capability registration,
which is the documented out-of-scope category.
Signed-off-by: Kai Huang <ahkcs@amazon.com>1 parent 5d31506 commit 7da02d5
1 file changed
Lines changed: 34 additions & 11 deletions
Lines changed: 34 additions & 11 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
| 15 | + | |
15 | 16 | | |
16 | 17 | | |
17 | 18 | | |
| |||
23 | 24 | | |
24 | 25 | | |
25 | 26 | | |
26 | | - | |
27 | | - | |
28 | | - | |
29 | | - | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
30 | 41 | | |
31 | | - | |
32 | | - | |
33 | | - | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
34 | 46 | | |
35 | | - | |
36 | | - | |
37 | | - | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
38 | 55 | | |
39 | 56 | | |
40 | 57 | | |
41 | 58 | | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
42 | 64 | | |
43 | 65 | | |
44 | 66 | | |
45 | | - | |
| 67 | + | |
| 68 | + | |
46 | 69 | | |
47 | 70 | | |
48 | 71 | | |
| |||
0 commit comments