Commit ced6b21
committed
Collapse array_length impls to single list<any1>
CI surfaced this on the post-rebase rex run:
Duplicate key FunctionAnchor{urn=extension:org.opensearch:scalar_functions,
key=array_length:list} (attempted merging values
array_length:list and array_length:list)
The Part 3 commit declared two impls — `list<varchar<L1>>` and `list<string>`
— with the intent of covering both element-type families produced by
`rex_extract_multi`'s pair of impls. But substrait's compound function key
drops the inner parametric element type at the key level, so both impls
collapse to the same key `array_length:list`. The YAML loader rejects the
collision when the analytics-backend-datafusion plugin's
`SimpleExtension.ExtensionCollection` merges the file in.
Replace the two impls with a single `list<any1>` polymorphic impl. The
`any1` type variable matches any element type at planning, so a call site
that produces `list<varchar<L1>>` (rex_extract_multi varchar overload) and
a call site that produces `list<string>` (rex_extract_multi string
overload) both bind to the one impl. Net effect on planning is equivalent
and the duplicate-key collision goes away.
The duplicate didn't surface on the original rex CI run because the prior
PPL_REX_MAX_MATCH_LIMIT NPE failed every query at plan time before the
function-extension merge was reached. Once the mavenLocal pin fix landed
the prior commit and queries actually reached the planner, this older
latent collision was unmasked.
Signed-off-by: Jialiang Liang <jiallian@amazon.com>1 parent b9b9da9 commit ced6b21
2 files changed
Lines changed: 6 additions & 11 deletions
File tree
- sandbox/plugins/analytics-backend-datafusion/src/main
- java/org/opensearch/be/datafusion
- resources
Lines changed: 0 additions & 6 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
140 | 140 | | |
141 | 141 | | |
142 | 142 | | |
143 | | - | |
144 | | - | |
145 | | - | |
146 | | - | |
147 | | - | |
148 | | - | |
149 | 143 | | |
150 | 144 | | |
151 | 145 | | |
| |||
Lines changed: 6 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
441 | 441 | | |
442 | 442 | | |
443 | 443 | | |
| 444 | + | |
| 445 | + | |
| 446 | + | |
| 447 | + | |
| 448 | + | |
444 | 449 | | |
445 | 450 | | |
446 | | - | |
447 | | - | |
448 | | - | |
449 | | - | |
450 | | - | |
| 451 | + | |
451 | 452 | | |
452 | 453 | | |
453 | 454 | | |
| |||
0 commit comments