Commit dc23a73
committed
Convert byte[] cells to ExprValue in fromObjectValue for IP / BINARY fields
`AnalyticsExecutionEngine.convertRows` reaches `ExprValueUtils.fromObjectValue`
for every cell in every row. OpenSearch `ip` and `binary` fields are stored
as raw bytes, so once the analytics-engine reader (post-#21681 commit 4)
materializes a parquet column, each cell arrives as a Java `byte[]`. The
fall-through catch-all then throws:
ExpressionEvaluationException: unsupported object class [B
at ExprValueUtils.fromObjectValue(ExprValueUtils.java:168)
at AnalyticsExecutionEngine.convertRows(AnalyticsExecutionEngine.java:128)
Every PPL query that even projects an ip column fails on the analytics route
with this error, regardless of operator or predicate.
`OpenSearchSchemaBuilder.mapFieldType` collapses both `ip` and `binary`
into Calcite `VARBINARY`, so the cell-level type system can't distinguish
the two. Length is the only signal available — IPv4 is exactly 4 bytes,
IPv6 is exactly 16, and `InetAddress.getByAddress` rejects every other
length, so the heuristic is unambiguous in practice:
- 4 or 16 bytes → ExprIpValue (canonical IP string)
- any other length → base64-encoded ExprStringValue (binary payload)
`InetAddress.getByAddress` may throw `UnknownHostException` only when the
length is not 4 or 16; we already gate on that, but the catch is kept as a
defensive fall-through to base64 for forward compatibility (e.g. future
RFC 6052-style 4or16-byte alternates).
Future work: differentiate `ip` and `binary` at the schema layer (separate
UDTs in `OpenSearchSchemaBuilder` + matching `schema_coerce.rs` handling)
so the type system can route directly without the length heuristic. Tracked
by the TODO already in `OpenSearchSchemaBuilder.mapFieldType`.
Adds three unit tests in `ExprValueUtilsTest` covering IPv4, IPv6, and
non-IP byte arrays.
Signed-off-by: Kai Huang <ahkcs@amazon.com>1 parent a224f19 commit dc23a73
2 files changed
Lines changed: 44 additions & 0 deletions
File tree
- core/src
- main/java/org/opensearch/sql/data/model
- test/java/org/opensearch/sql/data/model
Lines changed: 17 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
164 | 164 | | |
165 | 165 | | |
166 | 166 | | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
167 | 184 | | |
168 | 185 | | |
169 | 186 | | |
| |||
Lines changed: 27 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
244 | 244 | | |
245 | 245 | | |
246 | 246 | | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
247 | 274 | | |
248 | 275 | | |
249 | 276 | | |
| |||
0 commit comments