Skip to content

Commit 9a4587a

Browse files
committed
[Analytics Engine] Map BigDecimal cells to FloatingPoint in row-codec inference
{@code RowResponseCodec.scalarArrowType} ordered its instanceof checks {Long, Integer, Short, Byte, Double, Float, Boolean, CharSequence, byte[], Number(fallback) → Int(64)}. BigDecimal extends {@link Number} but isn't any of the typed scalar arms, so it fell through to the {@code Number} fallback and got encoded as a 64-bit integer column — silently truncating fractional digits. This bites PPL flows whose common element type is {@code DECIMAL} (e.g. {@code array(1, -1.5, 2, 1.0)} — the v2-side {@code ArrayImplementor.internalCast} explicitly maps the DECIMAL target to BigDecimal cells). The element values {@code -1.5} and {@code 1.0} round to {@code -1} and {@code 1} when forced through Int(64), so the array reads back as {@code [1, -1, 2, 1]} instead of {@code [1, -1.5, 2, 1.0]}. Promote BigDecimal cells to FloatingPoint(DOUBLE) — same precision the v2 engine uses for decimal-typed PPL results, so behavior matches across both execution paths. The list writer's {@code Float8Vector} arm already uses {@code ((Number) element).doubleValue()}, which correctly extracts the fractional value from a BigDecimal. Signed-off-by: Kai Huang <ahkcs@amazon.com>
1 parent f8c24f1 commit 9a4587a

1 file changed

Lines changed: 7 additions & 0 deletions

File tree

sandbox/plugins/analytics-engine/src/main/java/org/opensearch/analytics/exec/stage/RowResponseCodec.java

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -142,6 +142,13 @@ private static ArrowType scalarArrowType(Object value) {
142142
if (value instanceof Byte) return new ArrowType.Int(8, true);
143143
if (value instanceof Double) return new ArrowType.FloatingPoint(org.apache.arrow.vector.types.FloatingPointPrecision.DOUBLE);
144144
if (value instanceof Float) return new ArrowType.FloatingPoint(org.apache.arrow.vector.types.FloatingPointPrecision.SINGLE);
145+
// BigDecimal must be checked before the Number fallback below — a BigDecimal that
146+
// would round to a long but actually carries fractional precision (e.g. PPL
147+
// {@code array(1, -1.5)} where the common element type is DECIMAL and
148+
// {@code ArrayImplementor.internalCast} produces BigDecimal cells) would otherwise
149+
// get encoded as an integer Arrow vector and lose its fractional digits. Promote to
150+
// DOUBLE — the same path the v2 engine takes for decimal-typed PPL results.
151+
if (value instanceof java.math.BigDecimal) return new ArrowType.FloatingPoint(org.apache.arrow.vector.types.FloatingPointPrecision.DOUBLE);
145152
if (value instanceof Boolean) return ArrowType.Bool.INSTANCE;
146153
if (value instanceof CharSequence) return ArrowType.Utf8.INSTANCE;
147154
if (value instanceof byte[]) return ArrowType.Binary.INSTANCE;

0 commit comments

Comments
 (0)