fix: unsigned dtype handling in TUI, filter predicates, and Calcite (#216)

Follow-up to #208 (CSV export): the remaining consumers still read unsigned
integer columns through raw signed getters, so high-half values (bit set at the
width's sign position) were mishandled. Per site:

- cli tui GridRender / InspectorRender: U64/U32 now render via
  Long/Integer.toUnsignedString; U16/U8 render via the dtype-aware getInt
  (zero-extend). Display bug only, but visible on any unsigned column.
- cli FilterCommand.compareValue: the worst class — silent wrong query results.
  Filter predicates now widen U8/U16/U32 losslessly into a non-negative long and
  compare U64 with Long.compareUnsigned, so `magnesium >= 130` matches its
  high-half U8 rows instead of dropping them. Made package-private + unit-tested.
- calcite VortexTable: unsigned columns now map to the next wider signed SQL type
  so the declared type holds their full range (U8->SMALLINT, U16->INTEGER,
  U32->BIGINT) and value() widens accordingly. U64 has no wider SQL integer, so
  it stays BIGINT and value() throws VortexException for values >= 2^63 rather
  than surfacing a two's-complement negative.
- calcite VortexAggregates.scanSum: U32 elements widen via toUnsignedLong;
  U64 elements >= 2^63 throw (aligned with the VortexTable decision). Branch-split
  on the loop-invariant unsigned flag keeps the signed body vectorizable.

U64 decision: mapped to signed BIGINT because Calcite has no wider SQL integer;
a documented VortexException for the unrepresentable high half is chosen over a
silent negative (loud beats wrong) — both when a row is read and when it is
summed.

Closes #216

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Author: dfa1

CHANGELOG.md

@@ -11,6 +11,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 - Scanning a struct whose columns include a nested struct no longer fails chunk planning. A nested `vortex.struct` layout column (e.g. Raincloud `countries-of-the-world`'s `data` column) is now treated as a single full-range chunk source and decoded through the layout registry's new struct decoder into a `StructArray`, matching the Rust reference (a nested struct spans its parent's row range, not an independent chunking). `schema`/`count`/`inspect` already worked; plain scans and `select` of sibling columns now work too. CSV export of the struct column itself remains unsupported (a separate rendering limitation). ([#207](https://github.com/dfa1/vortex-java/issues/207))
 - CSV export renders unsigned integer columns (U8–U64) with their unsigned values — high-half values previously printed as two's-complement negatives (uci-wine `magnesium` U8 132 exported as -124), silent corruption found by the Raincloud conformance suite. ([#208](https://github.com/dfa1/vortex-java/issues/208))
+- Unsigned integer columns are handled unsigned in the remaining consumers: the CLI `filter` predicates (`magnesium >= 130` now matches its high-half U8 rows instead of silently dropping them — the worst class of the bug: wrong query results), the TUI grid and inspector views, and the Calcite adapter (U8/U16/U32 map to the next wider signed SQL type so their full range fits; U64 stays `BIGINT` and fails loud rather than surfacing a negative for values ≥ 2^63, both when read and when summed). ([#216](https://github.com/dfa1/vortex-java/issues/216))
 - `fastlanes.rle` decodes F64/F32 value pools (`LazyRleDoubleArray`, `LazyRleFloatArray`) — files from the Python bindings RLE-encode double columns with long constant runs, which previously failed to scan with `unsupported ptype F64`. ([#209](https://github.com/dfa1/vortex-java/issues/209))
 - Lazy dict decode now covers I8/U8/I16/U16 value columns (`DictByteArray`, `DictShortArray`) — files from the Python bindings dict-encode narrow-integer columns, which previously failed to scan with `unsupported ptype for lazy dict`. ([#206](https://github.com/dfa1/vortex-java/issues/206))
 - CSV export handles all-null (`DType.Null`) columns as empty fields instead of throwing `unsupported array type: NullArray`. ([#211](https://github.com/dfa1/vortex-java/issues/211))

calcite/src/main/java/io/github/dfa1/vortex/calcite/VortexAggregates.java

@@ -1,6 +1,8 @@
 package io.github.dfa1.vortex.calcite;
 
+import io.github.dfa1.vortex.core.error.VortexException;
 import io.github.dfa1.vortex.core.model.ColumnName;
+import io.github.dfa1.vortex.core.model.DType;
 import io.github.dfa1.vortex.reader.ArrayStats;
 import io.github.dfa1.vortex.reader.Chunk;
 import io.github.dfa1.vortex.reader.ScanIterator;
@@ -83,6 +85,10 @@ public static Summary of(VortexReader reader, String column) {
                 Source.ZONE_STATS_PUSHDOWN, sumSource);
     }
 
+    private static boolean isUnsigned(Array arr) {
+        return arr.dtype() instanceof DType.Primitive p && p.ptype().isUnsigned();
+    }
+
     private static long totalRows(VortexReader reader) {
         try (ScanIterator scan = reader.scan(ScanOptions.all())) {
             long total = 0L;
@@ -103,13 +109,36 @@ private static Number scanSum(VortexReader reader, String column) {
                     long n = chunk.rowCount();
                     switch (chunk.<Array>column(column)) {
                         case LongArray a -> {
-                            for (long i = 0; i < n; i++) {
-                                longSum += a.getLong(i);
+                            // Branch-split on the loop-invariant unsigned flag so the signed body
+                            // stays vectorizable. A U64 element with the high bit set (>= 2^63) has
+                            // no signed-long home and would corrupt the sum, so fail loud — aligned
+                            // with VortexTable mapping U64 to signed BIGINT.
+                            if (isUnsigned(a)) {
+                                for (long i = 0; i < n; i++) {
+                                    long v = a.getLong(i);
+                                    if (v < 0) {
+                                        throw new VortexException("U64 value " + Long.toUnsignedString(v)
+                                                + " exceeds the signed BIGINT range SUM accumulates into");
+                                    }
+                                    longSum += v;
+                                }
+                            } else {
+                                for (long i = 0; i < n; i++) {
+                                    longSum += a.getLong(i);
+                                }
                             }
                         }
                         case IntArray a -> {
-                            for (long i = 0; i < n; i++) {
-                                longSum += a.getInt(i);
+                            // U32 widens losslessly into the signed accumulator via toUnsignedLong;
+                            // branch-split keeps the signed body free of the per-element widen.
+                            if (isUnsigned(a)) {
+                                for (long i = 0; i < n; i++) {
+                                    longSum += Integer.toUnsignedLong(a.getInt(i));
+                                }
+                            } else {
+                                for (long i = 0; i < n; i++) {
+                                    longSum += a.getInt(i);
+                                }
                             }
                         }
                         case DoubleArray a -> {

calcite/src/main/java/io/github/dfa1/vortex/calcite/VortexTable.java

@@ -1,5 +1,6 @@
 package io.github.dfa1.vortex.calcite;
 
+import io.github.dfa1.vortex.core.error.VortexException;
 import io.github.dfa1.vortex.core.model.ColumnName;
 import io.github.dfa1.vortex.core.model.DType;
 import io.github.dfa1.vortex.reader.ArrayStats;
@@ -809,10 +810,17 @@ private static Object value(Object array, DType type, long r) {
             case DType.Primitive p -> switch (p.ptype()) {
                 case F64 -> ((DoubleArray) array).getDouble(r);
                 case F32 -> (double) ((FloatArray) array).getFloat(r);
-                case I64, U64 -> ((LongArray) array).getLong(r);
+                case I64 -> ((LongArray) array).getLong(r);
+                // U64 maps to signed BIGINT (no wider SQL integer exists): values with the high bit
+                // set have no signed-long representation, so fail loud rather than surface a
+                // negative to SQL — silent wrong results are the worse outcome.
+                case U64 -> unsignedBigint(((LongArray) array).getLong(r));
+                // U32 exceeds signed INTEGER, so it maps to BIGINT and widens losslessly here.
+                case U32 -> Integer.toUnsignedLong(((IntArray) array).getInt(r));
                 // Narrow ints decode to their own array width (Byte/Short/Int), not IntArray —
-                // each exposes getInt(r) with the correct sign/zero extension.
-                case I32, U32 -> ((IntArray) array).getInt(r);
+                // each exposes getInt(r) with the correct sign/zero extension. U16/U8 zero-extend
+                // into the wider signed SQL type (INTEGER/SMALLINT) they map to.
+                case I32 -> ((IntArray) array).getInt(r);
                 case I16, U16 -> ((ShortArray) array).getInt(r);
                 case I8, U8 -> ((ByteArray) array).getInt(r);
                 default -> throw new IllegalStateException("unsupported ptype: " + p.ptype());
@@ -823,6 +831,20 @@ private static Object value(Object array, DType type, long r) {
         };
     }
 
+    /// Guards a U64 value being surfaced to Calcite's signed `BIGINT`: values with the high bit set
+    /// (`>= 2^63`) have no lossless signed-long representation, so this throws rather than let a
+    /// negative reach SQL. Vortex has no wider unsigned SQL type to widen into, so loud beats wrong.
+    ///
+    /// @param raw the raw U64 bits read from the column
+    /// @return `raw` unchanged when it fits the non-negative signed-long range
+    private static long unsignedBigint(long raw) {
+        if (raw < 0) {
+            throw new VortexException("U64 value " + Long.toUnsignedString(raw)
+                    + " exceeds the signed BIGINT range Calcite maps U64 to");
+        }
+        return raw;
+    }
+
     /// Translates the Calcite predicates into a zone-map [RowFilter], keeping only the comparisons
     /// we can prune on (`=`, `<>`, `<`, `<=`, `>`, `>=` between a column and a literal, plus `AND`).
     /// Predicates we don't understand are simply not pushed — Calcite still applies them.
@@ -966,15 +988,23 @@ private static Object literalValue(RexLiteral lit, DType type) {
         };
     }
 
+    /// Maps a Vortex dtype to a Calcite SQL type. Unsigned integers map to the next wider signed
+    /// SQL type so the declared type can hold their full range (`U8`->`SMALLINT`, `U16`->`INTEGER`,
+    /// `U32`->`BIGINT`); `U64` is the exception — no wider SQL integer exists, so it maps to
+    /// `BIGINT` and [#value(Object, DType, long)] guards the high-half values that cannot fit.
+    ///
+    /// @param factory the Calcite type factory
+    /// @param type    the Vortex column dtype
+    /// @return the nullable-adjusted SQL type
     private static RelDataType toSqlType(RelDataTypeFactory factory, DType type) {
         RelDataType sql = switch (type) {
             case DType.Primitive p -> switch (p.ptype()) {
                 case F64 -> factory.createSqlType(SqlTypeName.DOUBLE);
                 case F32 -> factory.createSqlType(SqlTypeName.REAL);
-                case I64, U64 -> factory.createSqlType(SqlTypeName.BIGINT);
-                case I32, U32 -> factory.createSqlType(SqlTypeName.INTEGER);
-                case I16, U16 -> factory.createSqlType(SqlTypeName.SMALLINT);
-                case I8, U8 -> factory.createSqlType(SqlTypeName.TINYINT);
+                case I64, U64, U32 -> factory.createSqlType(SqlTypeName.BIGINT);
+                case I32, U16 -> factory.createSqlType(SqlTypeName.INTEGER);
+                case I16, U8 -> factory.createSqlType(SqlTypeName.SMALLINT);
+                case I8 -> factory.createSqlType(SqlTypeName.TINYINT);
                 default -> throw new IllegalStateException("unsupported ptype: " + p.ptype());
             };
             case DType.Utf8 _ -> factory.createSqlType(SqlTypeName.VARCHAR);

calcite/src/test/java/io/github/dfa1/vortex/calcite/UnsignedColumnTest.java

@@ -0,0 +1,132 @@
+package io.github.dfa1.vortex.calcite;
+
+import io.github.dfa1.vortex.core.error.VortexException;
+import io.github.dfa1.vortex.core.model.ColumnName;
+import io.github.dfa1.vortex.core.model.DType;
+import io.github.dfa1.vortex.reader.ReadRegistry;
+import io.github.dfa1.vortex.reader.VortexReader;
+import io.github.dfa1.vortex.writer.VortexWriter;
+import io.github.dfa1.vortex.writer.WriteOptions;
+
+import org.apache.calcite.linq4j.Enumerator;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.io.TempDir;
+
+import java.nio.channels.FileChannel;
+import java.nio.file.Path;
+import java.nio.file.StandardOpenOption;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+
+/// Unsigned high-half handling in the Calcite adapter (#216): U32 widens losslessly into `BIGINT`
+/// and U64 — mapped to signed `BIGINT` because no wider SQL integer exists — fails loud rather than
+/// surfacing a two's-complement negative to SQL. The canonical motivating shape is an unsigned
+/// integer column (like uci-wine's magnesium); here each column carries a boundary high-half value.
+class UnsignedColumnTest {
+
+    // A U32 value beyond signed INTEGER (0xB2D05E00): getInt reads it as a negative int.
+    private static final long U32_HIGH = 3_000_000_000L;
+    // A U64 value with the high bit set (2^63): getLong reads it as Long.MIN_VALUE.
+    private static final long U64_HIGH = Long.MIN_VALUE;
+
+    private static final DType.Struct SCHEMA = DType.structBuilder()
+            .field("u32", DType.U32)
+            .field("u64", DType.U64)
+            .build();
+
+    @TempDir
+    Path tmp;
+
+    @Test
+    void scan_widensU32ToLongAndKeepsInRangeU64() throws Exception {
+        // Given — a row whose U32 exceeds signed INTEGER and whose U64 stays in the low half
+        Path file = write("in-range.vortex", WriteOptions.defaults(), (int) U32_HIGH, 4000L);
+
+        // When — a full scan materializes the row through VortexTable.value
+        List<Object[]> rows = drain(new VortexTable(file));
+
+        // Then — U32 widened to a positive Long (not a negative int), U64 kept as its Long value
+        assertThat(rows).hasSize(1);
+        assertThat(rows.getFirst()[0]).isEqualTo(U32_HIGH);
+        assertThat(rows.getFirst()[1]).isEqualTo(4000L);
+    }
+
+    @Test
+    void scan_u64HighHalf_failsLoudRatherThanSurfaceNegative() throws Exception {
+        // Given — a U64 value with no lossless signed-BIGINT representation
+        Path file = write("u64-high.vortex", WriteOptions.defaults(), 0, U64_HIGH);
+        VortexTable table = new VortexTable(file);
+
+        // When / Then — materialization throws instead of yielding a negative BIGINT
+        assertThatThrownBy(() -> drain(table))
+                .isInstanceOf(VortexException.class)
+                .hasMessageContaining("U64 value")
+                .hasMessageContaining("BIGINT");
+    }
+
+    @Test
+    void sum_u32HighHalf_widensExactlyOnFullScan() throws Exception {
+        // Given — zone maps off forces the streaming scanSum path (no per-zone SUM to fold)
+        Path file = write("sum-u32.vortex", noZoneMaps(), (int) U32_HIGH, 1L);
+
+        // When
+        try (VortexReader reader = VortexReader.open(file, registry())) {
+            VortexAggregates.Summary result = VortexAggregates.of(reader, "u32");
+
+            // Then — the U32 element widens via toUnsignedLong, not sign-extension
+            assertThat(result.sumSource()).isEqualTo(VortexAggregates.Source.FULL_SCAN);
+            assertThat(result.sum()).isInstanceOf(Long.class).isEqualTo(U32_HIGH);
+        }
+    }
+
+    @Test
+    void sum_u64HighHalf_failsLoudOnFullScan() throws Exception {
+        // Given — a U64 element the signed accumulator cannot represent, zone maps off
+        Path file = write("sum-u64.vortex", noZoneMaps(), 0, U64_HIGH);
+
+        // When / Then — scanSum throws rather than corrupt the sum with a negative addend
+        try (VortexReader reader = VortexReader.open(file, registry())) {
+            assertThatThrownBy(() -> VortexAggregates.of(reader, "u64"))
+                    .isInstanceOf(VortexException.class)
+                    .hasMessageContaining("U64 value");
+        }
+    }
+
+    private Path write(String name, WriteOptions opts, int u32, long u64) throws Exception {
+        Path file = tmp.resolve(name);
+        try (FileChannel ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE);
+             VortexWriter writer = VortexWriter.create(ch, SCHEMA, opts)) {
+            writer.writeChunk(Map.of(
+                    ColumnName.of("u32"), new int[]{u32},
+                    ColumnName.of("u64"), new long[]{u64}));
+        }
+        return file;
+    }
+
+    private static WriteOptions noZoneMaps() {
+        // Same shape as the adapter coverage test's zone-maps-off options: the second flag disables
+        // zone maps so no per-zone SUM exists and VortexAggregates falls back to scanSum.
+        return new WriteOptions(65_536, false, 0.90, 0, true, false);
+    }
+
+    private static ReadRegistry registry() {
+        return ReadRegistry.builder().registerServiceLoaded().build();
+    }
+
+    private static List<Object[]> drain(VortexTable table) {
+        List<Object[]> rows = new ArrayList<>();
+        Enumerator<Object[]> en = table.scan(null, List.of(), null).enumerator();
+        try {
+            while (en.moveNext()) {
+                rows.add(en.current());
+            }
+        } finally {
+            en.close();
+        }
+        return rows;
+    }
+}

calcite/src/test/java/io/github/dfa1/vortex/calcite/VortexAdapterCoverageTest.java

@@ -82,11 +82,13 @@ void getRowType_mapsEveryColumnToItsSqlType() {
         // Given / When
         RelDataType rowType = new VortexTable(file).getRowType(new JavaTypeFactoryImpl());
 
-        // Then — one SqlTypeName per logical type
+        // Then — one SqlTypeName per logical type. Unsigned integers map to the next wider signed
+        // SQL type so the declared type holds their full range (#216); U64 is the exception (no
+        // wider SQL integer) and stays BIGINT with a value-side overflow guard.
         Map<String, SqlTypeName> expected = Map.ofEntries(
-                Map.entry("i8", SqlTypeName.TINYINT), Map.entry("u8", SqlTypeName.TINYINT),
-                Map.entry("i16", SqlTypeName.SMALLINT), Map.entry("u16", SqlTypeName.SMALLINT),
-                Map.entry("i32", SqlTypeName.INTEGER), Map.entry("u32", SqlTypeName.INTEGER),
+                Map.entry("i8", SqlTypeName.TINYINT), Map.entry("u8", SqlTypeName.SMALLINT),
+                Map.entry("i16", SqlTypeName.SMALLINT), Map.entry("u16", SqlTypeName.INTEGER),
+                Map.entry("i32", SqlTypeName.INTEGER), Map.entry("u32", SqlTypeName.BIGINT),
                 Map.entry("i64", SqlTypeName.BIGINT), Map.entry("u64", SqlTypeName.BIGINT),
                 Map.entry("f32", SqlTypeName.REAL), Map.entry("f64", SqlTypeName.DOUBLE),
                 Map.entry("s", SqlTypeName.VARCHAR), Map.entry("b", SqlTypeName.BOOLEAN));