Skip to content

Commit 0328db3

Browse files
feat(client-v2, jdbc-v2): add QBit data type support
QBit(element_type, dimension) is transmitted over RowBinary exactly like Array(element_type) - a var-int length followed by that many element values - so it is read and written as a Java array of its element type (float[] for BFloat16/Float32, double[] for Float64) reusing the existing array codec, while keeping dataType == QBit for metadata fidelity. In jdbc-v2 QBit maps to java.sql.Types.ARRAY and is exposed as a java.sql.Array. Element types: BFloat16, Float32, Float64. The bit-transposed on-disk layout is a server storage detail and is not exposed to the client. Implements: #2610
1 parent 1529baf commit 0328db3

11 files changed

Lines changed: 229 additions & 3 deletions

File tree

CHANGELOG.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,14 @@
1212
`getFloat`/`setFloat` and `getObject` accessors, and reported as such by `ResultSetMetaData` and `DatabaseMetaData`.
1313
Previously reading or writing a `BFloat16` column failed with an
1414
unsupported-data-type error. (https://github.com/ClickHouse/clickhouse-java/issues/2279)
15+
- **[client-v2, jdbc-v2]** Added support for the experimental `QBit(element_type, dimension)` vector data type
16+
(ClickHouse `25.10+`; the `allow_experimental_qbit_type` server setting is required to create a column), with
17+
`BFloat16`, `Float32`, or `Float64` element types. A `QBit` value is transmitted over `RowBinary` exactly like
18+
`Array(element_type)`, so it is read and written as a Java array of the element type (`float[]` for
19+
`BFloat16`/`Float32`, `double[]` for `Float64`) through generic records, binary readers, and POJO binding. In the
20+
JDBC driver (`jdbc-v2`) `QBit` maps to `java.sql.Types.ARRAY` and is returned as a `java.sql.Array` from
21+
`getObject`/`getArray`. Previously `QBit` was an unimplemented type constant and reading or writing such a column
22+
failed. (https://github.com/ClickHouse/clickhouse-java/issues/2610)
1523
- **[client-v2, jdbc-v2]** Added TLS cipher suite selection. `Client.Builder.setSSLCipherSuites(String...)` (client-v2)
1624
and the comma-separated `ssl_cipher_suites` connection property (client-v2 and jdbc-v2) restrict the cipher suites
1725
enabled on secure connections; when unset, the transport defaults are used. Cipher-suite selection is independent of the

clickhouse-data/src/main/java/com/clickhouse/data/ClickHouseColumn.java

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,7 @@ public final class ClickHouseColumn implements Serializable {
7373
private static final String KEYWORD_NESTED = ClickHouseDataType.Nested.name();
7474
private static final String KEYWORD_VARIANT = ClickHouseDataType.Variant.name();
7575
private static final String KEYWORD_JSON = ClickHouseDataType.JSON.name();
76+
private static final String KEYWORD_QBIT = ClickHouseDataType.QBit.name();
7677

7778
private int columnCount;
7879
private int columnIndex;
@@ -146,6 +147,17 @@ private static ClickHouseColumn update(ClickHouseColumn column) {
146147
}
147148
}
148149
break;
150+
case QBit:
151+
// QBit(element_type, dimension) is a one-level array of its element type on the
152+
// wire; the dimension parameter is kept as the column precision.
153+
if (!column.nested.isEmpty()) {
154+
column.arrayLevel = 1;
155+
column.arrayBaseColumn = column.nested.get(0);
156+
}
157+
if (size > 1) {
158+
column.precision = Integer.parseInt(column.parameters.get(1).trim());
159+
}
160+
break;
149161
case Bool:
150162
column.template = ClickHouseBoolValue.ofNull();
151163
break;
@@ -567,6 +579,26 @@ protected static int readColumn(String args, int startIndex, int len, String nam
567579
fixedLength = false;
568580
estimatedLength++;
569581
}
582+
} else if (args.startsWith(KEYWORD_QBIT, i)) {
583+
int index = args.indexOf('(', i + KEYWORD_QBIT.length());
584+
if (index < i) {
585+
throw new IllegalArgumentException(ERROR_MISSING_NESTED_TYPE);
586+
}
587+
List<String> params = new LinkedList<>();
588+
i = ClickHouseUtils.readParameters(args, index, len, params);
589+
if (params.size() < 2) {
590+
throw new IllegalArgumentException(
591+
"QBit requires an element type and a dimension, e.g. QBit(Float32, 8)");
592+
}
593+
// QBit(element_type, dimension) is transmitted over RowBinary exactly like
594+
// Array(element_type): a var-int length followed by that many element values. The
595+
// first parameter is the element type and drives the nested (item) column.
596+
List<ClickHouseColumn> nestedColumns = new LinkedList<>();
597+
nestedColumns.add(ClickHouseColumn.of("", params.get(0)));
598+
column = new ClickHouseColumn(ClickHouseDataType.QBit, name, args.substring(startIndex, i),
599+
nullable, lowCardinality, params, nestedColumns);
600+
fixedLength = false;
601+
estimatedLength++;
570602
}
571603

572604
if (column == null) {

clickhouse-data/src/test/java/com/clickhouse/data/ClickHouseColumnTest.java

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -508,4 +508,33 @@ public Object[][] testJSONBinaryFormat_dp() {
508508
{"JSON(max_dynamic_types=3,max_dynamic_paths=3, SKIP REGEXP '^-.*',SKIP ff, flags Array(Array(Array(Int8))), SKIP alt_count)", 2, Arrays.asList("flags")},
509509
};
510510
}
511+
512+
@DataProvider(name = "qbitTypesProvider")
513+
private static Object[][] qbitTypesProvider() {
514+
return new Object[][] {
515+
{ "QBit(BFloat16, 4)", ClickHouseDataType.BFloat16, 4 },
516+
{ "QBit(Float32, 8)", ClickHouseDataType.Float32, 8 },
517+
{ "QBit(Float64, 1536)", ClickHouseDataType.Float64, 1536 },
518+
};
519+
}
520+
521+
@Test(groups = { "unit" }, dataProvider = "qbitTypesProvider")
522+
public void testParseQBit(String typeName, ClickHouseDataType elementType, int dimension) {
523+
ClickHouseColumn column = ClickHouseColumn.of("vec", typeName);
524+
Assert.assertEquals(column.getDataType(), ClickHouseDataType.QBit);
525+
Assert.assertEquals(column.getOriginalTypeName(), typeName);
526+
// QBit(element_type, dimension) is a one-level array of its element type on the wire.
527+
Assert.assertEquals(column.getArrayNestedLevel(), 1);
528+
Assert.assertNotNull(column.getArrayBaseColumn());
529+
Assert.assertEquals(column.getArrayBaseColumn().getDataType(), elementType);
530+
Assert.assertEquals(column.getNestedColumns().size(), 1);
531+
Assert.assertEquals(column.getNestedColumns().get(0).getDataType(), elementType);
532+
// The fixed vector dimension is retained as the column precision.
533+
Assert.assertEquals(column.getPrecision(), dimension);
534+
}
535+
536+
@Test(groups = { "unit" }, expectedExceptions = IllegalArgumentException.class)
537+
public void testParseQBitRequiresDimension() {
538+
ClickHouseColumn.of("vec", "QBit(Float32)");
539+
}
511540
}

client-v2/src/main/java/com/clickhouse/client/api/data_formats/internal/BinaryStreamReader.java

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -234,6 +234,8 @@ public <T> T readValue(ClickHouseColumn column, Class<?> typeHint) throws IOExce
234234
return (T) readJsonData(input, actualColumn);
235235
}
236236
// case Object: // deprecated https://clickhouse.com/docs/en/sql-reference/data-types/object-data-type
237+
case QBit:
238+
// QBit(element_type, dimension) is transmitted like Array(element_type).
237239
case Array:
238240
if (typeHint == null) { typeHint = arrayDefaultTypeHint;}
239241
return convertArray(readArray(actualColumn), typeHint);

client-v2/src/main/java/com/clickhouse/client/api/data_formats/internal/SerializerUtils.java

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,8 @@ public class SerializerUtils {
6363
public static void serializeData(OutputStream stream, Object value, ClickHouseColumn column) throws IOException {
6464
//Serialize the value to the stream based on the data type
6565
switch (column.getDataType()) {
66+
case QBit:
67+
// QBit(element_type, dimension) is serialized like Array(element_type).
6668
case Array:
6769
serializeArrayData(stream, value, column);
6870
break;

client-v2/src/main/java/com/clickhouse/client/api/internal/DataTypeConverter.java

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,8 @@ public String convertToString(Object value, ClickHouseColumn column) {
6767
case IPv4:
6868
case IPv6:
6969
return ipvToString(value, column);
70+
case QBit:
71+
// QBit is rendered as an array literal of its element type, like Array(element_type).
7072
case Array:
7173
return arrayToString(value, column);
7274
case Point:

client-v2/src/test/java/com/clickhouse/client/datatypes/DataTypeTests.java

Lines changed: 92 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -92,8 +92,17 @@ public void tearDown() {
9292

9393
private <T> void writeReadVerify(String table, String tableDef, Class<T> dtoClass, List<T> data,
9494
BiConsumer<List<T>, T> rowVerifier) throws Exception {
95+
writeReadVerify(table, tableDef, dtoClass, data, rowVerifier, null);
96+
}
97+
98+
private <T> void writeReadVerify(String table, String tableDef, Class<T> dtoClass, List<T> data,
99+
BiConsumer<List<T>, T> rowVerifier, CommandSettings ddlSettings) throws Exception {
95100
client.execute("DROP TABLE IF EXISTS " + table).get();
96-
client.execute(tableDef);
101+
if (ddlSettings == null) {
102+
client.execute(tableDef);
103+
} else {
104+
client.execute(tableDef, ddlSettings).get();
105+
}
97106

98107
final TableSchema tableSchema = client.getTableSchema(table);
99108
client.register(dtoClass, tableSchema);
@@ -107,6 +116,88 @@ private <T> void writeReadVerify(String table, String tableDef, Class<T> dtoClas
107116
Assert.assertEquals(rowCount.get(), data.size());
108117
}
109118

119+
// QBit was introduced in ClickHouse 25.10.
120+
private static final String QBIT_UNSUPPORTED_VERSIONS = "(,25.9]";
121+
122+
@Data
123+
@AllArgsConstructor
124+
@NoArgsConstructor
125+
public static class QBitFloat32DTO {
126+
private long rowId;
127+
private float[] vec;
128+
private int tail;
129+
130+
public static String tblCreateSQL(String table) {
131+
return tableDefinition(table, "rowId Int64", "vec QBit(Float32, 8)", "tail Int32");
132+
}
133+
}
134+
135+
@Data
136+
@AllArgsConstructor
137+
@NoArgsConstructor
138+
public static class QBitFloat64DTO {
139+
private long rowId;
140+
private double[] vec;
141+
private int tail;
142+
143+
public static String tblCreateSQL(String table) {
144+
return tableDefinition(table, "rowId Int64", "vec QBit(Float64, 8)", "tail Int32");
145+
}
146+
}
147+
148+
@Data
149+
@AllArgsConstructor
150+
@NoArgsConstructor
151+
public static class QBitBFloat16DTO {
152+
private long rowId;
153+
private float[] vec;
154+
private int tail;
155+
156+
public static String tblCreateSQL(String table) {
157+
return tableDefinition(table, "rowId Int64", "vec QBit(BFloat16, 8)", "tail Int32");
158+
}
159+
}
160+
161+
@Test(groups = {"integration"})
162+
public void testQBit() throws Exception {
163+
if (isVersionMatch(QBIT_UNSUPPORTED_VERSIONS)) {
164+
throw new SkipException("QBit requires ClickHouse 25.10+");
165+
}
166+
167+
// A QBit(element_type, dimension) value is transmitted over RowBinary exactly like an
168+
// Array(element_type): a var-int length followed by that many element values. The full
169+
// client write -> server -> client read round-trip is verified for every supported
170+
// element type. The trailing Int32 column would shift (and the assertion fail) if the
171+
// QBit codec consumed the wrong number of bytes.
172+
CommandSettings ddl = (CommandSettings) new CommandSettings()
173+
.serverSetting("allow_experimental_qbit_type", "1");
174+
175+
final float[] f32 = {1f, -2f, 3.5f, 4f, 5f, 6f, 7f, 8f};
176+
writeReadVerify("test_qbit_f32", QBitFloat32DTO.tblCreateSQL("test_qbit_f32"),
177+
QBitFloat32DTO.class, Arrays.asList(new QBitFloat32DTO(0, f32, 42)),
178+
(all, dto) -> {
179+
Assert.assertEquals(dto.getVec(), f32);
180+
Assert.assertEquals(dto.getTail(), 42);
181+
}, ddl);
182+
183+
final double[] f64 = {1d, -2d, 3.5d, 4d, 5d, 6d, 7d, 8d};
184+
writeReadVerify("test_qbit_f64", QBitFloat64DTO.tblCreateSQL("test_qbit_f64"),
185+
QBitFloat64DTO.class, Arrays.asList(new QBitFloat64DTO(0, f64, 42)),
186+
(all, dto) -> {
187+
Assert.assertEquals(dto.getVec(), f64);
188+
Assert.assertEquals(dto.getTail(), 42);
189+
}, ddl);
190+
191+
// Integers up to 256 are exactly representable in BFloat16, so these round-trip bit-for-bit.
192+
final float[] bf16 = {1f, 2f, 4f, 8f, 16f, 32f, 64f, 128f};
193+
writeReadVerify("test_qbit_bf16", QBitBFloat16DTO.tblCreateSQL("test_qbit_bf16"),
194+
QBitBFloat16DTO.class, Arrays.asList(new QBitBFloat16DTO(0, bf16, 42)),
195+
(all, dto) -> {
196+
Assert.assertEquals(dto.getVec(), bf16);
197+
Assert.assertEquals(dto.getTail(), 42);
198+
}, ddl);
199+
}
200+
110201
@Test(groups = {"integration"})
111202
public void testNestedDataTypes() throws Exception {
112203
final String table = "test_nested_types";

docs/features.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ This document lists stable, user-visible behavior in `client-v2` and `jdbc-v2` t
2121
- JSONEachRow text reader: Can stream `JSONEachRow` responses through a caller-supplied `JsonParser`, with Jackson and Gson parser factory implementations available as optional classpath dependencies, and infers a best-effort schema from the first row.
2222
- Data type conversion: Maps ClickHouse types to Java values for binary reads, POJO binding, and SQL parameter formatting, including date/time handling.
2323
- BFloat16 type support: For ClickHouse `24.11+`, reads and writes the `BFloat16` type through generic records, binary readers, POJO binding, and `Nullable`/`Dynamic`/`Variant` wrappers. `BFloat16` maps to the Java `float` type: a read widens the stored 16-bit value losslessly, while a write keeps the high 16 bits of the `float`. In `jdbc-v2`, `BFloat16` maps to `java.sql.Types.FLOAT` / `java.lang.Float` and is read and written through the standard `getFloat`/`setFloat` and `getObject` accessors.
24+
- QBit type support: For ClickHouse `25.10+` (requires the `allow_experimental_qbit_type` server setting to create a column), reads and writes the experimental `QBit(element_type, dimension)` vector type, where `element_type` is `BFloat16`, `Float32`, or `Float64`. On the wire a `QBit` value is encoded exactly like `Array(element_type)` (a length-prefixed list of elements), so the client reads and writes it as a Java array of the element type — `float[]` for `BFloat16`/`Float32`, `double[]` for `Float64` — through generic records, binary readers, and POJO binding, using the same code path as `Array(element_type)`. The bit-transposed on-disk layout is a server storage detail and is not exposed to the client.
2425
- Geometry type support: For ClickHouse `25.11+`, where `Geometry` changed from a string alias to `Variant(Point, Ring, LineString, MultiLineString, Polygon, MultiPolygon)`, the client reads and writes `Geometry` values through generic records, binary readers, POJO binding, and SQL parameter formatting, using Java array dimensionality to represent the geometry shape.
2526
- Insert APIs: Supports inserting registered POJOs, raw streams, and callback-driven writers, with optional column lists and format selection.
2627
- Insert controls: Supports insert-specific settings such as deduplication token, query id, compression behavior, and request headers.
@@ -43,6 +44,7 @@ Compatibility-sensitive traits:
4344
- Identifier quoting behavior is stable API for helper callers: identifiers are double-quoted, embedded double quotes are doubled, and optional quoting keeps simple identifiers unchanged.
4445
- Instant formatting is type-sensitive and should not drift: `Date` formatting depends on an explicit timezone, `DateTime` is serialized as epoch seconds, and higher-precision timestamps preserve up to 9 fractional digits.
4546
- `BFloat16` conversion is precision-sensitive and should not drift: a write keeps only the high 16 bits of the `float` (the low mantissa bits are truncated toward zero, matching the server's `Float32``BFloat16` conversion), so values that are not exactly representable in `BFloat16` change when written; a read widens the 16-bit value back to `float` losslessly.
47+
- `QBit` is wire-compatible with `Array(element_type)` and should not drift: the client transmits the logical vector as a length-prefixed array of its element type (`float[]` for `BFloat16`/`Float32`, `double[]` for `Float64`) rather than the server's bit-transposed on-disk layout, so a `QBit(E, N)` value read from or written to the server round-trips as an array of `E`.
4648
- Timezone conversion helpers preserve nanoseconds and can intentionally shift local date or time when interpreted in a different timezone; this behavior is covered by tests and should not be normalized away.
4749
- `Geometry` handling is shape-sensitive: supported values are 1D through 4D Java arrays representing the nested geometry variants, and unsupported shapes or non-array values are rejected during serialization.
4850
- `Geometry` write inference is dimension-based rather than fully type-specific: point, ring/line string, polygon/multi-line string, and multi-polygon are selected from array depth, so writing `Geometry` cannot currently distinguish `Ring` from `LineString` or `Polygon` from `MultiLineString`.
@@ -76,6 +78,7 @@ Compatibility-sensitive traits:
7678
- Database metadata: Implements JDBC `DatabaseMetaData` for ClickHouse catalogs, schemas, tables, columns, and related capability reporting.
7779
- Parameter metadata: Reports prepared-statement parameter counts.
7880
- Type mapping and conversions: Maps ClickHouse types to JDBC types and Java classes, including date/time handling and `java.time` support.
81+
- QBit type mapping: For ClickHouse `25.10+`, JDBC exposes the experimental `QBit(element_type, dimension)` type as `ARRAY`, returning the vector as a `java.sql.Array` of the element type from `getObject()`/`getArray()`. Supported element types are `BFloat16` and `Float32` (both `java.lang.Float`) and `Float64` (`java.lang.Double`). The `allow_experimental_qbit_type` server setting is required to create a `QBit` column.
7982
- Custom result-set type map: `ResultSet#getObject(int|String, Map<String, Class<?>>)` accepts both ClickHouse type names and JDBC `SQLType` names as map keys. Only unwrapped type names are used — `Nullable(...)` and `LowCardinality(...)` wrappers are stripped before lookup, so a key like `"Int32"` matches both `Int32` and `Nullable(Int32)` columns, and keys like `"Nullable(Int32)"` are not recognized. Lookup order is `ClickHouseColumn#getDataType().name()` (e.g. `"Int32"`, `"String"`, `"DateTime"`) then `SQLType.getName()` (e.g. `"INTEGER"`, `"VARCHAR"`, `"TIMESTAMP"`); a missing entry leaves the value uncoerced (read as-is). The feature is supported for primitive ClickHouse types only — `Array`, `Tuple`, `Map`, `Nested`, and geometry types (`Point`, `Ring`, `LineString`, `Polygon`, `MultiPolygon`, `MultiLineString`, `Geometry`) bypass the type map and are returned in their native form.
8083
- Arrays and tuples: Supports JDBC arrays plus ClickHouse tuple values through custom `Array` and `Struct` implementations.
8184
- Geometry type mapping: For ClickHouse `25.11+`, where `Geometry` changed from a string alias to `Variant(Point, Ring, LineString, MultiLineString, Polygon, MultiPolygon)`, JDBC exposes `Geometry` as `ARRAY`, returns nested Java arrays from `getObject()`/`getArray()`, and accepts `Struct` or nested `Array` inputs for prepared-statement inserts depending on the geometry shape.

jdbc-v2/src/main/java/com/clickhouse/jdbc/internal/JdbcUtils.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -117,7 +117,7 @@ private static Map<ClickHouseDataType, SQLType> generateTypeMap() {
117117
map.put(ClickHouseDataType.AggregateFunction, JDBCType.OTHER);
118118
map.put(ClickHouseDataType.Variant, JDBCType.OTHER);
119119
map.put(ClickHouseDataType.Dynamic, JDBCType.OTHER);
120-
map.put(ClickHouseDataType.QBit, JDBCType.OTHER);
120+
map.put(ClickHouseDataType.QBit, JDBCType.ARRAY);
121121

122122

123123
return ImmutableMap.copyOf(map);

jdbc-v2/src/main/java/com/clickhouse/jdbc/types/Array.java

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,9 @@ public class Array implements java.sql.Array {
2727
public Array(ClickHouseColumn column, Object[] elements) throws SQLException {
2828
this.column = column;
2929
this.array = elements;
30-
ClickHouseColumn baseColumn = (this.column.isArray() ? this.column.getArrayBaseColumn() : this.column);
30+
// QBit is not an Array (isArray() == false) but is still a one-level array of its element
31+
// type, so use the array nesting level (set for both Array and QBit) to find the element column.
32+
ClickHouseColumn baseColumn = (this.column.getArrayNestedLevel() > 0 ? this.column.getArrayBaseColumn() : this.column);
3133
this.baseDataType = baseColumn.getDataType();
3234
this.elementTypeName = baseColumn.getOriginalTypeName();
3335
this.type = JdbcUtils.CLICKHOUSE_TO_SQL_TYPE_MAP.getOrDefault(baseDataType, JDBCType.OTHER).getVendorTypeNumber();

0 commit comments

Comments
 (0)