Skip to content

Commit 93dee9d

Browse files
committed
refactor: centralize type logic via HyperType
Introduce a Hyper-native internal type model (HyperType) in a new com.salesforce.datacloud.jdbc.core.types package and route every Arrow, pg_catalog, prepared-statement, and Spark conversion through it. Translation to java.sql.JDBCType / Types only happens at the JDBC API boundary via HyperTypes. New types package: - HyperTypeKind enum (BOOL, INT8-INT64, OID, FLOAT4/8, DECIMAL, CHAR, VARCHAR, BINARY, VARBINARY, DATE, TIME[_TZ], TIMESTAMP[_TZ], ARRAY, NULL, INTERVAL, JSON) - HyperType @value class with factories per kind and a visitor - HyperTypes utility: toJdbcType/Code/Name, toJavaClass, getPrecision, getScale, getDisplaySize, isSigned, isCaseSensitive, needsDecimalDigits/CharOctetLength, fromJdbcTypeCode, typeInfoRows - ArrowToHyperTypeMapper: Arrow Field -> HyperType (reads hyper:type / hyper:max_string_length metadata) - HyperTypeToArrow: HyperType -> Arrow FieldType for prepared-statement parameter schemas - PgCatalogTypeParser: Hyper format_type() string -> HyperType, throws IllegalArgumentException for any unrecognized name (no UNKNOWN catch-all; unsupported types fail loudly at the JDBC boundary) Deleted: - ColumnType.java, ArrowToColumnTypeMapper.java (replaced by HyperType and ArrowToHyperTypeMapper) - QueryMetadataUtil.dbTypeToSql map (which used PG internal aliases int2/int4/int8/bool/float4/float8 that Hyper's format_type never emits) and the isDecimalType/isCharType string helpers - ArrowUtils.SQL_TYPE_TO_FIELD_TYPE map (replaced by HyperTypeToArrow) Migrated callers: - ColumnMetadata holds HyperType - DataCloudResultSetMetaData delegates to HyperTypes.* - StreamingResultSet, DataCloudArray, DataCloudConnection.getSchemaForQueryId, ArrowUtils.toColumnMetaData route through ArrowToHyperTypeMapper - QueryMetadataUtil.constructColumnData routes through PgCatalogTypeParser - SimpleResultSet switches from JDBCType to HyperTypeKind; also picks up TINYINT numeric coercions and the getFloat wasNull/range-check fix that were planned as Phase 0 work - ParameterBinding holds HyperType; DataCloudPreparedStatement setXxx methods build HyperType factories directly - VectorPopulator.VectorValueSetterFactory keyed by HyperTypeKind - TypeMapping.scala (Spark): switches expanded to cover TINYINT, NUMERIC, CHAR, NULL, FLOAT, NCHAR, NVARCHAR, LONGNVARCHAR - DataCloudDatabaseMetadata.getTypeInfo() returns a real 18-column result derived from HyperTypeKind (was empty before) Observable behaviour changes in DatabaseMetaData.getColumns(): - TYPE_NAME is now consistently JDBC-uppercase ("BIGINT", "BOOLEAN", "INTEGER", "DECIMAL", ...) instead of a mix of Hyper-lowercase and uppercase strings. INTERVAL and JSON show the Hyper-uppercase form. - DATA_TYPE is always a java.sql.Types value (was a raw PG OID for types unmapped by the old dbTypeToSql). - Numeric/decimal unified to DECIMAL (both Arrow Decimal and Hyper numeric map to HyperTypeKind.DECIMAL / JDBCType.DECIMAL). - Unsupported column types throw SQLException (state 42704) rather than silently surfacing as Types.OTHER with a raw name. Exception contract: - PgCatalogTypeParser, HyperTypes.fromJdbcTypeCode, HyperTypeToArrow, and ArrowToHyperTypeMapper throw IllegalArgumentException for types they do not model. Every JDBC entry point that reaches them (DataCloudDatabaseMetadata.getColumns, DataCloudConnection.getSchemaForQueryId, DataCloudPreparedStatement.setNull / setObject / getQueryParamBuilder, StreamingResultSet.of) catches and wraps in SQLException with an appropriate SQL state (42704 / HYC00 / HY000 / 0A000). Also fixes pre-existing DataCloudDatabaseMetadataTest expectations that hadn't been updated when the recent "DatabaseMetaData spec violations" commit landed (e.g. testGetJDBCMajorVersion expecting 1 rather than 4, testGetProcedures expecting .isNull() on a method now throwing SQLFeatureNotSupportedException). Made-with: Cursor
1 parent d4b1dcf commit 93dee9d

46 files changed

Lines changed: 2834 additions & 1477 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

jdbc-core/src/main/java/com/salesforce/datacloud/jdbc/core/ColumnNameResolver.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
*/
55
package com.salesforce.datacloud.jdbc.core;
66

7-
import com.salesforce.datacloud.jdbc.core.metadata.ColumnMetadata;
7+
import com.salesforce.datacloud.jdbc.protocol.data.ColumnMetadata;
88
import java.sql.SQLException;
99
import java.util.HashMap;
1010
import java.util.List;

jdbc-core/src/main/java/com/salesforce/datacloud/jdbc/core/DataCloudConnection.java

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,10 +7,9 @@
77
import static com.salesforce.datacloud.jdbc.config.DriverVersion.formatDriverInfo;
88
import static com.salesforce.datacloud.jdbc.exception.QueryExceptionHandler.createException;
99
import static com.salesforce.datacloud.jdbc.logging.ElapsedLogger.logTimedValue;
10-
import static com.salesforce.datacloud.jdbc.util.ArrowUtils.toColumnMetaData;
10+
import static com.salesforce.datacloud.jdbc.protocol.data.ArrowUtils.toColumnMetaData;
1111

1212
import com.google.protobuf.Empty;
13-
import com.salesforce.datacloud.jdbc.core.metadata.ColumnMetadata;
1413
import com.salesforce.datacloud.jdbc.core.metadata.DataCloudResultSetMetaData;
1514
import com.salesforce.datacloud.jdbc.core.partial.DataCloudQueryPolling;
1615
import com.salesforce.datacloud.jdbc.exception.DataCloudJDBCException;
@@ -22,6 +21,8 @@
2221
import com.salesforce.datacloud.jdbc.protocol.RowRangeIterator;
2322
import com.salesforce.datacloud.jdbc.protocol.async.core.AsyncStreamObserverIterator;
2423
import com.salesforce.datacloud.jdbc.protocol.async.core.SyncIteratorAdapter;
24+
import com.salesforce.datacloud.jdbc.protocol.data.ColumnMetadata;
25+
import com.salesforce.datacloud.jdbc.protocol.data.DefaultParameterAccumulator;
2526
import com.salesforce.datacloud.jdbc.protocol.grpc.QueryAccessGrpcClient;
2627
import com.salesforce.datacloud.jdbc.util.Deadline;
2728
import com.salesforce.datacloud.jdbc.util.JdbcURL;
@@ -182,7 +183,7 @@ public PreparedStatement prepareStatement(String sql) {
182183
}
183184

184185
private DataCloudPreparedStatement getQueryPreparedStatement(String sql) {
185-
return new DataCloudPreparedStatement(this, sql, new DefaultParameterManager());
186+
return new DataCloudPreparedStatement(this, sql, new DefaultParameterAccumulator());
186187
}
187188

188189
/**
@@ -281,6 +282,12 @@ public ResultSetMetaData getSchemaForQueryId(String queryId) throws SQLException
281282
return new DataCloudResultSetMetaData(columns);
282283
} catch (StatusRuntimeException ex) {
283284
throw createException(connectionProperties.isIncludeCustomerDetailInReason(), null, queryId, ex);
285+
} catch (IllegalArgumentException ex) {
286+
// Thrown by ArrowToHyperTypeMapper for Arrow types the driver does not model.
287+
throw new SQLException(
288+
"Unsupported column type in query schema for queryId=" + queryId + ": " + ex.getMessage(),
289+
"0A000",
290+
ex);
284291
}
285292
}
286293

jdbc-core/src/main/java/com/salesforce/datacloud/jdbc/core/DataCloudDatabaseMetadata.java

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,8 @@
1313

1414
import com.google.common.collect.ImmutableList;
1515
import com.salesforce.datacloud.jdbc.config.DriverVersion;
16+
import com.salesforce.datacloud.jdbc.core.metadata.DataCloudResultSetMetaData;
17+
import com.salesforce.datacloud.jdbc.core.types.HyperTypes;
1618
import com.salesforce.datacloud.jdbc.util.JdbcURL;
1719
import com.salesforce.datacloud.jdbc.util.ThrowingJdbcSupplier;
1820
import java.sql.Connection;
@@ -753,7 +755,8 @@ public ResultSet getCrossReference(
753755

754756
@Override
755757
public ResultSet getTypeInfo() throws SQLException {
756-
return DataCloudMetadataResultSet.empty();
758+
return DataCloudMetadataResultSet.of(
759+
new DataCloudResultSetMetaData(MetadataSchemas.TYPE_INFO), HyperTypes.typeInfoRows());
757760
}
758761

759762
@Override

jdbc-core/src/main/java/com/salesforce/datacloud/jdbc/core/DataCloudMetadataResultSet.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,10 @@
44
*/
55
package com.salesforce.datacloud.jdbc.core;
66

7-
import com.salesforce.datacloud.jdbc.core.metadata.ColumnMetadata;
87
import com.salesforce.datacloud.jdbc.core.metadata.DataCloudResultSetMetaData;
98
import com.salesforce.datacloud.jdbc.core.resultset.ColumnAccessor;
109
import com.salesforce.datacloud.jdbc.core.resultset.SimpleResultSet;
10+
import com.salesforce.datacloud.jdbc.protocol.data.ColumnMetadata;
1111
import java.sql.SQLException;
1212
import java.sql.Statement;
1313
import java.util.Collections;

jdbc-core/src/main/java/com/salesforce/datacloud/jdbc/core/DataCloudPreparedStatement.java

Lines changed: 44 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -4,13 +4,15 @@
44
*/
55
package com.salesforce.datacloud.jdbc.core;
66

7-
import static com.salesforce.datacloud.jdbc.util.ArrowUtils.toArrowByteArray;
7+
import static com.salesforce.datacloud.jdbc.protocol.data.ArrowUtils.toArrowByteArray;
88
import static com.salesforce.datacloud.jdbc.util.DateTimeUtils.getUTCDateFromDateAndCalendar;
99
import static com.salesforce.datacloud.jdbc.util.DateTimeUtils.getUTCTimeFromTimeAndCalendar;
1010

1111
import com.google.common.collect.ImmutableMap;
1212
import com.google.common.collect.Maps;
1313
import com.google.protobuf.ByteString;
14+
import com.salesforce.datacloud.jdbc.protocol.data.HyperType;
15+
import com.salesforce.datacloud.jdbc.protocol.data.ParameterAccumulator;
1416
import com.salesforce.datacloud.jdbc.util.QueryTimeout;
1517
import com.salesforce.datacloud.jdbc.util.SqlErrorCodes;
1618
import java.io.IOException;
@@ -51,25 +53,29 @@
5153
@Slf4j
5254
public class DataCloudPreparedStatement extends DataCloudStatement implements PreparedStatement {
5355
private String sql;
54-
private final ParameterManager parameterManager;
56+
private final ParameterAccumulator parameterManager;
5557
private final Calendar calendar = Calendar.getInstance(TimeZone.getTimeZone("UTC"));
5658
// True if we are currently fetching metadata from the server, this influences the query param generation
5759
// to not return any data.
5860
private boolean fetchingMetadata = false;
5961

60-
DataCloudPreparedStatement(DataCloudConnection connection, ParameterManager parameterManager) {
62+
DataCloudPreparedStatement(DataCloudConnection connection, ParameterAccumulator parameterManager) {
6163
super(connection);
6264
this.parameterManager = parameterManager;
6365
}
6466

65-
DataCloudPreparedStatement(DataCloudConnection connection, String sql, ParameterManager parameterManager) {
67+
DataCloudPreparedStatement(DataCloudConnection connection, String sql, ParameterAccumulator parameterManager) {
6668
super(connection);
6769
this.sql = sql;
6870
this.parameterManager = parameterManager;
6971
}
7072

71-
private <T> void setParameter(int parameterIndex, int sqlType, T value) throws SQLException {
72-
parameterManager.setParameter(parameterIndex, sqlType, value);
73+
private <T> void setParameter(int parameterIndex, HyperType type, T value) throws SQLException {
74+
try {
75+
parameterManager.setParameter(parameterIndex, type, value);
76+
} catch (IllegalArgumentException ex) {
77+
throw new SQLException(ex.getMessage(), ex);
78+
}
7379
}
7480

7581
@Override
@@ -94,6 +100,10 @@ protected QueryParam.Builder getQueryParamBuilder(
94100
encodedRow = toArrowByteArray(parameterManager.getParameters(), calendar);
95101
} catch (IOException e) {
96102
throw new SQLException("Failed to encode parameters on prepared statement", e);
103+
} catch (IllegalArgumentException e) {
104+
// Thrown when a parameter is bound with a HyperType we cannot currently encode as
105+
// Arrow (e.g. INTERVAL, JSON). Surface as a JDBC spec-compliant SQLException.
106+
throw new SQLException("Failed to encode parameters on prepared statement: " + e.getMessage(), "HY000", e);
97107
}
98108

99109
if (fetchingMetadata) {
@@ -131,52 +141,61 @@ public int executeUpdate() throws SQLException {
131141

132142
@Override
133143
public void setNull(int parameterIndex, int sqlType) throws SQLException {
134-
setParameter(parameterIndex, sqlType, null);
144+
setParameter(parameterIndex, hyperTypeForJdbcCode(sqlType), null);
145+
}
146+
147+
private static HyperType hyperTypeForJdbcCode(int sqlType) throws SQLException {
148+
try {
149+
return com.salesforce.datacloud.jdbc.core.types.HyperTypes.fromJdbcTypeCode(sqlType, true);
150+
} catch (IllegalArgumentException ex) {
151+
throw new SQLException("Unsupported JDBC type code: " + sqlType, "HYC00", ex);
152+
}
135153
}
136154

137155
@Override
138156
public void setBoolean(int parameterIndex, boolean x) throws SQLException {
139-
setParameter(parameterIndex, Types.BOOLEAN, x);
157+
setParameter(parameterIndex, HyperType.bool(false), x);
140158
}
141159

142160
@Override
143161
public void setByte(int parameterIndex, byte x) throws SQLException {
144-
setParameter(parameterIndex, Types.TINYINT, x);
162+
setParameter(parameterIndex, HyperType.int8(false), x);
145163
}
146164

147165
@Override
148166
public void setShort(int parameterIndex, short x) throws SQLException {
149-
setParameter(parameterIndex, Types.SMALLINT, x);
167+
setParameter(parameterIndex, HyperType.int16(false), x);
150168
}
151169

152170
@Override
153171
public void setInt(int parameterIndex, int x) throws SQLException {
154-
setParameter(parameterIndex, Types.INTEGER, x);
172+
setParameter(parameterIndex, HyperType.int32(false), x);
155173
}
156174

157175
@Override
158176
public void setLong(int parameterIndex, long x) throws SQLException {
159-
setParameter(parameterIndex, Types.BIGINT, x);
177+
setParameter(parameterIndex, HyperType.int64(false), x);
160178
}
161179

162180
@Override
163181
public void setFloat(int parameterIndex, float x) throws SQLException {
164-
setParameter(parameterIndex, Types.FLOAT, x);
182+
setParameter(parameterIndex, HyperType.float8(false), x);
165183
}
166184

167185
@Override
168186
public void setDouble(int parameterIndex, double x) throws SQLException {
169-
setParameter(parameterIndex, Types.DOUBLE, x);
187+
setParameter(parameterIndex, HyperType.float8(false), x);
170188
}
171189

172190
@Override
173191
public void setBigDecimal(int parameterIndex, BigDecimal x) throws SQLException {
174-
setParameter(parameterIndex, Types.DECIMAL, x);
192+
HyperType type = x != null ? HyperType.decimal(x.precision(), x.scale(), true) : HyperType.decimal(0, 0, true);
193+
setParameter(parameterIndex, type, x);
175194
}
176195

177196
@Override
178197
public void setString(int parameterIndex, String x) throws SQLException {
179-
setParameter(parameterIndex, Types.VARCHAR, x);
198+
setParameter(parameterIndex, HyperType.varcharUnlimited(true), x);
180199
}
181200

182201
@Override
@@ -186,17 +205,17 @@ public void setBytes(int parameterIndex, byte[] x) throws SQLException {
186205

187206
@Override
188207
public void setDate(int parameterIndex, Date x) throws SQLException {
189-
setParameter(parameterIndex, Types.DATE, x);
208+
setParameter(parameterIndex, HyperType.date(true), x);
190209
}
191210

192211
@Override
193212
public void setTime(int parameterIndex, Time x) throws SQLException {
194-
setParameter(parameterIndex, Types.TIME, x);
213+
setParameter(parameterIndex, HyperType.time(true), x);
195214
}
196215

197216
@Override
198217
public void setTimestamp(int parameterIndex, Timestamp x) throws SQLException {
199-
setParameter(parameterIndex, Types.TIMESTAMP, toWallClockAsUtc(x, null));
218+
setParameter(parameterIndex, HyperType.timestamp(true), toWallClockAsUtc(x, null));
200219
}
201220

202221
@Override
@@ -229,17 +248,17 @@ public void setObject(int parameterIndex, Object x, int targetSqlType) throws SQ
229248
// or store LocalDateTime digits directly.
230249
if (targetSqlType == Types.TIMESTAMP) {
231250
if (x instanceof Timestamp) {
232-
setParameter(parameterIndex, Types.TIMESTAMP, toWallClockAsUtc((Timestamp) x, null));
251+
setParameter(parameterIndex, HyperType.timestamp(true), toWallClockAsUtc((Timestamp) x, null));
233252
return;
234253
}
235254
if (x instanceof LocalDateTime) {
236255
LocalDateTime ldt = (LocalDateTime) x;
237256
// Encode LDT digits as if they were in UTC — no JVM-TZ shift applied.
238-
setParameter(parameterIndex, Types.TIMESTAMP, Timestamp.from(ldt.toInstant(ZoneOffset.UTC)));
257+
setParameter(parameterIndex, HyperType.timestamp(true), Timestamp.from(ldt.toInstant(ZoneOffset.UTC)));
239258
return;
240259
}
241260
}
242-
setParameter(parameterIndex, targetSqlType, x);
261+
setParameter(parameterIndex, hyperTypeForJdbcCode(targetSqlType), x);
243262
}
244263

245264
@Override
@@ -307,18 +326,18 @@ public ResultSetMetaData getMetaData() throws SQLException {
307326
@Override
308327
public void setDate(int parameterIndex, Date x, Calendar cal) throws SQLException {
309328
val utcDate = getUTCDateFromDateAndCalendar(x, cal);
310-
setParameter(parameterIndex, Types.DATE, utcDate);
329+
setParameter(parameterIndex, HyperType.date(true), utcDate);
311330
}
312331

313332
@Override
314333
public void setTime(int parameterIndex, Time x, Calendar cal) throws SQLException {
315334
val utcTime = getUTCTimeFromTimeAndCalendar(x, cal);
316-
setParameter(parameterIndex, Types.TIME, utcTime);
335+
setParameter(parameterIndex, HyperType.time(true), utcTime);
317336
}
318337

319338
@Override
320339
public void setTimestamp(int parameterIndex, Timestamp x, Calendar cal) throws SQLException {
321-
setParameter(parameterIndex, Types.TIMESTAMP, toWallClockAsUtc(x, cal));
340+
setParameter(parameterIndex, HyperType.timestamp(true), toWallClockAsUtc(x, cal));
322341
}
323342

324343
/**

0 commit comments

Comments
 (0)