Skip to content

Commit f258df4

Browse files
committed
feat: add preserveBigNumericPrecision connection option (Thrift + SEA)
The result converter coerces DECIMAL to an IEEE-754 double and BIGINT to a JS number, silently rounding high-precision decimals and integers beyond 2^53. This affects BOTH backends (verified end-to-end on Thrift: CAST('123456789012345.6789' AS DECIMAL(38,4)) -> 123456789012345.67; CAST(9007199254740993 AS BIGINT) -> 9007199254740992). Expose `preserveBigNumericPrecision` as a public ConnectionOption (default false → existing representation preserved, non-breaking). When enabled, DECIMAL is returned as an exact string and BIGINT as a JS `bigint`. Threaded through ClientConfig to both the Thrift (ARROW_BASED / URL_BASED ArrowResultConverter) and SEA operation backends. SeaOperationBackend now reads the option instead of hardcoding it on. Verified end-to-end on Thrift: OFF → 123456789012345.67 / 9007199254740992 (number); ON → "123456789012345.6789" (string) / 9007199254740993 (bigint). Build (tsconfig.build.json) + unit tests + prettier + eslint pass. Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
1 parent 0b75982 commit f258df4

5 files changed

Lines changed: 29 additions & 6 deletions

File tree

lib/DBSQLClient.ts

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -152,6 +152,8 @@ export default class DBSQLClient extends EventEmitter implements IDBSQLClient, I
152152

153153
useLZ4Compression: true,
154154

155+
preserveBigNumericPrecision: false,
156+
155157
// Telemetry defaults are sourced from DEFAULT_TELEMETRY_CONFIG so
156158
// every component reads from the same single frozen const. Mapping the
157159
// unprefixed TelemetryConfiguration keys to the `telemetry`-prefixed
@@ -604,6 +606,11 @@ export default class DBSQLClient extends EventEmitter implements IDBSQLClient, I
604606
this.config.enableMetricViewMetadata = options.enableMetricViewMetadata;
605607
}
606608

609+
// Opt-in: preserve DECIMAL (string) / BIGINT (bigint) precision in results.
610+
if (options.preserveBigNumericPrecision !== undefined) {
611+
this.config.preserveBigNumericPrecision = options.preserveBigNumericPrecision;
612+
}
613+
607614
// Override telemetry config if provided in options. Per-key narrowed copy
608615
// preserves the structural type system: `ConnectionOptions` and
609616
// `ClientConfig` declare identical types for these knobs, so a user

lib/contracts/IClientContext.ts

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,12 @@ export interface ClientConfig {
2626
useLZ4Compression: boolean;
2727
enableMetricViewMetadata?: boolean;
2828

29+
// When true, DECIMAL values are returned as exact strings and 64-bit
30+
// integers as JS `bigint`, instead of being coerced to a lossy `number`.
31+
// Off by default to preserve the long-standing representation on both the
32+
// Thrift and SEA backends. See `ConnectionOptions.preserveBigNumericPrecision`.
33+
preserveBigNumericPrecision?: boolean;
34+
2935
// Telemetry configuration
3036
telemetryEnabled?: boolean;
3137
telemetryBatchSize?: number;

lib/contracts/IDBSQLClient.ts

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,15 @@ export type ConnectionOptions = {
5555
proxy?: ProxyOptions;
5656
enableMetricViewMetadata?: boolean;
5757

58+
/**
59+
* Preserve full numeric precision in results. When `true`, DECIMAL columns
60+
* are returned as exact strings and 64-bit integers (BIGINT) as JS `bigint`,
61+
* instead of the default lossy coercion to a JS `number` (which silently
62+
* rounds DECIMALs and integers beyond 2^53). Applies to both the Thrift and
63+
* SEA backends. Defaults to `false` to preserve the existing representation.
64+
*/
65+
preserveBigNumericPrecision?: boolean;
66+
5867
/**
5968
* Extra HTTP headers attached to driver-owned out-of-band requests
6069
* (telemetry POSTs and feature-flag GETs). Not applied to the primary

lib/sea/SeaOperationBackend.ts

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -767,13 +767,12 @@ export default class SeaOperationBackend implements IOperationBackend {
767767
// SeaResultsProvider consumes only `fetchNextBatch`; both the async result
768768
// handle and the blocking statement satisfy that surface.
769769
this.resultsProvider = new SeaResultsProvider(handle as unknown as SeaStatement);
770-
// The kernel always delivers native Arrow Decimal128 / Int64 (there is no
771-
// server-side "decimals as string" mode like Thrift's
772-
// `useArrowNativeTypes=false`), so preserve their precision here — DECIMAL
773-
// as an exact string, BIGINT as a `bigint` — matching the precise values
774-
// the Thrift backend returns by default.
770+
// DECIMAL/BIGINT precision preservation is opt-in via the
771+
// `preserveBigNumericPrecision` connection option (default off). The kernel
772+
// always delivers native Arrow Decimal128 / Int64, so when enabled the
773+
// converter renders DECIMAL as an exact string and BIGINT as a `bigint`.
775774
const converter = new ArrowResultConverter(this.context, this.resultsProvider, metadata, {
776-
preserveBigNumericPrecision: true,
775+
preserveBigNumericPrecision: this.context.getConfig().preserveBigNumericPrecision ?? false,
777776
});
778777
this.resultSlicer = new ResultSlicer(this.context, converter);
779778
return this.resultSlicer;

lib/thrift-backend/ThriftOperationBackend.ts

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -334,13 +334,15 @@ export default class ThriftOperationBackend implements IOperationBackend {
334334
this.context,
335335
new ArrowResultHandler(this.context, this._data, metadata),
336336
metadata,
337+
{ preserveBigNumericPrecision: this.context.getConfig().preserveBigNumericPrecision ?? false },
337338
);
338339
break;
339340
case TSparkRowSetType.URL_BASED_SET:
340341
resultSource = new ArrowResultConverter(
341342
this.context,
342343
new CloudFetchResultHandler(this.context, this._data, metadata, this.id),
343344
metadata,
345+
{ preserveBigNumericPrecision: this.context.getConfig().preserveBigNumericPrecision ?? false },
344346
);
345347
break;
346348
// no default

0 commit comments

Comments
 (0)