Skip to content

Commit f88de8c

Browse files
committed
fix(params): correct DATE and large-number parameter type inference
Two driver-side parameter-binding bugs surfaced by the SEA parity suite (the kernel binds (value_str, sql_type) faithfully — the defects are in how DBSQLParameter stringifies/types the value): - A JS `number` that is whole but outside the INT (i32) range was typed INTEGER, so e.g. `1e30` was rejected by the server as `invalid INT literal "1e+30"`. inferNumberType now picks the narrowest fitting type: INTEGER within i32, BIGINT within the safe-integer range, DOUBLE otherwise (and for non-integers). - A JS `Date` bound with an explicit DATE type was stringified with the full ISO-8601 timestamp (`...T00:00:00.000Z`), which the SEA wire rejects as a DATE literal ("trailing input"). It now projects the calendar date (`yyyy-mm-dd`); the no-explicit-type path still binds a Date as a full TIMESTAMP. Both paths are shared by the Thrift and SEA backends; the changes only affect values that the previous logic mis-typed (and which the server rejected), so existing behaviour is preserved. Unit tests added for the magnitude-based integer inference and the DATE projection. Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
1 parent cf1c1bb commit f88de8c

2 files changed

Lines changed: 83 additions & 2 deletions

File tree

lib/DBSQLParameter.ts

Lines changed: 39 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,37 @@ export enum DBSQLParameterType {
2929
INTERVALDAY = 'INTERVAL DAY',
3030
}
3131

32+
// 32-bit signed integer bounds — the range of the Spark `INT` type.
33+
const INT32_MIN = -2147483648;
34+
const INT32_MAX = 2147483647;
35+
36+
/**
37+
* Infer the Spark parameter type for a JS `number` when the caller didn't set
38+
* one explicitly.
39+
*
40+
* A JS `number` is an IEEE-754 double, so a whole-number value can still be far
41+
* outside the `INT` range (e.g. `1e30`). Typing such a value as `INTEGER`
42+
* makes the server reject it (`invalid INT literal "1e+30"`). Pick the
43+
* narrowest type that actually fits:
44+
* - non-integer / non-finite → `DOUBLE`
45+
* - integer within INT (i32) range → `INTEGER`
46+
* - integer within the safe-integer range → `BIGINT`
47+
* - anything larger → `DOUBLE` (can't be represented exactly as an integer
48+
* anyway; callers needing exact 64-bit integers should pass a `bigint`).
49+
*/
50+
function inferNumberType(value: number): DBSQLParameterType {
51+
if (!Number.isInteger(value)) {
52+
return DBSQLParameterType.DOUBLE;
53+
}
54+
if (value >= INT32_MIN && value <= INT32_MAX) {
55+
return DBSQLParameterType.INTEGER;
56+
}
57+
if (Number.isSafeInteger(value)) {
58+
return DBSQLParameterType.BIGINT;
59+
}
60+
return DBSQLParameterType.DOUBLE;
61+
}
62+
3263
interface DBSQLParameterOptions {
3364
type?: DBSQLParameterType;
3465
value: DBSQLParameterValue;
@@ -78,7 +109,7 @@ export class DBSQLParameter {
78109
if (typeof this.value === 'number') {
79110
return new TSparkParameter({
80111
name,
81-
type: wireType ?? (Number.isInteger(this.value) ? DBSQLParameterType.INTEGER : DBSQLParameterType.DOUBLE),
112+
type: wireType ?? inferNumberType(this.value),
82113
value: new TSparkParameterValue({
83114
stringValue: Number(this.value).toString(),
84115
}),
@@ -96,11 +127,17 @@ export class DBSQLParameter {
96127
}
97128

98129
if (this.value instanceof Date) {
130+
// A `Date` bound as `DATE` must project a calendar date (`yyyy-mm-dd`),
131+
// not a full ISO-8601 timestamp: the SEA wire rejects
132+
// `2024-03-14T00:00:00.000Z` as a DATE literal ("trailing input"), and
133+
// Thrift accepts the date-only form just as well. Without an explicit
134+
// DATE type the value still binds as a TIMESTAMP from the full ISO string.
135+
const isDateType = wireType === DBSQLParameterType.DATE;
99136
return new TSparkParameter({
100137
name,
101138
type: wireType ?? DBSQLParameterType.TIMESTAMP,
102139
value: new TSparkParameterValue({
103-
stringValue: this.value.toISOString(),
140+
stringValue: isDateType ? this.value.toISOString().slice(0, 10) : this.value.toISOString(),
104141
}),
105142
});
106143
}

tests/unit/DBSQLParameter.test.ts

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -124,4 +124,48 @@ describe('DBSQLParameter', () => {
124124
}),
125125
);
126126
});
127+
128+
it('infers a fitting integer type by magnitude', () => {
129+
const cases: Array<[number, DBSQLParameterType, string]> = [
130+
// Within INT (i32) range → INTEGER.
131+
[42, DBSQLParameterType.INTEGER, '42'],
132+
[2147483647, DBSQLParameterType.INTEGER, '2147483647'],
133+
[-2147483648, DBSQLParameterType.INTEGER, '-2147483648'],
134+
// Beyond i32 but a safe integer → BIGINT (INTEGER would overflow the
135+
// server's INT literal parse).
136+
[3000000000, DBSQLParameterType.BIGINT, '3000000000'],
137+
// Whole-number double outside the safe-integer range → DOUBLE, not
138+
// INTEGER. Regression: `Number.isInteger(1e30)` is `true`, so this used
139+
// to be typed INTEGER and rejected as `invalid INT literal "1e+30"`.
140+
[1e30, DBSQLParameterType.DOUBLE, '1e+30'],
141+
];
142+
for (const [value, type, stringValue] of cases) {
143+
expect(new DBSQLParameter({ value }).toSparkParameter()).to.deep.equal(
144+
new TSparkParameter({ type, value: new TSparkParameterValue({ stringValue }) }),
145+
);
146+
}
147+
});
148+
149+
it('binds a Date as a calendar date when typed DATE', () => {
150+
// Explicit DATE type → date-only `yyyy-mm-dd`. The full ISO timestamp is
151+
// rejected by the SEA wire as a DATE literal ("trailing input").
152+
expect(
153+
new DBSQLParameter({
154+
type: DBSQLParameterType.DATE,
155+
value: new Date(Date.UTC(2024, 0, 15, 10, 30, 0)),
156+
}).toSparkParameter(),
157+
).to.deep.equal(
158+
new TSparkParameter({
159+
type: DBSQLParameterType.DATE,
160+
value: new TSparkParameterValue({ stringValue: '2024-01-15' }),
161+
}),
162+
);
163+
// Without an explicit type a Date still binds as a full TIMESTAMP.
164+
expect(new DBSQLParameter({ value: new Date('2023-09-06T03:14:27.843Z') }).toSparkParameter()).to.deep.equal(
165+
new TSparkParameter({
166+
type: DBSQLParameterType.TIMESTAMP,
167+
value: new TSparkParameterValue({ stringValue: '2023-09-06T03:14:27.843Z' }),
168+
}),
169+
);
170+
});
127171
});

0 commit comments

Comments
 (0)