Skip to content

Commit f65be0e

Browse files
feat(appkit): infer numeric SQL type for sql.number(), add typed variants (#323)
* feat(appkit): infer numeric SQL type for sql.number(), add typed variants Currently sql.number() unconditionally produces __sql_type: "NUMERIC", which Databricks SQL binds as DECIMAL(10,0). That breaks LIMIT and OFFSET — Spark requires an integer-typed expression and rejects the parameter with INVALID_LIMIT_LIKE_EXPRESSION.DATA_TYPE — and silently truncates non-integer JS numbers (sql.number(3.14) sent value="3.14" into a DECIMAL(10,0) slot). Two changes: 1. sql.number() now infers the wire type from the value: - integer JS number -> BIGINT - non-integer JS number -> DOUBLE - numeric string -> NUMERIC (preserves caller's precision intent; a string is an explicit choice to skip JS-number coercion) Picks the most-precise type the value can losslessly represent. Spark coerces numeric types implicitly so existing queries against BIGINT/DECIMAL/DOUBLE columns keep working, plus LIMIT now accepts sql.number(10) directly. 2. Typed variants for callers who need to override the inference: - sql.int(value) -> INT - sql.bigint(value) -> BIGINT (also accepts JS bigint for values beyond Number.MAX_SAFE_INTEGER) - sql.float(value) -> FLOAT - sql.double(value) -> DOUBLE - sql.decimal(value) -> NUMERIC (decimal precision preserved) sql.int() and sql.bigint() reject non-integer inputs at the helper boundary so the failure surfaces early, before the wire. SQLNumberMarker.__sql_type widens from "NUMERIC" to a union of the numeric SQL types. Non-breaking for callers: any code that previously held an SQLNumberMarker still type-checks (the union is wider in return position). The two unit tests that pinned `type: "NUMERIC"` for sql.number(integer) are updated to expect BIGINT. Discovered while building a parameterized analytics query against LIMIT — the bare `LIMIT :limit` case is the most visible failure but the underlying issue affects any query where the column type matters. Signed-off-by: James Broadhead <jamesbroadhead@gmail.com> * docs: regenerate sql.* API reference for typed numeric variants Co-authored-by: Isaac * fix(shared): address review feedback on typed numeric SQL variants P1 fixes - Reject JS integers outside Number.MAX_SAFE_INTEGER in sql.number(), sql.int(), and sql.bigint(number). The marker would otherwise advertise BIGINT for a value already lost to JS-double precision. - Widen sql.number("10") to BIGINT for integer-shaped strings so handler code that passes req.query strings works with LIMIT/OFFSET. Decimal- shaped strings still emit NUMERIC. - Type-generator: extract INT/BIGINT/TINYINT/SMALLINT/FLOAT/DOUBLE/DECIMAL -- @param annotations, give them sensible defaults, and route each SQL type to its closest typed helper in sqlTypeToHelper. P2 fixes - Reject Infinity / -Infinity / NaN, hex / scientific-only / whitespace strings via a strict NUMERIC_LITERAL_RE. - Emit BIGINT wire values via BigInt(value).toString() so 1e15 -> "1000000000000000" rather than exponent text. - Bound-check sql.int (32-bit signed) and sql.bigint (64-bit signed) inputs; surface a descriptive error pointing to the wider helper. - Narrow each typed helper's return type to the exact __sql_type literal. - Add bound, boundary, error, and precision-loss tests for the typed helpers, plus an integration test for LIMIT :n OFFSET :m bindings. - Add @returns and @example for every new helper; regenerate docs/docs/api/appkit/Variable.sql.md. - Rename sql.decimal -> sql.numeric so name matches the wire literal (existing typed helpers all match name -> wire). Update analytics.md to mention the new helpers and refresh the LIMIT example. Co-authored-by: Isaac Signed-off-by: James Broadhead <jamesbroadhead@gmail.com> * fix(shared): address ACE multi-model review findings Two confirmed issues from a follow-up multi-model review of the prior commit's typed numeric variants: - sql.number(integer-shaped string) widened to BIGINT without bounds-checking. "9223372036854775808" overflowed the 64-bit wire type silently. Now runs the same BIGINT range check as sql.bigint() and throws with a hint to sql.numeric() for arbitrary-precision integers. - The @param extraction regex matched TIMESTAMP_NTZ as TIMESTAMP, losing the NTZ specificity. Reordered the alternation so TIMESTAMP_NTZ wins, added a \b boundary, and gave it a default literal in defaultForType. Plus drive-bys: - Remove stale sql.interval() reference in SQLTypeMarker JSDoc. - Pin behaviour with boundary tests at +/- 2^63 and a regression test that TIMESTAMP_NTZ is not partially matched. Co-authored-by: Isaac Signed-off-by: James Broadhead <jamesbroadhead@gmail.com> * fix(shared): sql.number infers INT (not BIGINT) for LIMIT/OFFSET compatibility Empirical testing against e2-dogfood.staging (from @calvarjorge) showed that Spark's LIMIT/OFFSET operators require IntegerType specifically — LongType/BIGINT is rejected with INVALID_LIMIT_LIKE_EXPRESSION.DATA_TYPE: The offset expression must be integer type, but got "BIGINT". So the original PR's "LIMIT now Just Works" claim was false: BIGINT fails for the exact motivating case. Catalyst auto-widens INT to BIGINT/DECIMAL/DOUBLE for wider columns, so defaulting to INT is strictly better than defaulting to BIGINT. Change sql.number inference: - JS integer in [-2^31, 2^31 - 1] → INT (was BIGINT) - JS integer outside INT but within MAX_SAFE_INTEGER → BIGINT - integer-shaped string in INT range → INT (was BIGINT) - integer-shaped string outside INT, within BIGINT → BIGINT - everything else unchanged (DOUBLE for non-integer, NUMERIC for decimal strings, throw for out-of-BIGINT and non-numeric) Update analytics.md to recommend `-- @param limit INT` and explain the Spark IntegerType requirement. Update unit + integration tests to pin INT bindings for in-range values, with explicit boundary coverage at +/- 2^31. Regenerate the API reference page. Co-authored-by: Isaac Signed-off-by: James Broadhead <jamesbroadhead@gmail.com> * docs(shared): document how to re-validate LIMIT/OFFSET mocked tests against prod The LIMIT/OFFSET integration tests assert wire-type strings produced by sql.number — they don't round-trip against a real warehouse. The original PR claimed BIGINT worked for LIMIT and tests passed, but production Spark rejected the binding. Add a comment in the test block with a copy-pasteable /api/2.0/sql/statements payload, the expected success and failure states, and the exact error string. If Spark's LIMIT type contract ever changes, this comment is the smoke test to catch it before the mocked assertions silently drift away from production behaviour. Verified empirically against two independent SQL Warehouses on e2-dogfood.staging (including dd43ee29fedd958d, the warehouse the original PR cited as evidence): INT succeeds with 2 rows, BIGINT fails with [INVALID_LIMIT_LIKE_EXPRESSION.DATA_TYPE] ... SQLSTATE: 42K0E. Co-authored-by: Isaac Signed-off-by: James Broadhead <jamesbroadhead@gmail.com> --------- Signed-off-by: James Broadhead <jamesbroadhead@gmail.com>
1 parent 7a679d6 commit f65be0e

10 files changed

Lines changed: 927 additions & 63 deletions

File tree

apps/dev-playground/shared/appkit-types/analytics.d.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -270,7 +270,7 @@ declare module "@databricks/appkit-ui/react" {
270270
parameters: {
271271
/** STRING - use sql.string() */
272272
stringParam: SQLStringMarker;
273-
/** NUMERIC - use sql.number() */
273+
/** NUMERIC - use sql.numeric() */
274274
numberParam: SQLNumberMarker;
275275
/** BOOLEAN - use sql.boolean() */
276276
booleanParam: SQLBooleanMarker;

docs/docs/api/appkit/Variable.sql.md

Lines changed: 212 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,25 @@
22

33
```ts
44
const sql: {
5+
bigint: SQLNumberMarker & {
6+
__sql_type: "BIGINT";
7+
};
58
binary: SQLBinaryMarker;
69
boolean: SQLBooleanMarker;
710
date: SQLDateMarker;
11+
double: SQLNumberMarker & {
12+
__sql_type: "DOUBLE";
13+
};
14+
float: SQLNumberMarker & {
15+
__sql_type: "FLOAT";
16+
};
17+
int: SQLNumberMarker & {
18+
__sql_type: "INT";
19+
};
820
number: SQLNumberMarker;
21+
numeric: SQLNumberMarker & {
22+
__sql_type: "NUMERIC";
23+
};
924
string: SQLStringMarker;
1025
timestamp: SQLTimestampMarker;
1126
};
@@ -15,6 +30,43 @@ SQL helper namespace
1530

1631
## Type Declaration
1732

33+
### bigint()
34+
35+
```ts
36+
bigint(value: string | number | bigint): SQLNumberMarker & {
37+
__sql_type: "BIGINT";
38+
};
39+
```
40+
41+
Creates a `BIGINT` (64-bit signed integer) parameter. Accepts JS
42+
`bigint` so callers can round-trip values outside `Number.MAX_SAFE_INTEGER`
43+
without precision loss; for `number` inputs, requires
44+
`Number.isSafeInteger(value)`.
45+
46+
Rejects values outside the signed 64-bit range `[-2^63, 2^63 - 1]`.
47+
48+
#### Parameters
49+
50+
| Parameter | Type | Description |
51+
| ------ | ------ | ------ |
52+
| `value` | `string` \| `number` \| `bigint` | Integer number, bigint, or integer-shaped string |
53+
54+
#### Returns
55+
56+
`SQLNumberMarker` & \{
57+
`__sql_type`: `"BIGINT"`;
58+
\}
59+
60+
Marker pinned to `BIGINT`
61+
62+
#### Example
63+
64+
```typescript
65+
sql.bigint(42); // { __sql_type: "BIGINT", value: "42" }
66+
sql.bigint(9007199254740993n); // { __sql_type: "BIGINT", value: "9007199254740993" }
67+
sql.bigint("9007199254740993"); // { __sql_type: "BIGINT", value: "9007199254740993" }
68+
```
69+
1870
### binary()
1971

2072
```ts
@@ -134,14 +186,133 @@ const params = { startDate: sql.date("2024-01-01") };
134186
params = { startDate: "2024-01-01" }
135187
```
136188

189+
### double()
190+
191+
```ts
192+
double(value: string | number): SQLNumberMarker & {
193+
__sql_type: "DOUBLE";
194+
};
195+
```
196+
197+
Creates a `DOUBLE` (double-precision, 64-bit) parameter. Same precision
198+
as a JS `number`, so `sql.double(value)` is exact for any JS number.
199+
200+
#### Parameters
201+
202+
| Parameter | Type | Description |
203+
| ------ | ------ | ------ |
204+
| `value` | `string` \| `number` | Number or numeric string |
205+
206+
#### Returns
207+
208+
`SQLNumberMarker` & \{
209+
`__sql_type`: `"DOUBLE"`;
210+
\}
211+
212+
Marker pinned to `DOUBLE`
213+
214+
#### Example
215+
216+
```typescript
217+
sql.double(3.14); // { __sql_type: "DOUBLE", value: "3.14" }
218+
```
219+
220+
### float()
221+
222+
```ts
223+
float(value: string | number): SQLNumberMarker & {
224+
__sql_type: "FLOAT";
225+
};
226+
```
227+
228+
Creates a `FLOAT` (single-precision, 32-bit) parameter. Note that JS
229+
numbers are 64-bit doubles, so values may be rounded to fit FLOAT
230+
precision at bind time.
231+
232+
#### Parameters
233+
234+
| Parameter | Type | Description |
235+
| ------ | ------ | ------ |
236+
| `value` | `string` \| `number` | Number or numeric string |
237+
238+
#### Returns
239+
240+
`SQLNumberMarker` & \{
241+
`__sql_type`: `"FLOAT"`;
242+
\}
243+
244+
Marker pinned to `FLOAT`
245+
246+
#### Example
247+
248+
```typescript
249+
sql.float(3.14); // { __sql_type: "FLOAT", value: "3.14" }
250+
```
251+
252+
### int()
253+
254+
```ts
255+
int(value: string | number): SQLNumberMarker & {
256+
__sql_type: "INT";
257+
};
258+
```
259+
260+
Creates an `INT` (32-bit signed integer) parameter. Use when the column
261+
or context requires `INT` specifically (e.g. legacy schemas, or to make
262+
the wire type explicit).
263+
264+
Rejects non-integers, values outside `Number.MAX_SAFE_INTEGER` (for
265+
number inputs), and values outside the signed 32-bit range
266+
`[-2^31, 2^31 - 1]`.
267+
268+
#### Parameters
269+
270+
| Parameter | Type | Description |
271+
| ------ | ------ | ------ |
272+
| `value` | `string` \| `number` | Integer number or integer-shaped string |
273+
274+
#### Returns
275+
276+
`SQLNumberMarker` & \{
277+
`__sql_type`: `"INT"`;
278+
\}
279+
280+
Marker pinned to `INT`
281+
282+
#### Example
283+
284+
```typescript
285+
sql.int(42); // { __sql_type: "INT", value: "42" }
286+
sql.int("42"); // { __sql_type: "INT", value: "42" }
287+
```
288+
137289
### number()
138290

139291
```ts
140292
number(value: string | number): SQLNumberMarker;
141293
```
142294

143-
Creates a NUMERIC type parameter
144-
Accepts numbers or numeric strings
295+
Creates a numeric type parameter. The wire SQL type is inferred from the
296+
value so the parameter binds correctly in any context, including `LIMIT`
297+
and `OFFSET`:
298+
299+
- JS integer in `[-2^31, 2^31 - 1]``INT`
300+
- JS integer outside `INT` but within `Number.MAX_SAFE_INTEGER``BIGINT`
301+
- JS non-integer (`3.14`) → `DOUBLE`
302+
- integer-shaped string in `INT` range → `INT` (common HTTP-input case)
303+
- integer-shaped string outside `INT` but within `BIGINT``BIGINT`
304+
- decimal-shaped string (`"123.45"`) → `NUMERIC` (preserves precision)
305+
306+
Why default to `INT`? Spark's `LIMIT` and `OFFSET` operators require
307+
`IntegerType` specifically — `BIGINT` (`LongType`) is rejected with
308+
`INVALID_LIMIT_LIKE_EXPRESSION.DATA_TYPE`. Catalyst auto-widens `INT`
309+
to `BIGINT` / `DECIMAL` / `DOUBLE` for wider columns, so `INT` is a
310+
strictly better default than `BIGINT`.
311+
312+
Throws on `NaN`, `Infinity`, JS integers outside `Number.MAX_SAFE_INTEGER`,
313+
integer-shaped strings outside the `BIGINT` range, or non-numeric strings.
314+
Reach for `sql.int()`, `sql.bigint()`, `sql.float()`, `sql.double()`, or
315+
`sql.numeric()` to override the inferred type.
145316

146317
#### Parameters
147318

@@ -153,18 +324,51 @@ Accepts numbers or numeric strings
153324

154325
`SQLNumberMarker`
155326

156-
Marker object for NUMERIC type parameter
327+
Marker for a numeric SQL parameter
157328

158-
#### Examples
329+
#### Example
159330

160331
```typescript
161-
const params = { userId: sql.number(123) };
162-
params = { userId: "123" }
332+
sql.number(123); // { __sql_type: "INT", value: "123" }
333+
sql.number(3_000_000_000); // { __sql_type: "BIGINT", value: "3000000000" }
334+
sql.number(0.5); // { __sql_type: "DOUBLE", value: "0.5" }
335+
sql.number("10"); // { __sql_type: "INT", value: "10" }
336+
sql.number("123.45"); // { __sql_type: "NUMERIC", value: "123.45" }
337+
```
338+
339+
### numeric()
340+
341+
```ts
342+
numeric(value: string | number): SQLNumberMarker & {
343+
__sql_type: "NUMERIC";
344+
};
163345
```
164346

347+
Creates a `NUMERIC` (fixed-point DECIMAL) parameter. Use when you need
348+
exact decimal arithmetic (currency, percentages) — pass values as
349+
strings to avoid JS-number precision loss.
350+
351+
Note: passing a JS `number` is accepted but lossy for many values
352+
(e.g. `0.1 + 0.2``"0.30000000000000004"`). Prefer strings.
353+
354+
#### Parameters
355+
356+
| Parameter | Type | Description |
357+
| ------ | ------ | ------ |
358+
| `value` | `string` \| `number` | Number or numeric string (strings preferred for precision) |
359+
360+
#### Returns
361+
362+
`SQLNumberMarker` & \{
363+
`__sql_type`: `"NUMERIC"`;
364+
\}
365+
366+
Marker pinned to `NUMERIC`
367+
368+
#### Example
369+
165370
```typescript
166-
const params = { userId: sql.number("123") };
167-
params = { userId: "123" }
371+
sql.numeric("12345.6789"); // { __sql_type: "NUMERIC", value: "12345.6789" }
168372
```
169373

170374
### string()

docs/docs/plugins/analytics.md

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -43,14 +43,23 @@ Use `:paramName` placeholders and optionally annotate parameter types using SQL
4343
```sql
4444
-- @param startDate DATE
4545
-- @param endDate DATE
46-
-- @param limit NUMERIC
46+
-- @param limit INT
4747
SELECT ...
4848
WHERE usage_date BETWEEN :startDate AND :endDate
4949
LIMIT :limit
5050
```
5151

52+
`LIMIT` / `OFFSET` require Spark `IntegerType` specifically — `BIGINT`
53+
(`LongType`) is rejected with `INVALID_LIMIT_LIKE_EXPRESSION.DATA_TYPE`.
54+
Annotate with `INT`, or use `sql.number()` (auto-infers `INT` for values in
55+
`[-2^31, 2^31-1]`, falling back to `BIGINT` for wider values) / `sql.int()`
56+
at the call site.
57+
5258
**Supported `-- @param` types** (case-insensitive):
53-
- `STRING`, `NUMERIC`, `BOOLEAN`, `DATE`, `TIMESTAMP`, `BINARY`
59+
- `STRING`, `BOOLEAN`, `DATE`, `TIMESTAMP`, `BINARY`
60+
- `INT`, `BIGINT`, `TINYINT`, `SMALLINT` — bind via `sql.int()` / `sql.bigint()`
61+
- `FLOAT`, `DOUBLE` — bind via `sql.float()` / `sql.double()`
62+
- `NUMERIC`, `DECIMAL` — bind via `sql.numeric()` (pass strings for precision)
5463

5564
## Server-injected parameters
5665

0 commit comments

Comments
 (0)