Skip to content

Commit f73e65a

Browse files
msrathore-dbmsrathore-db
committed
ci: cross-OS bug-bash suite for the kernel (SEA) RC
Adds a workflow_dispatch GitHub Actions workflow that runs the kernel bug-bash suite (useKernel:true) against the published RC (@databricks/sql@1.16.0-rc.1) across every supported platform, to confirm the per-platform @databricks/databricks-sql-kernel-<triple> napi binary installs, loads, and works: - linux-x64-gnu (Node 18/20/22), linux-arm64-gnu - darwin-x64, darwin-arm64 - win32-x64-msvc - linux-x64-musl / linux-arm64-musl (Alpine via docker) Vendored under ci/kernel-rc-bugbash/ with its own pinned package.json so it is isolated from the driver's own build. Reuses the repo's existing e2e secrets (DATABRICKS_HOST / TEST_PECO_WAREHOUSE_HTTP_PATH / DATABRICKS_TOKEN). The kernel loader requires Node >= 18, so the matrix omits 14/16. Workspace/warehouse ids are derived from the host/path so the suite is portable across warehouses; the SPOG scenario skips when they can't be derived (non-Azure hosts). Co-authored-by: Isaac
1 parent e448f94 commit f73e65a

21 files changed

Lines changed: 2902 additions & 0 deletions

‎.github/workflows/kernel-rc-cross-os.yml‎

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
name: kernel-rc-cross-os
2+
3+
# Cross-OS / arch / Node validation of the SQL kernel (Rust/SEA) backend against
4+
# the PUBLISHED release candidate (@databricks/sql@1.16.0-rc.1) in kernel mode
5+
# (useKernel:true). Confirms the per-platform @databricks/databricks-sql-kernel-<triple>
6+
# napi binary installs, loads, and runs the full bug-bash suite on each OS.
7+
#
8+
# The kernel loader requires Node >= 18, so the matrix is 18/20/22 (not 14/16).
9+
# Manual trigger only — it runs heavy queries against a real warehouse.
10+
on:
11+
workflow_dispatch:
12+
inputs:
13+
rc_version:
14+
description: '@databricks/sql version to test'
15+
required: true
16+
default: '1.16.0-rc.1'
17+
18+
defaults:
19+
run:
20+
working-directory: ci/kernel-rc-bugbash
21+
22+
# Reuses the repo's existing e2e secrets (same as main.yml):
23+
# DATABRICKS_HOST, TEST_PECO_WAREHOUSE_HTTP_PATH, DATABRICKS_TOKEN
24+
env:
25+
DATABRICKS_PECOTESTING_SERVER_HOSTNAME: ${{ secrets.DATABRICKS_HOST }}
26+
DATABRICKS_PECOTESTING_HTTP_PATH2: ${{ secrets.TEST_PECO_WAREHOUSE_HTTP_PATH }}
27+
DATABRICKS_PECOTESTING_TOKEN: ${{ secrets.DATABRICKS_TOKEN }}
28+
29+
jobs:
30+
# glibc Linux (x64 + arm64), macOS (x64 + arm64), Windows x64.
31+
suite:
32+
name: ${{ matrix.label }} · node ${{ matrix.node }}
33+
runs-on: ${{ matrix.os }}
34+
timeout-minutes: 30
35+
strategy:
36+
fail-fast: false
37+
max-parallel: 3 # cap concurrent load on the single warehouse
38+
matrix:
39+
include:
40+
- { label: linux-x64-gnu, os: ubuntu-latest, node: 18 }
41+
- { label: linux-x64-gnu, os: ubuntu-latest, node: 20 }
42+
- { label: linux-x64-gnu, os: ubuntu-latest, node: 22 }
43+
- { label: linux-arm64-gnu, os: ubuntu-24.04-arm, node: 20 } # needs ARM runner availability
44+
- { label: darwin-x64, os: macos-13, node: 20 }
45+
- { label: darwin-arm64, os: macos-latest, node: 20 }
46+
- { label: win32-x64-msvc, os: windows-latest, node: 20 }
47+
steps:
48+
- uses: actions/checkout@v4
49+
- uses: actions/setup-node@v4
50+
with:
51+
node-version: ${{ matrix.node }}
52+
- name: Platform info
53+
run: node -e "console.log(process.version, process.platform, process.arch)"
54+
- name: Install RC ${{ inputs.rc_version }} (resolves per-platform kernel binary)
55+
run: |
56+
npm install --no-audit --no-fund
57+
npm install --no-audit --no-fund "@databricks/sql@${{ inputs.rc_version }}"
58+
- name: Verify kernel native binary loads
59+
run: node -e "console.log('kernel napi', require('@databricks/sql/native/kernel').version())"
60+
- name: Run full bug-bash suite (kernel mode)
61+
run: node scripts/run-all.js
62+
63+
# musl Linux (Alpine) — run via docker on a glibc runner (a `container:` job
64+
# breaks JS actions: the runner's glibc node can't exec inside musl).
65+
linux-musl:
66+
name: ${{ matrix.label }}
67+
runs-on: ${{ matrix.runner }}
68+
timeout-minutes: 30
69+
strategy:
70+
fail-fast: false
71+
matrix:
72+
include:
73+
- { label: linux-x64-musl, runner: ubuntu-latest }
74+
- { label: linux-arm64-musl, runner: ubuntu-24.04-arm } # needs ARM runner availability
75+
steps:
76+
- uses: actions/checkout@v4
77+
- name: Run suite in node:20-alpine
78+
working-directory: .
79+
run: |
80+
docker run --rm \
81+
-e DATABRICKS_PECOTESTING_SERVER_HOSTNAME \
82+
-e DATABRICKS_PECOTESTING_HTTP_PATH2 \
83+
-e DATABRICKS_PECOTESTING_TOKEN \
84+
-v "$PWD":/work -w /work/ci/kernel-rc-bugbash node:20-alpine \
85+
sh -c "node -e 'console.log(process.version, process.platform, process.arch)' && \
86+
npm install --no-audit --no-fund && \
87+
npm install --no-audit --no-fund '@databricks/sql@${{ inputs.rc_version }}' && \
88+
node -e \"console.log('kernel napi', require('@databricks/sql/native/kernel').version())\" && \
89+
node scripts/run-all.js"

‎ci/kernel-rc-bugbash/README.md‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# Kernel Bug Blitz — Node.js driver (kernel mode)
2+
3+
Runnable scripts for the "Kernel Bug Blitz" CUJ scenarios, exercising
4+
`@databricks/sql@1.16.0-rc.1` in **kernel mode** (`useKernel: true`) against a
5+
real warehouse.
6+
7+
## Setup (clean-room, like a new user)
8+
```bash
9+
cd ~/kernel-bugbash # this folder
10+
npm install @databricks/sql@1.16.0-rc.1 # already installed here
11+
node -e "console.log(require('@databricks/sql/native/kernel').version())" # -> 0.2.0
12+
```
13+
14+
## Credentials (env — source ~/.zshrc)
15+
- `DATABRICKS_PECOTESTING_SERVER_HOSTNAME` — host (e.g. adb-…azuredatabricks.net)
16+
- `DATABRICKS_PECOTESTING_HTTP_PATH2` — `/sql/1.0/warehouses/00adc7b6c00429b8`
17+
- `DATABRICKS_PECOTESTING_TOKEN` — PAT
18+
- `DATABRICKS_PECO_CLIENT_ID_PERSONAL` / `DATABRICKS_PECO_CLIENT_SECRET_PERSONAL` — M2M (the non-`_PERSONAL` pair is rejected `invalid_client`)
19+
20+
## Run
21+
```bash
22+
node scripts/00-smoke.js # SELECT 1 via kernel — run first
23+
node scripts/02-query-execution.js # one category
24+
node scripts/run-all.js # everything, combined results table
25+
```
26+
Each scenario prints `PASS|FAIL|SKIP | <name> | <note> | <ms>`.
27+
28+
## Scripts ↔ sheet categories
29+
| Script | Category |
30+
|---|---|
31+
| `00-smoke.js` | kernel SELECT 1 smoke |
32+
| `01-connection-auth.js` | Connection & Auth (M2M, U2M*, SPOG, proxy*) |
33+
| `02-query-execution.js` | inline, CloudFetch, STATEMENT_TIMEOUT, cancel, concurrent |
34+
| `03-metadata-tags.js` | metadata calls, query tags, metric view |
35+
| `04-data-types.js` | primitives, complex, timestamps, VARIANT, GEO |
36+
| `05-errors-resilience.js` | invalid-query error, retry* |
37+
| `06-cujs.js` | CRUD+switch, large cells, bulk 50k, negative paths |
38+
| `07-lifecycle-session.js` | reuse, multi-op, re-run, double/after-close, async, interleave, session config, UA, query id |
39+
| `08-params-fetch.js` | named/positional params, scalar types, zero-rows, wide row, NULL-heavy, unicode, chunks, multi-statement |
40+
41+
## Skipped / can't-run-here (need infra or manual)
42+
- **OAuth U2M** — interactive browser SSO; run by hand.
43+
- **HTTP forward proxy (Basic auth)** — needs a running proxy.
44+
- **Retry 503/429 + backoff** — needs a fault-injecting proxy.
45+
- **getImportedKeys / getExportedKeys** — not in the Node public API (only `getCrossReference`); expected limitation.
46+
- **Python-only rows** (cursor.rowcount, Arrow/pandas) — N/A for Node.
47+
48+
`lib.js` holds the shared connect/run helpers.

0 commit comments

Comments
 (0)