Skip to content

README.md

Latest commit

 

History

History
94 lines (76 loc) · 4.43 KB

README.md

File metadata and controls

94 lines (76 loc) · 4.43 KB
Raw

native/sea/ — consumer-side directory for the Rust napi binding

The Rust binding source lives in the kernel repo at databricks-sql-kernel/napi/. Building it requires a local checkout of that repo — see "Build for local dev" below. The published npm package is @databricks/databricks-sql-kernel-<triple>.

Workspace topology

The napi crate is a standalone Cargo workspace ([workspace] members = ["."] in napi/Cargo.toml), not a sibling of pyo3/ in the kernel root workspace.

The reason is Cargo feature unification. pyo3 builds the kernel with the default tls-native feature (system OpenSSL via native-tls). The napi crate has to opt INTO tls-rustls instead: napi modules are loaded into Node.js processes that statically link OpenSSL 3.x, and dynamically linking the system's OpenSSL 1.1 (which native-tls pulls in on Linux) collides with Node's symbols at module-load time and segfaults the process before any Rust code runs. rustls is pure Rust + ring and avoids the conflict entirely.

If napi lived in the same workspace as pyo3, cargo build --workspace would unify the kernel's feature set to tls-native ∪ tls-rustls, link both TLS stacks into the resulting napi cdylib, and reintroduce the same clash. Standalone-workspace is the fix.

What lives in this directory

  • index.d.ts — TypeScript declarations consumed by lib/sea/. Generated by napi-rs from the Rust source; checked in as the consumer-facing type contract.
  • index.js — napi-rs's per-platform router shim. Gitignored; populated by npm run build:native for local dev. In published tarballs it ships alongside the .d.ts and require()s the right @databricks/databricks-sql-kernel-<triple> optional dependency.
  • index.*.node — the actual native binary, one per platform. Gitignored. In production these live in the per-triple optional dependencies (@databricks/databricks-sql-kernel-linux-x64-gnu, etc.); for local dev npm run build:native copies one into this directory.

Build for local dev

# From the nodejs repo root:
export DATABRICKS_SQL_KERNEL_REPO=/path/to/your/databricks-sql-kernel
npm run build:native           # release build (default)
BUILD_PROFILE=  npm run build:native  # debug build (empty BUILD_PROFILE drops --release)

DATABRICKS_SQL_KERNEL_REPO points at the kernel repo root (the directory containing napi/) and is required when your kernel checkout isn't at ../../databricks-sql-kernel relative to the nodejs repo.

Production load path

At release time the kernel's CI publishes @databricks/databricks-sql-kernel-<triple> npm packages — one per supported platform — each containing a single .node binary. native/sea/index.js (the napi-rs router) require()s the package matching the consumer's process.platform / process.arch at load time.

M0 status: these per-platform packages are not yet published, so they are intentionally not declared in the driver's optionalDependencies. (npm refuses an npm ci against a pinned dependency it cannot resolve from the registry, so declaring an unpublished package would break every install.) Until they ship, the binding is produced locally via npm run build:native (which copies index.<triple>.node into this directory). Once the packages are published, add @databricks/databricks-sql-kernel-<triple> back to optionalDependencies — npm then installs only the matching one.

Supported platforms (M0)

M0 targets a single triple: linux-x64-gnu (package @databricks/databricks-sql-kernel-linux-x64-gnu, once published).

On every other platform (macOS, Windows, linux-arm64, linux-x64-musl / Alpine, …) the SEA binding is simply absent: SeaNativeLoader returns undefined from tryGet() / throws a structured MODULE_NOT_FOUND hint from get(), and the driver continues to use the Thrift backend exclusively. This is expected, not a regression — additional triples are added to optionalDependencies as the kernel CI starts publishing them in later milestones.

Supply-chain note

The unpublished triple names (@databricks/databricks-sql-kernel-darwin-arm64, …-win32-x64-msvc, etc.) referenced by the router are not squat-able: @databricks is a Databricks-owned npm scope, and npm only allows org members to publish under a scope it owns. A third party therefore cannot register @databricks/databricks-sql-kernel-* and have the router autoload it. No placeholder packages are required.