Skip to content

Commit 26acafe

Browse files
committed
[SEA-NodeJS] Unify kernel + driver logging into one DBSQLLogger sink
The Rust kernel emits its diagnostics via `tracing`; in a Node process nothing subscribed to them, so kernel logs were silently dropped — the `DBSQLLogger` only ever saw driver-side lines. This wires the kernel into the SAME logger (and file) the driver uses, so logs from all three layers — driver, napi shim, kernel — land in one place. How: - `SeaBackend.connect()` calls `installKernelLogBridge(binding, logger, level)` (new `lib/sea/SeaLogging.ts`), which registers a JS callback with the binding's `initKernelLogging`. The callback maps each kernel `LogRecord` (`{ level, target, message }`) onto the driver's `LogLevel` and re-emits it through `IClientContext.getLogger()`, tagged `[kernel <target>] …`. - Kernel verbosity follows the driver logger's level: `IDBSQLLogger` gains an optional `getLevel()` (implemented by `DBSQLLogger`); the bridge passes it to the kernel so events the sink would drop never cross the bridge. Defaults to `info` for loggers that don't expose a level. - Process-global, last-writer-wins + graceful no-op on an older binding without the bridge export (logging is advisory) — see SeaLogging docs. Requires kernel napi `initKernelLogging`/`setKernelLogLevel`/`LogRecord` (databricks-sql-kernel#126); `KERNEL_REV` bumped + binding rebuilt so the generated `native/sea/index.d.ts` carries them. Verified: tsc clean, eslint + prettier clean, SEA unit suite 256 passing (incl. new SeaLogging unit tests), and a live end-to-end run confirms a single DBSQLLogger file holds both driver lines and 29 `[kernel databricks::sql::kernel]` lines for one `SELECT 1` (committed as tests/e2e/sea/logging-e2e.test.ts). Co-authored-by: Isaac Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
1 parent 4b9e16e commit 26acafe

10 files changed

Lines changed: 377 additions & 2 deletions

File tree

KERNEL_REV

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
8bedaabf69f5bce5a957a8775f29dbb8dbdd2e71
1+
99d6ffc8cb5165d304cd9ab8f57649e885493438

lib/DBSQLLogger.ts

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,10 @@ export default class DBSQLLogger implements IDBSQLLogger {
2626
this.logger.log({ level, message });
2727
}
2828

29+
getLevel(): LogLevel {
30+
return (this.transports.console.level as LogLevel) ?? LogLevel.info;
31+
}
32+
2933
setLevel(level: LogLevel) {
3034
this.transports.console.level = level;
3135
if (this.transports.file) {

lib/contracts/IDBSQLLogger.ts

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,15 @@ export interface LoggerOptions {
55

66
export default interface IDBSQLLogger {
77
log(level: LogLevel, message: string): void;
8+
9+
/**
10+
* Optional: the logger's current level. When implemented, the SEA/kernel
11+
* backend uses it to set the verbosity of the kernel-side (Rust) log bridge,
12+
* so kernel logs are filtered at the same level as the driver's own logs and
13+
* land in the same sink. Loggers that don't implement it leave the kernel
14+
* bridge at its `info` default.
15+
*/
16+
getLevel?(): LogLevel;
817
}
918

1019
export enum LogLevel {

lib/sea/SeaBackend.ts

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ import HiveDriverError from '../errors/HiveDriverError';
2222
import { getSeaNative, SeaNativeBinding, SeaConnection } from './SeaNativeLoader';
2323
import { decodeNapiKernelError } from './SeaErrorMapping';
2424
import { buildSeaConnectionOptions, SeaNativeConnectionOptions } from './SeaAuth';
25+
import { installKernelLogBridge } from './SeaLogging';
2526
import SeaSessionBackend from './SeaSessionBackend';
2627

2728
export interface SeaBackendOptions {
@@ -81,6 +82,16 @@ export default class SeaBackend implements IBackend {
8182
// we ever touch the native binding.
8283
this.nativeOptions = buildSeaConnectionOptions(options);
8384

85+
// Bridge the Rust kernel's `tracing` logs into the SAME `DBSQLLogger` the
86+
// driver logs through, so logs from all three layers (driver, napi shim,
87+
// kernel) land in one place — and one file when the logger has a file
88+
// transport. Kernel verbosity follows the logger's own level; loggers that
89+
// don't expose `getLevel()` leave the bridge at `info`. No-op on a binding
90+
// that predates the bridge (logging is advisory).
91+
const logger = this.context.getLogger();
92+
const kernelLogLevel = logger.getLevel?.() ?? LogLevel.info;
93+
installKernelLogBridge(this.binding, logger, kernelLogLevel);
94+
8495
// Warn on the insecure combo: a `customCaCert` paired with
8596
// `checkServerCertificate: false` is almost always a mistake — verification
8697
// is fully off, so the custom trust anchor is never used. The combo is

lib/sea/SeaLogging.ts

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
// Copyright (c) 2026 Databricks, Inc.
2+
//
3+
// Licensed under the Apache License, Version 2.0 (the "License");
4+
// you may not use this file except in compliance with the License.
5+
// You may obtain a copy of the License at
6+
//
7+
// http://www.apache.org/licenses/LICENSE-2.0
8+
//
9+
// Unless required by applicable law or agreed to in writing, software
10+
// distributed under the License is distributed on an "AS IS" BASIS,
11+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
// See the License for the specific language governing permissions and
13+
// limitations under the License.
14+
15+
/**
16+
* Kernel → driver log bridge.
17+
*
18+
* The Rust kernel emits its diagnostics via `tracing`. In a Node process those
19+
* events have no subscriber and are dropped — so by default the driver's
20+
* `DBSQLLogger` only ever saw JS-side lines. The napi binding's
21+
* `initKernelLogging` installs a process-global subscriber that forwards
22+
* kernel events (batched) to a JS callback; this module wires that callback
23+
* into the **same** `IDBSQLLogger` the driver logs through, so logs from all
24+
* three layers (driver, napi shim, kernel) land in one place — and one file
25+
* when the logger has a file transport.
26+
*
27+
* Verbosity follows the driver's logger level (see `installKernelLogBridge`),
28+
* filtered kernel-side so we don't pay the channel/bridge cost for events the
29+
* sink would discard anyway.
30+
*/
31+
32+
import IDBSQLLogger, { LogLevel } from '../contracts/IDBSQLLogger';
33+
import { SeaNativeBinding, SeaNativeLogRecord } from './SeaNativeLoader';
34+
35+
/**
36+
* Map a kernel level string (`error`/`warn`/`info`/`debug`/`trace`) onto the
37+
* driver's `LogLevel`. The kernel's `trace` has no `LogLevel` analogue, so it
38+
* folds into `debug` (the most verbose driver level).
39+
*/
40+
export function kernelLevelToLogLevel(level: string): LogLevel {
41+
switch (level) {
42+
case 'error':
43+
return LogLevel.error;
44+
case 'warn':
45+
return LogLevel.warn;
46+
case 'info':
47+
return LogLevel.info;
48+
case 'debug':
49+
case 'trace':
50+
return LogLevel.debug;
51+
default:
52+
// Unknown kernel level — surface it rather than drop it; debug is the
53+
// least surprising bucket for an unrecognised severity.
54+
return LogLevel.debug;
55+
}
56+
}
57+
58+
/**
59+
* Map a driver `LogLevel` onto the kernel level string the napi bridge expects.
60+
* The `LogLevel` enum values are already the kernel-compatible lower-case
61+
* strings, so this is the identity at runtime — kept as an explicit function so
62+
* the boundary is named and a future divergence has one place to live.
63+
*/
64+
export function logLevelToKernelLevel(level: LogLevel): string {
65+
return level;
66+
}
67+
68+
/**
69+
* Format one kernel log record into a single driver log line, tagged with its
70+
* origin so kernel lines are distinguishable from driver lines in a shared
71+
* sink/file.
72+
*/
73+
export function formatKernelLine(record: SeaNativeLogRecord): string {
74+
return `[kernel ${record.target}] ${record.message}`;
75+
}
76+
77+
/**
78+
* Install the kernel→driver log bridge: forward kernel `tracing` events into
79+
* `logger` at `level`.
80+
*
81+
* - **Verbosity** is set kernel-side to `level` so events the sink would drop
82+
* never cross the bridge.
83+
* - **Process-global, last-writer-wins:** the napi binding holds a single
84+
* process-global subscriber + sink (a `tracing` global subscriber installs
85+
* once). Each call retargets the sink to `logger`, so in a multi-client
86+
* process the most recently connected client's logger receives kernel logs —
87+
* mirroring the Python connector's `pyo3_log` model. Single-client apps, the
88+
* common case, are unaffected.
89+
* - **Graceful on older bindings:** if the loaded `.node` predates
90+
* `initKernelLogging`, this is a no-op (kernel logs simply stay unbridged)
91+
* rather than a hard failure — logging is advisory.
92+
*/
93+
export function installKernelLogBridge(binding: SeaNativeBinding, logger: IDBSQLLogger, level: LogLevel): void {
94+
// Defensive: a stale/older binding without the bridge export must not break
95+
// connect() — logging is non-critical.
96+
if (typeof binding.initKernelLogging !== 'function') {
97+
return;
98+
}
99+
100+
const callback = (err: Error | null, records: Array<SeaNativeLogRecord>): void => {
101+
if (err || !records) {
102+
return;
103+
}
104+
for (const record of records) {
105+
logger.log(kernelLevelToLogLevel(record.level), formatKernelLine(record));
106+
}
107+
};
108+
109+
binding.initKernelLogging(callback, logLevelToKernelLevel(level));
110+
}

lib/sea/SeaNativeLoader.ts

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@ import type {
3939
AsyncStatement as NativeAsyncStatement,
4040
AsyncResultHandle as NativeAsyncResultHandle,
4141
CancellableExecution as NativeCancellableExecution,
42+
LogRecord as NativeLogRecord,
4243
} from '../../native/sea';
4344

4445
// SEA-prefixed re-exports. The kernel-generated `.d.ts` keeps the
@@ -78,6 +79,11 @@ export type SeaNativeAsyncResultHandle = NativeAsyncResultHandle;
7879
// returns.
7980
export type SeaNativeCancellableExecution = NativeCancellableExecution;
8081

82+
// One kernel log event forwarded over the napi log bridge (see SeaLogging.ts):
83+
// `{ level, target, message }`. Re-exported so the bridge can name the shape
84+
// without re-declaring it (stays in lock-step with the kernel contract).
85+
export type SeaNativeLogRecord = NativeLogRecord;
86+
8187
/**
8288
* The full native binding surface, derived from the generated module
8389
* so it can never drift from the `.d.ts` contract: when the kernel

native/sea/index.d.ts

Lines changed: 30 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

native/sea/index.js

Lines changed: 3 additions & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

tests/e2e/sea/logging-e2e.test.ts

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
// Copyright (c) 2026 Databricks, Inc.
2+
//
3+
// Licensed under the Apache License, Version 2.0 (the "License");
4+
// you may not use this file except in compliance with the License.
5+
// You may obtain a copy of the License at
6+
//
7+
// http://www.apache.org/licenses/LICENSE-2.0
8+
//
9+
// Unless required by applicable law or agreed to in writing, software
10+
// distributed under the License is distributed on an "AS IS" BASIS,
11+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
// See the License for the specific language governing permissions and
13+
// limitations under the License.
14+
15+
// End-to-end proof that kernel (Rust) logs and Node-driver logs land in the
16+
// SAME `DBSQLLogger` sink — here, one file. Goes through the full public
17+
// `DBSQLClient` surface (not the raw binding) so the `SeaBackend` →
18+
// `installKernelLogBridge` → napi `initKernelLogging` wiring is exercised.
19+
20+
import { expect } from 'chai';
21+
import fs from 'fs';
22+
import os from 'os';
23+
import path from 'path';
24+
import { tryGetSeaNative } from '../../../lib/sea/SeaNativeLoader';
25+
import { DBSQLClient } from '../../../lib';
26+
import DBSQLLogger from '../../../lib/DBSQLLogger';
27+
import { LogLevel } from '../../../lib/contracts/IDBSQLLogger';
28+
import config from '../utils/config';
29+
import { InternalConnectionOptions } from '../../../lib/contracts/InternalConnectionOptions';
30+
31+
const delay = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));
32+
33+
describe('SEA — unified kernel + driver logging', function unifiedLogging() {
34+
// Live-warehouse round-trip plus async log flush.
35+
this.timeout(60_000);
36+
37+
const binding = tryGetSeaNative();
38+
if (binding === undefined) {
39+
it.skip('SEA native binding not available on this platform');
40+
return;
41+
}
42+
43+
it('routes kernel (Rust) logs into the same DBSQLLogger file as driver logs', async () => {
44+
const logFile = path.join(fs.mkdtempSync(path.join(os.tmpdir(), 'dbsql-kernel-log-')), 'unified.log');
45+
// debug so the kernel's per-statement lifecycle events cross the bridge.
46+
const logger = new DBSQLLogger({ level: LogLevel.debug, filepath: logFile });
47+
const client = new DBSQLClient({ logger });
48+
49+
await client.connect({
50+
host: config.host,
51+
path: config.path,
52+
token: config.token,
53+
// Route through the kernel backend (internal opt-in flag).
54+
...({ useSEA: true } as InternalConnectionOptions),
55+
});
56+
const session = await client.openSession();
57+
const operation = await session.executeStatement('SELECT 1');
58+
await operation.fetchAll();
59+
await operation.close();
60+
await session.close();
61+
await client.close();
62+
63+
// winston's file transport + the batched kernel bridge are async; give them
64+
// a beat to flush before reading the file back.
65+
await delay(1_500);
66+
67+
const contents = fs.readFileSync(logFile, 'utf8');
68+
const lines = contents.split('\n').filter((l) => l.trim().length > 0);
69+
70+
const kernelLines = lines.filter((l) => l.includes('[kernel '));
71+
const driverLines = lines.filter((l) => !l.includes('[kernel '));
72+
73+
// Both layers present in the one file → unified.
74+
expect(driverLines.length, 'expected driver-origin log lines').to.be.greaterThan(0);
75+
expect(kernelLines.length, 'expected kernel-origin ([kernel …) log lines').to.be.greaterThan(0);
76+
// The kernel target tag is preserved.
77+
expect(contents).to.match(/\[kernel databricks::sql::kernel\]/);
78+
79+
fs.rmSync(path.dirname(logFile), { recursive: true, force: true });
80+
});
81+
});

0 commit comments

Comments
 (0)