Skip to content

Commit e690b8d

Browse files
authored
feat: schema-aware semantic diagnostics (unknown tables/columns, ambiguous columns) (#163)
## Summary Linting was syntax-only, even though the extension already accepts a full `SQLNamespace` schema (used by hover). This adds **`sqlSemanticLinter`**, a schema-aware linter that reports the highest-value SQL diagnostics: - **Unknown table** — `FROM`/`JOIN` references and DML targets (`INSERT`/`UPDATE`/`DELETE`) resolved against the namespace. Case-insensitive; qualified paths like `mydb.users` traverse nested namespaces; CTE names and DDL targets (`CREATE TABLE x` defines `x`) are skipped. - **Unknown column** — qualified refs (`u.name`, `users.name`) resolved through a per-scope alias map; unqualified refs checked only when the statement reads exactly one known table. - **Ambiguous column** — unqualified refs that exist in 2+ referenced tables (suppressed for `USING`/`NATURAL` joins). Severities are configurable per check (`"error" | "warning" | "off"`, default `"warning"`), and semantic diagnostics carry `source: "sql-schema"` vs. `"sql-parser"` for syntax errors. ## False positives are fatal to trust The analyzer builds proper query scopes from the AST and skips anything not confidently resolvable, preferring under-reporting: - checks only run on statements that parse cleanly (no semantic noise on top of syntax errors) - CTE outputs, FROM-subquery results, and table functions are opaque — their columns are never guessed - unqualified columns are never checked inside correlated subqueries (they may legally resolve to the outer scope) - SELECT-list aliases referenced in `ORDER BY`/`GROUP BY` are skipped - unresolved qualifiers (possible outer-scope aliases) are skipped - under-qualified names match tables nested deeper in the schema, so `FROM users` with schema `{mydb: {users}}` never flags - without a schema (or with an empty placeholder while it lazily loads upstream) the linter is fully inert — zero parser calls ## Shared schema facet Also introduces **`sqlSchemaFacet`** so the schema is defined once for hover + semantic linting (and future completion) instead of per sub-extension. `sqlExtension` gains top-level `schema`, `enableSemanticLinting` (default true; inert without a schema), and `semanticLinterConfig`. Per-extension `schema` config still overrides for back-compat. ## Implementation note on positions The plan called for `parseOptions: { includeLocations: true }`, but node-sql-parser does not attach `loc` to `column_ref`/table nodes even with it enabled (verified empirically; only some expression wrappers get it). Diagnostics are instead positioned by word-bounded, case-insensitive text search within the statement's document range — searching the original doc text keeps positions exact even around inline comments (covered by a test). ## Testing - 24 new tests in `src/sql/__tests__/semantic-diagnostics.test.ts` covering all checks, CTE/alias/subquery/correlation scoping, nested namespaces, `self/children` namespaces, Completion-object columns, severity overrides, facet fallback/precedence, inertness gating, and diagnostic positioning - `pnpm test`: 334 passed | 1 expected fail (pre-existing) - `pnpm run typecheck` and `pnpm exec oxlint`: clean (also fixed the one pre-existing oxlint warning in `extension.ts`) - demo updated to exercise semantic linting (`pnpm run demo` builds) <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Add schema-aware SQL linting that flags unknown tables/columns and ambiguous columns, shared with hover via a new schema facet. Introduces `sqlSemanticLinter` and `sqlSchemaFacet`, and adds top-level `schema` and `enableSemanticLinting` options to `sqlExtension`; also refines parser/analyzer inheritance to avoid dialect mismatches. - **New Features** - `sqlSemanticLinter`: reports unknown tables, unknown columns, and ambiguous unqualified columns; per-check severity (`error` | `warning` | `off`, default `warning`); diagnostics use `source: "sql-schema"`; skips unresolvable cases to avoid false positives. - `sqlSchemaFacet`: shared schema for hover and semantic lint; supports function sources; per-feature `schema` still overrides; exported `resolveSqlSchema`. - `sqlExtension`: new `schema`, `enableSemanticLinting` (default true; inert without a schema), and `semanticLinterConfig`; hover falls back to the shared schema; semantic linter reuses `linterConfig.parser` and inherits `linterConfig.structureAnalyzer` only when the linter parser is inherited, preventing analyzer/dialect mismatches. - **Migration** - If you already pass a schema to hover, move it to `sqlExtension({ schema })` to share across features. - To enable semantic linting, supply a schema and optionally set severities via `semanticLinterConfig.severity`. - To disable, set `enableSemanticLinting: false`. <sup>Written for commit fb7ca9e. Summary will update on new commits.</sup> <a href="https://cubic.dev/pr/marimo-team/codemirror-sql/pull/163?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->
1 parent 575c72e commit e690b8d

11 files changed

Lines changed: 1256 additions & 8 deletions

File tree

README.md

Lines changed: 37 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ A CodeMirror extension for SQL linting and visual gutter indicators. Built by an
55
## Features
66

77
-**Real-time validation** - Per-statement SQL syntax checking as you type, with detailed error messages for every broken statement
8+
- 🧠 **Schema-aware linting** - Warns about unknown tables, unknown columns, and ambiguous column references based on your schema
89
- 🎨 **Visual gutter** - Color-coded statement indicators and error highlighting
910
- 💡 **Hover tooltips** - Schema info, keywords, and column details on hover
1011
- 🔮 **CTE autocomplete** - Auto-complete support for CTEs
@@ -47,6 +48,8 @@ const editor = new EditorView({
4748
autocomplete: cteCompletionSource,
4849
}),
4950
sqlExtension({
51+
// Shared by hover tooltips and semantic linting
52+
schema: schema,
5053
linterConfig: {
5154
delay: 250, // Validation delay in ms
5255
},
@@ -57,7 +60,6 @@ const editor = new EditorView({
5760
},
5861
enableHover: true,
5962
hoverConfig: {
60-
schema: schema,
6163
hoverTime: 300,
6264
enableKeywords: true,
6365
enableTables: true,
@@ -69,6 +71,40 @@ const editor = new EditorView({
6971
});
7072
```
7173

74+
### Schema-aware semantic linting
75+
76+
When a schema is provided (via the top-level `schema` option, the
77+
`sqlSchemaFacet`, or `semanticLinterConfig.schema`), queries are validated
78+
against it: unknown tables, unknown columns, and ambiguous column references
79+
are reported as warnings (configurable per check). Without a schema the
80+
semantic linter is inert.
81+
82+
```ts
83+
import { EditorView } from "codemirror";
84+
import { sqlSemanticLinter } from "@marimo-team/codemirror-sql";
85+
86+
const editor = new EditorView({
87+
extensions: [
88+
sqlSemanticLinter({
89+
schema: { users: ["id", "name"], posts: ["id", "user_id"] },
90+
severity: {
91+
unknownTable: "error", // "error" | "warning" | "off" (default: "warning")
92+
unknownColumn: "warning",
93+
ambiguousColumn: "warning",
94+
},
95+
}),
96+
],
97+
parent: document.querySelector("#editor"),
98+
});
99+
```
100+
101+
Checks only run on statements that parse cleanly, and skip anything that
102+
can't be confidently resolved (CTE outputs, subquery results, aliases from
103+
outer scopes), preferring under-reporting over false positives. Semantic
104+
diagnostics carry `source: "sql-schema"`; syntax diagnostics use
105+
`source: "sql-parser"`. If the schema is provided as a function, it is called
106+
on every lint pass and should be cheap/memoized.
107+
72108
## Additional Dialects
73109

74110
This extension adds support for additional dialects:

_typos.toml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,4 +7,6 @@ extend-exclude = [
77
"src/data/duckdb-keywords.json",
88
# Contains intentionally misspelled SQL (e.g. SELCT) to trigger parse errors
99
"src/sql/__tests__/diagnostics.test.ts",
10+
# Contains intentionally misspelled identifiers (e.g. usres) to trigger schema diagnostics
11+
"src/sql/__tests__/semantic-diagnostics.test.ts",
1012
]

demo/index.ts

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -134,11 +134,18 @@ function initializeEditor() {
134134
databaseField,
135135
baseSqlCompartment.of(baseSqlExtension(defaultDialect)),
136136
sqlExtension({
137+
// Shared schema for hover tooltips and schema-aware linting
138+
schema: schema,
137139
// Linter extension configuration
138140
linterConfig: {
139141
delay: 250, // Delay before running validation
140142
parser,
141143
},
144+
// Schema-aware linting (unknown tables/columns, ambiguous columns)
145+
semanticLinterConfig: {
146+
delay: 250,
147+
parser,
148+
},
142149

143150
// Gutter extension configuration
144151
gutterConfig: {

src/__tests__/index.test.ts

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,9 +14,12 @@ describe("index.ts exports", () => {
1414
"SqlStructureAnalyzer",
1515
"cteCompletionSource",
1616
"defaultSqlHoverTheme",
17+
"resolveSqlSchema",
1718
"sqlExtension",
1819
"sqlHover",
1920
"sqlLinter",
21+
"sqlSchemaFacet",
22+
"sqlSemanticLinter",
2023
"sqlStructureGutter",
2124
]
2225
`);

src/index.ts

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
export { cteCompletionSource } from "./sql/cte-completion-source.js";
22
export { type SqlLinterConfig, sqlLinter } from "./sql/diagnostics.js";
3-
export { sqlExtension } from "./sql/extension.js";
3+
export { type SqlExtensionConfig, sqlExtension } from "./sql/extension.js";
44
export type {
55
KeywordTooltipData,
66
NamespaceTooltipData,
@@ -15,6 +15,12 @@ export {
1515
type ParserOption,
1616
type SupportedDialects,
1717
} from "./sql/parser.js";
18+
export { resolveSqlSchema, type SqlSchemaSource, sqlSchemaFacet } from "./sql/schema-facet.js";
19+
export {
20+
type SemanticSeverity,
21+
type SqlSemanticLinterConfig,
22+
sqlSemanticLinter,
23+
} from "./sql/semantic-diagnostics.js";
1824
export type { SqlStatement } from "./sql/structure-analyzer.js";
1925
export { SqlStructureAnalyzer } from "./sql/structure-analyzer.js";
2026
export type { SqlGutterConfig } from "./sql/structure-extension.js";

src/sql/__tests__/extension.test.ts

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
1+
import { EditorState } from "@codemirror/state";
12
import { describe, expect, it } from "vitest";
23
import { sqlExtension } from "../extension.js";
4+
import { sqlSchemaFacet } from "../schema-facet.js";
35

46
describe("sqlExtension", () => {
57
it("should return an array of extensions with default config", () => {
@@ -35,12 +37,37 @@ describe("sqlExtension", () => {
3537
it("should handle all features disabled", () => {
3638
const extensions = sqlExtension({
3739
enableLinting: false,
40+
enableSemanticLinting: false,
3841
enableGutterMarkers: false,
3942
enableHover: false,
4043
});
4144
expect(extensions).toEqual([]);
4245
});
4346

47+
it("should register the shared schema facet when a schema is provided", () => {
48+
const schema = { users: ["id"] };
49+
const withSchema = sqlExtension({
50+
schema,
51+
enableLinting: false,
52+
enableSemanticLinting: false,
53+
enableGutterMarkers: false,
54+
enableHover: false,
55+
});
56+
expect(withSchema).toHaveLength(1);
57+
58+
const state = EditorState.create({ extensions: withSchema });
59+
expect(state.facet(sqlSchemaFacet)).toBe(schema);
60+
61+
const withoutSchema = sqlExtension({
62+
enableLinting: false,
63+
enableSemanticLinting: false,
64+
enableGutterMarkers: false,
65+
enableHover: false,
66+
});
67+
const emptyState = EditorState.create({ extensions: withoutSchema });
68+
expect(emptyState.facet(sqlSchemaFacet)).toBeNull();
69+
});
70+
4471
it("should pass config objects to individual extensions", () => {
4572
const linterConfig = { delay: 500 };
4673
const gutterConfig = { backgroundColor: "#ff0000" };

0 commit comments

Comments
 (0)