You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
feat(sql): support JOIN ... USING / NATURAL / CROSS (SQLR-5) (#158)
PR #99 shipped INNER/LEFT/RIGHT/FULL OUTER JOIN ... ON; the three
related shapes SQLite supports were still rejected as NotImplemented.
This wires all three through the existing nested-loop join driver.
Parser (src/sql/parser/select.rs):
- JoinClause.on: Expr → constraint: JoinConstraintKind (On/Using/Natural).
- USING (cols) is narrowed to a list of column names; NATURAL is carried
as-is (it needs schemas the parser doesn't have); CROSS JOIN is
rewritten to ON true at parse time.
Executor (src/sql/executor.rs):
- resolve_join_constraint lowers USING/NATURAL into the synthesized
`left.col = right.col [AND …]` predicate, schema-aware: the left
qualifier is picked per column so join chains resolve correctly, and
NATURAL auto-discovers the shared column names (none ⇒ cross product,
matching SQLite).
- SELECT * de-duplicates USING/NATURAL columns per the SQLite
convention — the joined-on column shows once, taking the left copy.
Tests: 8 new executor tests (USING≡ON rows, SELECT* dedup for USING and
NATURAL, NATURAL multi-column AND, NATURAL-without-common-cols cross
product, CROSS cartesian product, LEFT OUTER USING, USING unknown-column
error), replacing the old "returns NotImplemented" test. Docs updated
(supported-sql, sql-engine, roadmap, README, web docs).
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: README.md
+3-4Lines changed: 3 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -175,7 +175,7 @@ sqlrite> DELETE FROM users WHERE age < 30;
175
175
|`CREATE TABLE`|`PRIMARY KEY`, `UNIQUE`, `NOT NULL`; `IF NOT EXISTS` (idempotent re-create); duplicate-column detection; types `INTEGER`/`INT`/`BIGINT`/`SMALLINT`, `TEXT`/`VARCHAR`, `REAL`/`FLOAT`/`DOUBLE`/`DECIMAL`, `BOOLEAN`. Auto-creates `sqlrite_autoindex_<table>_<col>` for every PK + UNIQUE column |
176
176
|`CREATE [UNIQUE] INDEX`| Single-column, named indexes; `IF NOT EXISTS`; persists as a dedicated cell-based B-Tree. INTEGER + TEXT columns only |
177
177
|`INSERT INTO`| Explicit column list required; auto-ROWID for `INTEGER PRIMARY KEY`; multi-row `VALUES (…), (…)`; UNIQUE enforcement; clean type errors (no panics); NULL padding for omitted columns |
178
-
|`SELECT`|`*` or column list with optional `AS alias`; `WHERE`; `DISTINCT`; `GROUP BY col[, col …]`; aggregate projections `COUNT(*)` / `COUNT([DISTINCT] col)` / `SUM` / `AVG` / `MIN` / `MAX`; `[INNER\|LEFT OUTER\|RIGHT OUTER\|FULL OUTER] JOIN... ON ...` with table aliases and qualified `t.col` references; single-column `ORDER BY [ASC\|DESC]` (also resolves alias and aggregate display names); `LIMIT n`. `WHERE col = literal` probes an index when one exists. Catalog introspection via `SELECT … FROM sqlrite_master`|
178
+
|`SELECT`|`*` or column list with optional `AS alias`; `WHERE`; `DISTINCT`; `GROUP BY col[, col …]`; aggregate projections `COUNT(*)` / `COUNT([DISTINCT] col)` / `SUM` / `AVG` / `MIN` / `MAX`; `[INNER\|LEFT OUTER\|RIGHT OUTER\|FULL OUTER\|CROSS] JOIN` with `ON ...` / `USING (...)` / `NATURAL` constraints, table aliases and qualified `t.col` references; single-column `ORDER BY [ASC\|DESC]` (also resolves alias and aggregate display names); `LIMIT n`. `WHERE col = literal` probes an index when one exists. Catalog introspection via `SELECT … FROM sqlrite_master`|
179
179
|`UPDATE`| Multi-column `SET`; `WHERE`; UNIQUE + type enforcement; arithmetic in assignments (`SET age = age + 1`) |
180
180
|`DELETE`|`WHERE` predicate or full-table delete |
181
181
|`BEGIN` / `COMMIT` / `ROLLBACK`| Real transactions, snapshot-based; WAL-backed commit; single-level (no savepoints); auto-rollback if `COMMIT`'s disk write fails |
@@ -193,7 +193,7 @@ Expressions in `WHERE` and `UPDATE`'s `SET` RHS:
193
193
- String concat — `||`
194
194
- Literals — integer + real numbers, `'single-quoted strings'`, `TRUE` / `FALSE`, `NULL`; parentheses for grouping
195
195
196
-
**Not yet supported** (common ones): subqueries, CTEs, `HAVING`, `LIKE … ESCAPE '<char>'`, `IN (subquery)`, `DISTINCT` on `SUM`/`AVG`/`MIN`/`MAX`, GROUP BY on expressions, expressions in the projection list, `OFFSET`, multi-column `ORDER BY`, savepoints, `JOIN ... USING`, `NATURAL JOIN`, `CROSS JOIN`, comma joins, aggregates / DISTINCT / GROUP BY *over* JOIN results. The [full list with context](docs/supported-sql.md#not-yet-supported) lives in the reference.
196
+
**Not yet supported** (common ones): subqueries, CTEs, `HAVING`, `LIKE … ESCAPE '<char>'`, `IN (subquery)`, `DISTINCT` on `SUM`/`AVG`/`MIN`/`MAX`, GROUP BY on expressions, expressions in the projection list, `OFFSET`, multi-column `ORDER BY`, savepoints, comma joins (`FROM a, b`), aggregates / DISTINCT / GROUP BY *over* JOIN results. The [full list with context](docs/supported-sql.md#not-yet-supported) lives in the reference.
197
197
198
198
#### Meta commands
199
199
@@ -250,7 +250,7 @@ The project is staged in phases, each independently shippable. A finished phase
250
250
-[x]`CREATE TABLE` with `PRIMARY KEY`, `UNIQUE`, `NOT NULL`; duplicate-column detection; in-memory `BTreeMap` indexes on PK/UNIQUE columns
251
251
-[x]`INSERT` with auto-ROWID for `INTEGER PRIMARY KEY`, UNIQUE enforcement, NULL padding for missing columns
Copy file name to clipboardExpand all lines: docs/roadmap.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -561,7 +561,7 @@ The biggest single SQL-surface jump in the project's history.
561
561
- Self-joins require an alias on at least one side.
562
562
-`WHERE` runs after joins (the standard `LEFT JOIN ... WHERE right.col IS NULL` anti-join idiom works).
563
563
564
-
Not yet supported: `CROSS JOIN`, comma-separated FROMs, `NATURAL JOIN`, `JOIN ... USING (col)`, aggregates / `GROUP BY` / `DISTINCT`*over* a join, `fts_match` / `bm25_score` inside a join expression. Algorithm: plain nested-loop, O(N×M) per level — hash / merge joins are a future optimization.
564
+
`ON`, `USING (...)`, `NATURAL`, and `CROSS JOIN` are all supported. Not yet supported: comma-separated FROMs (`FROM a, b`), aggregates / `GROUP BY` / `DISTINCT`*over* a join, `fts_match` / `bm25_score` inside a join expression. Algorithm: plain nested-loop, O(N×M) per level — hash / merge joins are a future optimization.
Copy file name to clipboardExpand all lines: docs/sql-engine.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -53,7 +53,7 @@ The `sqlparser` AST is designed to cover every SQL dialect, so its types are hug
53
53
54
54
`SelectQuery::joins` (SQLR-5) is a `Vec<JoinClause>` evaluated left-to-right by `execute_select_rows_joined`. Each clause carries a `JoinType` (`Inner` / `LeftOuter` / `RightOuter` / `FullOuter`), the right-table name + optional alias, and a required `ON` expression. Empty = single-table SELECT, the existing fast path with HNSW / FTS / bounded-heap optimizations.
55
55
56
-
Each parser module still rejects features we don't implement with `SQLRiteError::NotImplemented` — `JOIN ... USING`, `NATURAL JOIN`, `CROSS JOIN`, comma joins, aggregates / GROUP BY / DISTINCT over JOINs, `HAVING`, `DISTINCT ON (...)`, `GROUP BY` on expressions, `LIKE … ESCAPE '<char>'`, `IN (subquery)`, `OFFSET`, multi-table DELETE, tuple assignment targets, etc. These errors carry the feature name in the message so the user knows what isn't there.
56
+
Each parser module still rejects features we don't implement with `SQLRiteError::NotImplemented` — comma joins (`FROM a, b`), aggregates / GROUP BY / DISTINCT over JOINs, `HAVING`, `DISTINCT ON (...)`, `GROUP BY` on expressions, `LIKE … ESCAPE '<char>'`, `IN (subquery)`, `OFFSET`, multi-table DELETE, tuple assignment targets, etc. These errors carry the feature name in the message so the user knows what isn't there. (`JOIN ... USING`, `NATURAL JOIN`, and `CROSS JOIN` are now supported — see [`supported-sql.md`](supported-sql.md#join-semantics-sqlr-5).)
Four flavors are supported, all with explicit `ON` conditions:
213
+
Four flavors are supported, with `ON`, `USING (...)`, or `NATURAL` match
214
+
conditions, plus `CROSS JOIN`:
214
215
215
216
| Flavor | Keeps unmatched rows from… |
216
217
|---|---|
217
-
|`INNER JOIN`| …neither side. Only ON-matched pairs survive. |
218
+
|`INNER JOIN`| …neither side. Only matched pairs survive. |
218
219
|`LEFT [OUTER] JOIN`| …the left side; right-side columns become `NULL` for unmatched left rows. |
219
220
|`RIGHT [OUTER] JOIN`| …the right side; left-side columns become `NULL` for unmatched right rows. |
220
221
|`FULL [OUTER] JOIN`| …both sides, NULL-padded on the unmatched side. |
222
+
|`CROSS JOIN`| …both sides (cross product — every left row paired with every right row). |
221
223
222
224
-**Engine choice:** SQLite ships only `INNER` and `LEFT OUTER`. SQLRite implements all four because the per-flavor differences boil down to NULL-padding policy on top of one shared nested-loop driver — adding `RIGHT` / `FULL` was effectively free once the executor had a multi-table scope. See [`docs/design-decisions.md`](design-decisions.md) for the rationale.
225
+
-**Match conditions:**
226
+
-**`ON <expr>`** — any boolean expression over the in-scope tables.
227
+
-**`USING (col[, col…])`** — shorthand for `left.col = right.col` AND-chained over each named column. The column must exist on the right side and on some left-side table; in a chain (`A JOIN B USING(x) JOIN C USING(x)`) each `x` resolves against the first left table that has it.
228
+
-**`NATURAL`** — equivalent to `USING (<every column name the two sides share>)`, discovered automatically from the schemas. If the sides share no column names, a `NATURAL JOIN` degrades to a cross product (matching SQLite). Combines with a flavor: `NATURAL LEFT JOIN`.
229
+
-**`CROSS JOIN`** — the cross product; the engine treats it as `INNER JOIN ... ON true`.
230
+
-**`SELECT *` with `USING` / `NATURAL`:** each joined-on column appears **once** (SQLite convention), taking the left side's value; the right side's duplicate is omitted. Plain `ON` joins keep both copies.
223
231
-**Aliases:**`FROM customers AS c INNER JOIN orders AS o ON c.id = o.customer_id`. When an alias is supplied the original table name leaves scope (SQL standard) — qualifier resolution uses the alias.
224
232
-**Qualified column references:**`<table>.<col>` and `<alias>.<col>` resolve to that specific side. Bare `<col>` references must resolve to exactly one in-scope table; ambiguous references error with a "qualify it as `<table>.col`" hint.
225
-
-**Output of `SELECT *`** over a join is every column of every in-scope table, in source order. Duplicate header names are permitted (SQLite-style). Disambiguate with explicit `SELECT t.col AS t_col, u.col AS u_col`.
233
+
-**Output of `SELECT *`** over a join is every column of every in-scope table, in source order (minus `USING` / `NATURAL` duplicates, see above). Duplicate header names are otherwise permitted (SQLite-style). Disambiguate with explicit `SELECT t.col AS t_col, u.col AS u_col`.
226
234
-**Multi-join** chains left-fold: `A JOIN B ON ... JOIN C ON ...` evaluates as `(A ⨝ B) ⨝ C`. Each new clause sees every prior alias / table in its `ON` expression.
227
235
-**Self-joins** require an alias on at least one side: `FROM nodes AS p INNER JOIN nodes AS c ON p.id = c.parent_id`. Without one, you get a `duplicate table reference` error so qualifiers stay unambiguous.
228
236
-**`WHERE` runs after joins.** A `WHERE right.col IS NULL` filter on a `LEFT JOIN` correctly returns left rows with no match (the standard "anti-join via outer-join" idiom).
@@ -231,8 +239,7 @@ Four flavors are supported, all with explicit `ON` conditions:
231
239
232
240
#### What's not supported in JOINs
233
241
234
-
-`JOIN ... USING (col)` and `NATURAL JOIN` — explicit `ON` only. (Both are deferred — `USING` is straightforward but adds a column-resolution rule we haven't needed yet.)
235
-
-`CROSS JOIN` (write `INNER JOIN ... ON true` instead) and comma-separated FROM lists.
242
+
- Comma-separated FROM lists (`FROM a, b`) — use an explicit `JOIN` / `CROSS JOIN` instead.
236
243
- Aggregates / `GROUP BY` / `DISTINCT`*over* a join. The single-table aggregator is wired against one rowid stream; rewiring it for joined rows is a separate increment. Surfaces as a clean `NotImplemented` at parse time.
237
244
-`fts_match` / `bm25_score` inside a JOIN expression. They need to look up an FTS index by column, which is single-table-bound today. Use them on a single-table SELECT first, or fold the FTS lookup into the FROM side.
238
245
@@ -255,7 +262,7 @@ The executor includes a tiny optimizer: if the `WHERE` is exactly `<indexed_col>
255
262
256
263
### What doesn't work
257
264
258
-
-**`CROSS JOIN`**, **comma-separated FROM lists**, **`NATURAL JOIN`**, **`JOIN ... USING (col)`** — explicit `INNER` / `LEFT` / `RIGHT` / `FULL OUTER JOIN ... ON ...` only (see [JOIN semantics](#join-semantics-sqlr-5))
265
+
-**Comma-separated FROM lists** (`FROM a, b`) — use an explicit `JOIN` / `CROSS JOIN`. `INNER` / `LEFT` / `RIGHT` / `FULL OUTER` / `CROSS` with `ON` / `USING` / `NATURAL` are all supported (see [JOIN semantics](#join-semantics-sqlr-5))
259
266
-**Aggregates** / **`GROUP BY`** / **`DISTINCT`** over a JOIN — pipe through a subquery once subqueries land
260
267
-**Subqueries**, CTEs (`WITH`), views
261
268
-**`HAVING`** — pre-aggregation `WHERE` works; post-aggregation filtering does not yet
@@ -700,7 +707,7 @@ A REPL launched with `sqlrite --readonly foo.sqlrite` (or `sqlrite::open_databas
700
707
For context when you hit `NotImplemented`. See [Roadmap](roadmap.md) for when these land:
701
708
702
709
### Joins & composition
703
-
-`CROSS JOIN`, comma joins, `NATURAL JOIN`, `JOIN ... USING` — explicit `INNER` / `LEFT` / `RIGHT` / `FULL OUTER JOIN ... ON ...` works (SQLR-5); the others don't
710
+
-`INNER` / `LEFT` / `RIGHT` / `FULL OUTER` / `CROSS JOIN` with `ON` / `USING (...)` / `NATURAL` all work (SQLR-5). Comma-separated FROM joins (`FROM a, b`) don't — use an explicit `JOIN` / `CROSS JOIN`
704
711
- Aggregates / `GROUP BY` / `DISTINCT`*over* a JOIN — pipe through a subquery once subqueries land
705
712
-`fts_match` / `bm25_score` inside a JOIN expression — single-table-bound today
0 commit comments