Skip to content

Commit 47c8963

Browse files
joaoh82claude
andauthored
feat(sql): support JOIN ... USING / NATURAL / CROSS (SQLR-5) (#158)
PR #99 shipped INNER/LEFT/RIGHT/FULL OUTER JOIN ... ON; the three related shapes SQLite supports were still rejected as NotImplemented. This wires all three through the existing nested-loop join driver. Parser (src/sql/parser/select.rs): - JoinClause.on: Expr → constraint: JoinConstraintKind (On/Using/Natural). - USING (cols) is narrowed to a list of column names; NATURAL is carried as-is (it needs schemas the parser doesn't have); CROSS JOIN is rewritten to ON true at parse time. Executor (src/sql/executor.rs): - resolve_join_constraint lowers USING/NATURAL into the synthesized `left.col = right.col [AND …]` predicate, schema-aware: the left qualifier is picked per column so join chains resolve correctly, and NATURAL auto-discovers the shared column names (none ⇒ cross product, matching SQLite). - SELECT * de-duplicates USING/NATURAL columns per the SQLite convention — the joined-on column shows once, taking the left copy. Tests: 8 new executor tests (USING≡ON rows, SELECT* dedup for USING and NATURAL, NATURAL multi-column AND, NATURAL-without-common-cols cross product, CROSS cartesian product, LEFT OUTER USING, USING unknown-column error), replacing the old "returns NotImplemented" test. Docs updated (supported-sql, sql-engine, roadmap, README, web docs). Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent d508336 commit 47c8963

7 files changed

Lines changed: 468 additions & 57 deletions

File tree

README.md

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -175,7 +175,7 @@ sqlrite> DELETE FROM users WHERE age < 30;
175175
| `CREATE TABLE` | `PRIMARY KEY`, `UNIQUE`, `NOT NULL`; `IF NOT EXISTS` (idempotent re-create); duplicate-column detection; types `INTEGER`/`INT`/`BIGINT`/`SMALLINT`, `TEXT`/`VARCHAR`, `REAL`/`FLOAT`/`DOUBLE`/`DECIMAL`, `BOOLEAN`. Auto-creates `sqlrite_autoindex_<table>_<col>` for every PK + UNIQUE column |
176176
| `CREATE [UNIQUE] INDEX` | Single-column, named indexes; `IF NOT EXISTS`; persists as a dedicated cell-based B-Tree. INTEGER + TEXT columns only |
177177
| `INSERT INTO` | Explicit column list required; auto-ROWID for `INTEGER PRIMARY KEY`; multi-row `VALUES (…), (…)`; UNIQUE enforcement; clean type errors (no panics); NULL padding for omitted columns |
178-
| `SELECT` | `*` or column list with optional `AS alias`; `WHERE`; `DISTINCT`; `GROUP BY col[, col …]`; aggregate projections `COUNT(*)` / `COUNT([DISTINCT] col)` / `SUM` / `AVG` / `MIN` / `MAX`; `[INNER\|LEFT OUTER\|RIGHT OUTER\|FULL OUTER] JOIN ... ON ...` with table aliases and qualified `t.col` references; single-column `ORDER BY [ASC\|DESC]` (also resolves alias and aggregate display names); `LIMIT n`. `WHERE col = literal` probes an index when one exists. Catalog introspection via `SELECT … FROM sqlrite_master` |
178+
| `SELECT` | `*` or column list with optional `AS alias`; `WHERE`; `DISTINCT`; `GROUP BY col[, col …]`; aggregate projections `COUNT(*)` / `COUNT([DISTINCT] col)` / `SUM` / `AVG` / `MIN` / `MAX`; `[INNER\|LEFT OUTER\|RIGHT OUTER\|FULL OUTER\|CROSS] JOIN` with `ON ...` / `USING (...)` / `NATURAL` constraints, table aliases and qualified `t.col` references; single-column `ORDER BY [ASC\|DESC]` (also resolves alias and aggregate display names); `LIMIT n`. `WHERE col = literal` probes an index when one exists. Catalog introspection via `SELECT … FROM sqlrite_master` |
179179
| `UPDATE` | Multi-column `SET`; `WHERE`; UNIQUE + type enforcement; arithmetic in assignments (`SET age = age + 1`) |
180180
| `DELETE` | `WHERE` predicate or full-table delete |
181181
| `BEGIN` / `COMMIT` / `ROLLBACK` | Real transactions, snapshot-based; WAL-backed commit; single-level (no savepoints); auto-rollback if `COMMIT`'s disk write fails |
@@ -193,7 +193,7 @@ Expressions in `WHERE` and `UPDATE`'s `SET` RHS:
193193
- String concat — `||`
194194
- Literals — integer + real numbers, `'single-quoted strings'`, `TRUE` / `FALSE`, `NULL`; parentheses for grouping
195195

196-
**Not yet supported** (common ones): subqueries, CTEs, `HAVING`, `LIKE … ESCAPE '<char>'`, `IN (subquery)`, `DISTINCT` on `SUM`/`AVG`/`MIN`/`MAX`, GROUP BY on expressions, expressions in the projection list, `OFFSET`, multi-column `ORDER BY`, savepoints, `JOIN ... USING`, `NATURAL JOIN`, `CROSS JOIN`, comma joins, aggregates / DISTINCT / GROUP BY *over* JOIN results. The [full list with context](docs/supported-sql.md#not-yet-supported) lives in the reference.
196+
**Not yet supported** (common ones): subqueries, CTEs, `HAVING`, `LIKE … ESCAPE '<char>'`, `IN (subquery)`, `DISTINCT` on `SUM`/`AVG`/`MIN`/`MAX`, GROUP BY on expressions, expressions in the projection list, `OFFSET`, multi-column `ORDER BY`, savepoints, comma joins (`FROM a, b`), aggregates / DISTINCT / GROUP BY *over* JOIN results. The [full list with context](docs/supported-sql.md#not-yet-supported) lives in the reference.
197197

198198
#### Meta commands
199199

@@ -250,7 +250,7 @@ The project is staged in phases, each independently shippable. A finished phase
250250
- [x] `CREATE TABLE` with `PRIMARY KEY`, `UNIQUE`, `NOT NULL`; duplicate-column detection; in-memory `BTreeMap` indexes on PK/UNIQUE columns
251251
- [x] `INSERT` with auto-ROWID for `INTEGER PRIMARY KEY`, UNIQUE enforcement, NULL padding for missing columns
252252
- [x] `SELECT` — projection, `WHERE`, `ORDER BY`, `LIMIT`
253-
- [x] `JOIN``INNER`, `LEFT OUTER`, `RIGHT OUTER`, `FULL OUTER` with `ON` (SQLR-5)
253+
- [x] `JOIN``INNER`, `LEFT OUTER`, `RIGHT OUTER`, `FULL OUTER`, `CROSS` with `ON` / `USING (...)` / `NATURAL` (SQLR-5)
254254
- [x] `UPDATE ... SET ... WHERE ...` with type + UNIQUE enforcement at write time
255255
- [x] `DELETE ... WHERE ...`
256256
- [x] Expression evaluator: `=`/`<>`/`<`/`<=`/`>`/`>=`, `AND`/`OR`/`NOT`, arithmetic `+`/`-`/`*`/`/`/`%`, string concat `||`, NULL-as-false in `WHERE`
@@ -332,7 +332,6 @@ Lockstep versioning — one dispatch bumps every product to the same `vX.Y.Z`. T
332332
- [ ] *(deferred to Phase 8)* Full-text search with BM25 + hybrid retrieval
333333

334334
**Possible extras** *(no committed phase)*
335-
- Joins (`INNER`, `LEFT OUTER`, `CROSS` — SQLite does not support `RIGHT`/`FULL OUTER`)
336335
- `HAVING`, `IN (subquery)`, `BETWEEN`, `GLOB` / `REGEXP`, `GROUP_CONCAT`, window functions
337336
- Composite and expression indexes (with cost analysis)
338337
- Alternate storage engines — LSM/SSTable for write-heavy workloads alongside the B-Tree

docs/roadmap.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -561,7 +561,7 @@ The biggest single SQL-surface jump in the project's history.
561561
- Self-joins require an alias on at least one side.
562562
- `WHERE` runs after joins (the standard `LEFT JOIN ... WHERE right.col IS NULL` anti-join idiom works).
563563

564-
Not yet supported: `CROSS JOIN`, comma-separated FROMs, `NATURAL JOIN`, `JOIN ... USING (col)`, aggregates / `GROUP BY` / `DISTINCT` *over* a join, `fts_match` / `bm25_score` inside a join expression. Algorithm: plain nested-loop, O(N×M) per level — hash / merge joins are a future optimization.
564+
`ON`, `USING (...)`, `NATURAL`, and `CROSS JOIN` are all supported. Not yet supported: comma-separated FROMs (`FROM a, b`), aggregates / `GROUP BY` / `DISTINCT` *over* a join, `fts_match` / `bm25_score` inside a join expression. Algorithm: plain nested-loop, O(N×M) per level — hash / merge joins are a future optimization.
565565

566566
### ✅ Phase 9g — Prepared statements + parameter binding *(v0.9.0, SQLR-23)*
567567

docs/sql-engine.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,7 @@ The `sqlparser` AST is designed to cover every SQL dialect, so its types are hug
5353

5454
`SelectQuery::joins` (SQLR-5) is a `Vec<JoinClause>` evaluated left-to-right by `execute_select_rows_joined`. Each clause carries a `JoinType` (`Inner` / `LeftOuter` / `RightOuter` / `FullOuter`), the right-table name + optional alias, and a required `ON` expression. Empty = single-table SELECT, the existing fast path with HNSW / FTS / bounded-heap optimizations.
5555

56-
Each parser module still rejects features we don't implement with `SQLRiteError::NotImplemented``JOIN ... USING`, `NATURAL JOIN`, `CROSS JOIN`, comma joins, aggregates / GROUP BY / DISTINCT over JOINs, `HAVING`, `DISTINCT ON (...)`, `GROUP BY` on expressions, `LIKE … ESCAPE '<char>'`, `IN (subquery)`, `OFFSET`, multi-table DELETE, tuple assignment targets, etc. These errors carry the feature name in the message so the user knows what isn't there.
56+
Each parser module still rejects features we don't implement with `SQLRiteError::NotImplemented`comma joins (`FROM a, b`), aggregates / GROUP BY / DISTINCT over JOINs, `HAVING`, `DISTINCT ON (...)`, `GROUP BY` on expressions, `LIKE … ESCAPE '<char>'`, `IN (subquery)`, `OFFSET`, multi-table DELETE, tuple assignment targets, etc. These errors carry the feature name in the message so the user knows what isn't there. (`JOIN ... USING`, `NATURAL JOIN`, and `CROSS JOIN` are now supported — see [`supported-sql.md`](supported-sql.md#join-semantics-sqlr-5).)
5757

5858
## Statement dispatch
5959

docs/supported-sql.md

Lines changed: 14 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -210,19 +210,27 @@ COUNT([DISTINCT] <column>) -- counts non-NULL values, option
210210

211211
### `JOIN` semantics (SQLR-5)
212212

213-
Four flavors are supported, all with explicit `ON` conditions:
213+
Four flavors are supported, with `ON`, `USING (...)`, or `NATURAL` match
214+
conditions, plus `CROSS JOIN`:
214215

215216
| Flavor | Keeps unmatched rows from… |
216217
|---|---|
217-
| `INNER JOIN` | …neither side. Only ON-matched pairs survive. |
218+
| `INNER JOIN` | …neither side. Only matched pairs survive. |
218219
| `LEFT [OUTER] JOIN` | …the left side; right-side columns become `NULL` for unmatched left rows. |
219220
| `RIGHT [OUTER] JOIN` | …the right side; left-side columns become `NULL` for unmatched right rows. |
220221
| `FULL [OUTER] JOIN` | …both sides, NULL-padded on the unmatched side. |
222+
| `CROSS JOIN` | …both sides (cross product — every left row paired with every right row). |
221223

222224
- **Engine choice:** SQLite ships only `INNER` and `LEFT OUTER`. SQLRite implements all four because the per-flavor differences boil down to NULL-padding policy on top of one shared nested-loop driver — adding `RIGHT` / `FULL` was effectively free once the executor had a multi-table scope. See [`docs/design-decisions.md`](design-decisions.md) for the rationale.
225+
- **Match conditions:**
226+
- **`ON <expr>`** — any boolean expression over the in-scope tables.
227+
- **`USING (col[, col…])`** — shorthand for `left.col = right.col` AND-chained over each named column. The column must exist on the right side and on some left-side table; in a chain (`A JOIN B USING(x) JOIN C USING(x)`) each `x` resolves against the first left table that has it.
228+
- **`NATURAL`** — equivalent to `USING (<every column name the two sides share>)`, discovered automatically from the schemas. If the sides share no column names, a `NATURAL JOIN` degrades to a cross product (matching SQLite). Combines with a flavor: `NATURAL LEFT JOIN`.
229+
- **`CROSS JOIN`** — the cross product; the engine treats it as `INNER JOIN ... ON true`.
230+
- **`SELECT *` with `USING` / `NATURAL`:** each joined-on column appears **once** (SQLite convention), taking the left side's value; the right side's duplicate is omitted. Plain `ON` joins keep both copies.
223231
- **Aliases:** `FROM customers AS c INNER JOIN orders AS o ON c.id = o.customer_id`. When an alias is supplied the original table name leaves scope (SQL standard) — qualifier resolution uses the alias.
224232
- **Qualified column references:** `<table>.<col>` and `<alias>.<col>` resolve to that specific side. Bare `<col>` references must resolve to exactly one in-scope table; ambiguous references error with a "qualify it as `<table>.col`" hint.
225-
- **Output of `SELECT *`** over a join is every column of every in-scope table, in source order. Duplicate header names are permitted (SQLite-style). Disambiguate with explicit `SELECT t.col AS t_col, u.col AS u_col`.
233+
- **Output of `SELECT *`** over a join is every column of every in-scope table, in source order (minus `USING` / `NATURAL` duplicates, see above). Duplicate header names are otherwise permitted (SQLite-style). Disambiguate with explicit `SELECT t.col AS t_col, u.col AS u_col`.
226234
- **Multi-join** chains left-fold: `A JOIN B ON ... JOIN C ON ...` evaluates as `(A ⨝ B) ⨝ C`. Each new clause sees every prior alias / table in its `ON` expression.
227235
- **Self-joins** require an alias on at least one side: `FROM nodes AS p INNER JOIN nodes AS c ON p.id = c.parent_id`. Without one, you get a `duplicate table reference` error so qualifiers stay unambiguous.
228236
- **`WHERE` runs after joins.** A `WHERE right.col IS NULL` filter on a `LEFT JOIN` correctly returns left rows with no match (the standard "anti-join via outer-join" idiom).
@@ -231,8 +239,7 @@ Four flavors are supported, all with explicit `ON` conditions:
231239

232240
#### What's not supported in JOINs
233241

234-
- `JOIN ... USING (col)` and `NATURAL JOIN` — explicit `ON` only. (Both are deferred — `USING` is straightforward but adds a column-resolution rule we haven't needed yet.)
235-
- `CROSS JOIN` (write `INNER JOIN ... ON true` instead) and comma-separated FROM lists.
242+
- Comma-separated FROM lists (`FROM a, b`) — use an explicit `JOIN` / `CROSS JOIN` instead.
236243
- Aggregates / `GROUP BY` / `DISTINCT` *over* a join. The single-table aggregator is wired against one rowid stream; rewiring it for joined rows is a separate increment. Surfaces as a clean `NotImplemented` at parse time.
237244
- `fts_match` / `bm25_score` inside a JOIN expression. They need to look up an FTS index by column, which is single-table-bound today. Use them on a single-table SELECT first, or fold the FTS lookup into the FROM side.
238245

@@ -255,7 +262,7 @@ The executor includes a tiny optimizer: if the `WHERE` is exactly `<indexed_col>
255262

256263
### What doesn't work
257264

258-
- **`CROSS JOIN`**, **comma-separated FROM lists**, **`NATURAL JOIN`**, **`JOIN ... USING (col)`** — explicit `INNER` / `LEFT` / `RIGHT` / `FULL OUTER JOIN ... ON ...` only (see [JOIN semantics](#join-semantics-sqlr-5))
265+
- **Comma-separated FROM lists** (`FROM a, b`) — use an explicit `JOIN` / `CROSS JOIN`. `INNER` / `LEFT` / `RIGHT` / `FULL OUTER` / `CROSS` with `ON` / `USING` / `NATURAL` are all supported (see [JOIN semantics](#join-semantics-sqlr-5))
259266
- **Aggregates** / **`GROUP BY`** / **`DISTINCT`** over a JOIN — pipe through a subquery once subqueries land
260267
- **Subqueries**, CTEs (`WITH`), views
261268
- **`HAVING`** — pre-aggregation `WHERE` works; post-aggregation filtering does not yet
@@ -700,7 +707,7 @@ A REPL launched with `sqlrite --readonly foo.sqlrite` (or `sqlrite::open_databas
700707
For context when you hit `NotImplemented`. See [Roadmap](roadmap.md) for when these land:
701708

702709
### Joins & composition
703-
- `CROSS JOIN`, comma joins, `NATURAL JOIN`, `JOIN ... USING` — explicit `INNER` / `LEFT` / `RIGHT` / `FULL OUTER JOIN ... ON ...` works (SQLR-5); the others don't
710+
- `INNER` / `LEFT` / `RIGHT` / `FULL OUTER` / `CROSS JOIN` with `ON` / `USING (...)` / `NATURAL` all work (SQLR-5). Comma-separated FROM joins (`FROM a, b`) don't — use an explicit `JOIN` / `CROSS JOIN`
704711
- Aggregates / `GROUP BY` / `DISTINCT` *over* a JOIN — pipe through a subquery once subqueries land
705712
- `fts_match` / `bm25_score` inside a JOIN expression — single-table-bound today
706713
- Subqueries (scalar, `IN (SELECT ...)`, correlated)

0 commit comments

Comments
 (0)