Skip to content

feat(sql): IS NULL / IS NOT NULL + typed Option<Value> INSERT pipeline (SQLR-7)#95

Merged
joaoh82 merged 1 commit into
mainfrom
sqlr-7-is-null
May 5, 2026
Merged

feat(sql): IS NULL / IS NOT NULL + typed Option<Value> INSERT pipeline (SQLR-7)#95
joaoh82 merged 1 commit into
mainfrom
sqlr-7-is-null

Conversation

@joaoh82

@joaoh82 joaoh82 commented May 5, 2026

Copy link
Copy Markdown
Owner

Summary

  • Adds WHERE col IS NULL / IS NOT NULL to the executor (two new eval_expr arms; full-scan fallback since NULLs aren't indexed).
  • Refactors the INSERT pipeline from stringly-typed Vec<Vec<String>> (with "Null" as a sentinel) to typed Vec<Vec<Option<Value>>> from the parser through Table::insert_row and validate_unique_constraint.
  • Fixes Table::restore_row to accept NULL for INTEGER / REAL / BOOLEAN / VECTOR columns so persisted NULLs reopen cleanly — without this, a saved DB containing any NULL outside TEXT failed to reload.

Bugs fixed

  • INSERT INTO t (n) VALUES (NULL) for INTEGER / REAL / BOOLEAN / VECTOR columns errored via "Null".parse::<T>() failing.
  • INSERT INTO t (s) VALUES (NULL) for TEXT silently stored the literal string "Null".
  • SQL UNIQUE now allows multiple NULLs (standard three-valued logic).
  • DEFAULT NULL collapses to "no default" — explicit NULL in INSERT is preserved over the column DEFAULT (matches SQLite).
  • Restored SQLR-2's default_does_not_override_explicit_null test that had to be dropped because of this bug.

Test plan

  • cargo test --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs — 397 engine tests pass (was 396)
  • cargo fmt --all -- --check clean
  • cargo clippy --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --all-targets clean (no new warnings beyond the pre-existing set)
  • REPL smoke test: create, insert mixed NULL / non-NULL rows, run IS NULL / IS NOT NULL, close, reopen — values round-trip
  • New file-backed null_values_round_trip_through_disk exercises every column type's NULL through save/reload
  • 6 INSERT-NULL tests (one per column type: INTEGER / TEXT / REAL / BOOLEAN / VECTOR / JSON)
  • 5 IS NULL / IS NOT NULL executor tests (non-indexed, indexed UNIQUE, omitted-column, combined-with-AND)

Out of scope (deferred)

  • Removing the legacy Row::Text "Null" sentinel-strip on read (kept as back-compat for pre-fix saved DBs; revisit at next file-format bump).
  • Negative numeric literals in INSERT VALUES (pre-existing Expr::UnaryOp(Minus, …) gap).
  • IS NULL index optimization (would need a NULL-rowid sidecar; full scan is fine for now).

🤖 Generated with Claude Code

…e (SQLR-7)

Two related fixes from SQLR-2 fallout:

1) `WHERE col IS NULL` / `IS NOT NULL` now work — two new arms in
   `eval_expr`. NULLs aren't stored in secondary / HNSW / FTS indexes,
   so these predicates correctly fall through to a full scan via the
   existing `select_rowids` logic.

2) INSERT no longer carries values as `Vec<Vec<String>>` with the
   string sentinel `"Null"` standing in for SQL NULL. `InsertQuery.rows`
   is now `Vec<Vec<Option<Value>>>`; the parser emits `None` for NULL
   and typed `Value::*` for everything else. `Table::insert_row` and
   `Table::validate_unique_constraint` dispatch on `Option<Value>`,
   leaving the per-column BTreeMap entry absent for NULL (reads come
   back as `Value::Null` via the existing missing-rowid path).

   Fixes:
   - `INSERT INTO t (n) VALUES (NULL)` for INTEGER / REAL / BOOLEAN /
     VECTOR columns no longer errors via `"Null".parse::<T>()`
   - `INSERT INTO t (s) VALUES (NULL)` for TEXT no longer stores the
     literal string `"Null"`
   - `Table::restore_row` accepts NULL for non-text column types so
     persisted NULLs reopen cleanly (without this, a saved DB
     containing any NULL outside TEXT failed to reload)
   - SQL UNIQUE allows multiple NULLs (standard SQL three-valued
     logic)
   - `DEFAULT NULL` collapses to "no default" — explicit NULL in
     INSERT is preserved over a column DEFAULT, matching SQLite

Tests: 8 new INSERT-NULL tests across all column types; 5 IS NULL /
IS NOT NULL executor tests (non-indexed, indexed, omitted-column,
combined-with-AND); restored the SQLR-2 `default_does_not_override_
explicit_null` test that had to be dropped because of this bug; new
file-backed `null_values_round_trip_through_disk`. The previous
`process_command_insert_missing_integer_returns_error_test` codified
the bug as a graceful error and is replaced with a positive
"omitted INTEGER stores NULL" assertion.

Docs: removed "not yet supported" notes for IS NULL / IS NOT NULL
from README, supported-sql.md, usage.md, roadmap.md.

Out of scope (deferred):
  - Removing the legacy `Row::Text "Null"` sentinel-strip on read
    (kept as back-compat for pre-fix saved DBs; revisit at next
    file-format bump)
  - Negative numeric literals in INSERT VALUES (pre-existing gap)
  - IS NULL index optimization (full scan is fine for now)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@joaoh82 joaoh82 merged commit 5db377c into main May 5, 2026
16 checks passed
@joaoh82 joaoh82 mentioned this pull request May 5, 2026
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant