Skip to content

Commit f939f20

Browse files
committed
docs(sql): add SQL extensions section and expand existing pages
Add eight new SQL reference pages covering JSON operators, COPY, LISTEN/NOTIFY, custom types, window frames, lateral subqueries, recursive CTEs, and grouping sets. Register them under a new "SQL Extensions" sidebar group in oxidoc.toml, alongside a new "SQL Index" group for the index page. Expand existing pages with RETURNING clause examples, UPDATE/DELETE FROM join syntax, additional JOIN types, RRF fusion scoring details, FTS ranking and highlighting functions, and CRDT merge strategy notes.
1 parent 0fb0c56 commit f939f20

18 files changed

Lines changed: 1196 additions & 2 deletions

docs/sql/copy.rdx

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
---
2+
title: COPY FROM / COPY TO
3+
description: Bulk import and export of collections using NDJSON, JSON array, and CSV formats.
4+
---
5+
6+
# COPY FROM / COPY TO
7+
8+
`COPY` moves data between a NodeDB collection and a file on the server's filesystem. It is the fastest way to load or dump large datasets.
9+
10+
## COPY FROM — import
11+
12+
```sql
13+
COPY <collection> FROM '<path>' [WITH (FORMAT <fmt> [, DELIMITER '<char>'] [, HEADER <bool>])]
14+
```
15+
16+
Reads the file at `<path>` on the server and inserts every record into `<collection>`. The format is either specified explicitly or auto-detected from the file extension.
17+
18+
### Auto-detected formats
19+
20+
| Extension | Format |
21+
| ------------------- | ------------ |
22+
| `.ndjson`, `.jsonl` | NDJSON |
23+
| `.json` | JSON array |
24+
| `.csv` | CSV |
25+
26+
### Examples
27+
28+
```sql
29+
-- NDJSON (one JSON object per line) — format auto-detected from extension
30+
COPY users FROM '/data/users.ndjson';
31+
32+
-- JSON array (a single top-level array of objects)
33+
COPY users FROM '/data/users.json' WITH (FORMAT json_array);
34+
35+
-- CSV with header row (default when FORMAT csv)
36+
COPY orders FROM '/data/orders.csv' WITH (FORMAT csv);
37+
38+
-- CSV without header row
39+
COPY orders FROM '/data/orders.csv' WITH (FORMAT csv, HEADER false);
40+
41+
-- CSV with non-comma delimiter
42+
COPY products FROM '/data/products.tsv' WITH (FORMAT csv, DELIMITER '\t');
43+
```
44+
45+
### WITH options
46+
47+
| Option | Values | Default | Description |
48+
| ----------- | ------------------------------- | ----------------- | --------------------------------------- |
49+
| `FORMAT` | `ndjson`, `json_array`, `csv` | auto from extension | File format |
50+
| `DELIMITER` | Any single character | `,` | Field separator (CSV only) |
51+
| `HEADER` | `true` / `false` | `true` | Whether the first CSV row is a header |
52+
53+
## COPY TO — export
54+
55+
```sql
56+
COPY <collection> TO '<path>' [WITH (FORMAT <fmt> [, DELIMITER '<char>'] [, HEADER <bool>])]
57+
COPY (SELECT ...) TO '<path>' [WITH (...)]
58+
```
59+
60+
Writes every row from `<collection>` (or the result of a SELECT) to a file at `<path>` on the server. The format is auto-detected from the extension or specified explicitly.
61+
62+
### Examples
63+
64+
```sql
65+
-- Export entire collection as NDJSON
66+
COPY users TO '/exports/users.ndjson';
67+
68+
-- Export with explicit format
69+
COPY users TO '/exports/users.json' WITH (FORMAT json_array);
70+
71+
-- Export as CSV with header
72+
COPY orders TO '/exports/orders.csv' WITH (FORMAT csv);
73+
74+
-- Export the result of a query
75+
COPY (SELECT id, name, email FROM users WHERE status = 'active')
76+
TO '/exports/active_users.csv' WITH (FORMAT csv);
77+
78+
-- Query export as NDJSON
79+
COPY (SELECT * FROM orders WHERE created_at > '2025-01-01')
80+
TO '/exports/recent_orders.ndjson';
81+
```
82+
83+
## Notes
84+
85+
- Paths are resolved on the **server** filesystem. The client never streams file bytes — this is not `\copy` (psql client-side copy).
86+
- `COPY FROM STDIN` is reserved for the backup/restore path and is not available as a general import mechanism.
87+
- Rows that fail type coercion during import are rejected with a parse error; no partial-row insertion occurs.
88+
- `COPY TO` is non-transactional: the file is written even if the session is later rolled back.
89+
- The `COPY (SELECT ...) TO` form supports all SELECT features including JOINs, CTEs, and WHERE clauses.

docs/sql/crdt-operations.rdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,4 +23,4 @@ Applies a Loro CRDT delta to the document. On Origin, the delta goes through Raf
2323

2424
## Conflict Policies
2525

26-
Conflict policies (last-writer-wins, field-merge, custom) are configured per-collection at create time and via the native NodeDB protocol's `ALTER POLICY` command. SQL DDL for inspecting and updating conflict policies — `SHOW CONFLICT POLICY ON <collection>` and `ALTER COLLECTION ... SET ON CONFLICT ...` — is wired in the v0.1.0 release alongside the rest of the SQL surface. See the protocol reference for the native-protocol form, and [CRDT Sync](/docs/crdt-sync/overview) for full sync protocol details.
26+
Conflict policies (last-writer-wins, field-merge, custom) are configured per-collection at create time and via the native NodeDB protocol's `ALTER POLICY` command. SQL DDL for inspecting and updating conflict policies — `SHOW CONFLICT POLICY ON <collection>` and `ALTER COLLECTION ... SET ON CONFLICT ...` — is available alongside the rest of the SQL surface. See the protocol reference for the native-protocol form, and [CRDT Sync](/docs/crdt-sync/overview) for full sync protocol details.

docs/sql/custom-types.rdx

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
---
2+
title: Custom Types
3+
description: CREATE TYPE for enums and composite types, ALTER TYPE ADD VALUE, and DROP TYPE.
4+
---
5+
6+
# Custom Types
7+
8+
NodeDB supports user-defined types in two forms: **enum types** (a fixed, ordered set of string labels) and **composite types** (a named record with typed fields). Both can be used as column types in `CREATE COLLECTION` and `CREATE TABLE` statements.
9+
10+
## Enum Types
11+
12+
```sql
13+
CREATE TYPE <name> AS ENUM ('<label1>', '<label2>', ...)
14+
```
15+
16+
Defines a type whose valid values are the listed labels. Labels are case-sensitive strings. Duplicates are rejected at definition time.
17+
18+
```sql
19+
CREATE TYPE order_status AS ENUM ('pending', 'processing', 'shipped', 'delivered', 'cancelled');
20+
21+
CREATE TYPE priority AS ENUM ('low', 'medium', 'high', 'critical');
22+
```
23+
24+
### Using enum types
25+
26+
```sql
27+
CREATE COLLECTION orders (
28+
id TEXT PRIMARY KEY,
29+
status order_status,
30+
priority priority
31+
) WITH (engine = 'document_strict');
32+
33+
INSERT INTO orders (id, status, priority) VALUES ('o1', 'pending', 'high');
34+
35+
SELECT * FROM orders WHERE status = 'pending';
36+
```
37+
38+
### Adding a new label
39+
40+
```sql
41+
ALTER TYPE order_status ADD VALUE 'on_hold';
42+
```
43+
44+
The new value is appended to the end of the enum. Existing rows are unaffected. `ADD VALUE` is the only supported `ALTER TYPE` form.
45+
46+
## Composite Types
47+
48+
```sql
49+
CREATE TYPE <name> AS (<field1> <type1>, <field2> <type2>, ...)
50+
```
51+
52+
Defines a named record type. Each field has a name and a SQL data type.
53+
54+
```sql
55+
CREATE TYPE address AS (
56+
street TEXT,
57+
city TEXT,
58+
zip TEXT,
59+
country TEXT
60+
);
61+
62+
CREATE TYPE money_amount AS (
63+
value FLOAT,
64+
currency TEXT
65+
);
66+
```
67+
68+
### Using composite types
69+
70+
```sql
71+
CREATE COLLECTION users (
72+
id TEXT PRIMARY KEY,
73+
name TEXT,
74+
billing address
75+
) WITH (engine = 'document_strict');
76+
77+
INSERT INTO users (id, name, billing)
78+
VALUES ('u1', 'Alice', ROW('123 Main St', 'Springfield', '12345', 'US'));
79+
80+
SELECT billing.city FROM users WHERE id = 'u1';
81+
```
82+
83+
## Inspecting types
84+
85+
```sql
86+
SHOW TYPES;
87+
```
88+
89+
Lists all user-defined types in the current tenant, their kind (enum or composite), and their labels or fields.
90+
91+
## Dropping types
92+
93+
```sql
94+
DROP TYPE order_status;
95+
DROP TYPE IF EXISTS address;
96+
```
97+
98+
`DROP TYPE` is **protected**: it fails if any collection has a column of that type. Drop or alter the dependent collections first, then drop the type.
99+
100+
## OIDs
101+
102+
Custom types are assigned OIDs starting at 70001. These appear in `pg_type` catalog queries and in the pgwire type-description messages for columns that use the custom type.
103+
104+
<Callout kind="warning" title="pgwire binary codec">
105+
Custom types are currently sent over pgwire in TEXT format regardless of the client's requested format. Binary-format decoding for custom type OIDs is not yet implemented. Most PostgreSQL client libraries fall back gracefully to text parsing.
106+
</Callout>
107+
108+
## Validation rules
109+
110+
- **Enum labels** must be non-empty strings. Duplicate labels within the same type are rejected.
111+
- **Composite fields** must have distinct names within the same type. The field types can be any built-in SQL type; nested composite types are not yet supported.
112+
- **Type names** are lowercased and must be unique within the tenant.

docs/sql/fulltext-search.rdx

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,61 @@ CJK text is automatically tokenized via character bigrams:
2828
SELECT title FROM articles WHERE text_match(body, '全文検索');
2929
```
3030

31+
## NOT Operator
32+
33+
Exclude documents that match specific terms using the `NOT` keyword or the `-` prefix (Lucene-style, no space before the term).
34+
35+
Both forms require at least one positive term — a query consisting only of negations is rejected.
36+
37+
```sql
38+
-- Exclude documents that mention 'python'
39+
SELECT title FROM articles
40+
WHERE text_match(body, 'rust NOT python');
41+
42+
-- Multiple exclusions
43+
SELECT title FROM articles
44+
WHERE text_match(body, 'database NOT mysql NOT oracle');
45+
46+
-- Lucene-style dash prefix (equivalent to NOT)
47+
SELECT title FROM articles
48+
WHERE text_match(body, 'database -mysql -oracle');
49+
```
50+
51+
Parenthesised NOT groups are not supported. Use flat negations:
52+
53+
```sql
54+
-- Not supported: rust NOT (python OR ruby)
55+
-- Supported: rust NOT python NOT ruby
56+
SELECT title FROM articles
57+
WHERE text_match(body, 'rust NOT python NOT ruby');
58+
```
59+
60+
## Synonym Groups
61+
62+
A synonym group makes the FTS engine treat a set of terms as interchangeable during indexing and querying. When a document contains any term in the group it also matches queries for any other term in the group.
63+
64+
```sql
65+
-- Define synonyms
66+
CREATE SYNONYM GROUP db_terms AS ('database', 'db', 'datastore');
67+
CREATE SYNONYM GROUP ml_terms AS ('machine learning', 'ml', 'artificial intelligence', 'ai');
68+
69+
-- Now a query for 'db' also matches documents that contain 'database' or 'datastore'
70+
SELECT title FROM articles WHERE text_match(body, 'db performance');
71+
```
72+
73+
### Managing synonym groups
74+
75+
```sql
76+
-- List all groups
77+
SHOW SYNONYM GROUPS;
78+
79+
-- Drop a group
80+
DROP SYNONYM GROUP db_terms;
81+
DROP SYNONYM GROUP IF EXISTS ml_terms;
82+
```
83+
84+
Synonym group names are case-insensitive. Terms within a group are stored lowercased. Duplicate terms within the same group are rejected at creation time.
85+
3186
## Hybrid Search (BM25 + Vector)
3287

3388
```sql

docs/sql/fusion-rrf.rdx

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,32 @@ SEARCH entities USING FUSION(
4848

4949
Vector search finds seed nodes, graph BFS expands context, RRF merges both rankings.
5050

51+
## Three-Source Fusion (Vector + Text + Graph)
52+
53+
When you want vector similarity, BM25 text relevance, and graph context all merged in one pass, add the `BM25 '<query>' ON '<field>'` leg to the `SEARCH … USING FUSION` DSL. The `RRF_K` tuple becomes a triple.
54+
55+
```sql
56+
SEARCH entities USING FUSION(
57+
ARRAY[0.1, 0.3, -0.2, ...]
58+
VECTOR_FIELD 'embedding' VECTOR_TOP_K 50
59+
BM25 'transformer attention' ON 'body'
60+
DEPTH 2 LABEL 'related_to'
61+
TOP 10 RRF_K (60.0, 35.0, 50.0)
62+
);
63+
```
64+
65+
The three `RRF_K` values correspond to the vector leg, the graph expansion leg, and the BM25 leg respectively. When all three are present the planner routes through the three-source RRF plan variant.
66+
67+
The two-source `GRAPH RAG FUSION ON … BM25 '…' ON '…'` form also accepts the extra text leg inline:
68+
69+
```sql
70+
GRAPH RAG FUSION ON entities
71+
QUERY $embedding VECTOR_FIELD 'embedding' VECTOR_TOP_K 50
72+
BM25 'transformer attention' ON 'body'
73+
EXPANSION_DEPTH 2 EDGE_LABEL 'related_to' FINAL_TOP_K 10
74+
RRF_K (60.0, 35.0, 50.0);
75+
```
76+
5177
## Cross-Model Queries
5278

5379
All engines share the same snapshot. A query that combines vector similarity, graph traversal, spatial filtering, and document field access sees a consistent point-in-time view.

0 commit comments

Comments
 (0)