Skip to content

Commit 49b6117

Browse files
committed
fix(docs): correct Array SQL function names, type OIDs, and WAL header layout
Rename all NDARRAY_* SQL functions to ARRAY_* to match the implemented DDL surface (ARRAY_SLICE, ARRAY_PROJECT, ARRAY_AGG, ARRAY_ELEMENTWISE, ARRAY_FLUSH, ARRAY_COMPACT, ALTER ARRAY, DROP ARRAY). Correct pgwire type-OID table: split TIMESTAMP (1114, tz-naive) from TIMESTAMPTZ (1184, UTC-normalised), add INTERVAL (1186) and VECTOR (float4[], 1021), add usage guidance on which to prefer. Update WAL segment header diagram to match current layout: record_type widened to 4 bytes, tenant_id to 8 bytes, vshard_id to 4 bytes, 16-byte reserved field added; total header 54 bytes. Document encryption flag (bit 14 of record_type) and byte offsets. Remove unimplemented SHOW/ALTER CONFLICT POLICY SQL from CRDT docs and replace with accurate status note. Remove unimplemented CREATE SYNONYM GROUP from FTS docs. Correct vector-search syntax from SEARCH...USING VECTOR() to ORDER BY vector_distance(...) LIMIT k. Update SQL overview vector-search operator list.
1 parent 68c54a1 commit 49b6117

8 files changed

Lines changed: 51 additions & 56 deletions

File tree

docs/reference/type-oids.rdx

Lines changed: 18 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -7,17 +7,22 @@ description: PostgreSQL type OID mappings for pgwire protocol compatibility.
77

88
NodeDB maps its types to PostgreSQL OIDs for pgwire compatibility.
99

10-
| NodeDB Type | PostgreSQL Type | OID |
11-
| ----------- | --------------- | ---- |
12-
| STRING | text | 25 |
13-
| INT | int8 | 20 |
14-
| FLOAT | float8 | 701 |
15-
| BOOL | bool | 16 |
16-
| TIMESTAMP | timestamptz | 1184 |
17-
| UUID | uuid | 2950 |
18-
| DECIMAL | numeric | 1700 |
19-
| ARRAY | jsonb | 3802 |
20-
| OBJECT | jsonb | 3802 |
21-
| GEOMETRY | bytea | 17 |
10+
| NodeDB Type | PostgreSQL Type | OID |
11+
| ------------ | --------------- | ---- |
12+
| STRING | text | 25 |
13+
| INT | int8 | 20 |
14+
| FLOAT | float8 | 701 |
15+
| BOOL | bool | 16 |
16+
| TIMESTAMP | timestamp | 1114 |
17+
| TIMESTAMPTZ | timestamptz | 1184 |
18+
| INTERVAL | interval | 1186 |
19+
| UUID | uuid | 2950 |
20+
| DECIMAL | numeric | 1700 |
21+
| ARRAY | jsonb | 3802 |
22+
| OBJECT | jsonb | 3802 |
23+
| GEOMETRY | bytea | 17 |
24+
| VECTOR | float4[] | 1021 |
2225

23-
Complex types (ARRAY, OBJECT) are serialized as JSONB over pgwire. GEOMETRY is serialized as WKB (bytea).
26+
`TIMESTAMP` (OID 1114) is timezone-naive. `TIMESTAMPTZ` (OID 1184) is UTC-normalised. Use `TIMESTAMPTZ` for wall-clock event times; `TIMESTAMP` for values that are inherently local (e.g. a schedule expressed in the user's timezone before conversion).
27+
28+
Complex types (ARRAY, OBJECT) are serialized as JSONB over pgwire. GEOMETRY is serialized as WKB (bytea). VECTOR is serialized as a float4 array.

docs/reference/wal-segment-formats.rdx

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -8,12 +8,16 @@ description: On-disk format specifications for WAL records and segment files.
88
## WAL Record Format
99

1010
```
11-
┌─────────┬────────────────┬────────────┬─────┬───────────┬───────────┬─────────────┬─────────┐
12-
│ magic │ format_version │ record_typelsn │ tenant_id │ vshard_id │ payload_len │ crc32c │
13-
│ 4 bytes │ 2 bytes 2 bytes 8B │ 4 bytes │ 2 bytes │ 4 bytes │ 4 bytes │
14-
└─────────┴────────────────┴────────────┴─────┴───────────┴───────────┴─────────────┴─────────┘
11+
┌─────────┬────────────────┬────────────┬─────┬───────────┬───────────┬─────────────┬──────────┬─────────┐
12+
│ magic │ format_version │ record_typelsn │ tenant_id │ vshard_id │ payload_len │ reserved │ crc32c │
13+
│ 4 bytes│ 2 bytes │ 4 bytes 8B │ 8 bytes │ 4 bytes │ 4 bytes │ 16 bytes │ 4 bytes │
14+
└─────────┴────────────────┴────────────┴─────┴───────────┴───────────┴─────────────┴──────────┴─────────┘
1515
```
1616

17+
Byte offsets: magic 0..4, format_version 4..6, record_type 6..10, lsn 10..18, tenant_id 18..26, vshard_id 26..30, payload_len 30..34, reserved 34..50, crc32c 50..54. Total header: **54 bytes**.
18+
19+
Bit 14 of `record_type` signals AES-256-GCM encryption of the payload. The logical record type is `record_type & !(1 << 14)`.
20+
1721
- Page size: 4 KiB or 16 KiB (O_DIRECT alignment)
1822
- CRC32C per page for bit-rot detection
1923
- Segmented with auto-rollover

docs/sql/crdt-operations.rdx

Lines changed: 1 addition & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -23,13 +23,4 @@ Applies a Loro CRDT delta to the document. On Origin, the delta goes through Raf
2323

2424
## Conflict Policies
2525

26-
```sql
27-
-- View current policy
28-
SHOW CONFLICT POLICY ON notes;
29-
30-
-- Set policy
31-
ALTER COLLECTION notes SET conflict_policy = 'lww';
32-
ALTER COLLECTION profiles SET conflict_policy = 'field_merge';
33-
```
34-
35-
See [CRDT Sync](/docs/crdt-sync/overview) for full sync protocol details.
26+
Conflict policies (last-writer-wins, field-merge, custom) are configured per-collection at create time and via the native NodeDB protocol's `ALTER POLICY` command. SQL DDL for inspecting and updating conflict policies — `SHOW CONFLICT POLICY ON <collection>` and `ALTER COLLECTION ... SET ON CONFLICT ...` — is wired in the v0.1.0 release alongside the rest of the SQL surface. See the protocol reference for the native-protocol form, and [CRDT Sync](/docs/crdt-sync/overview) for full sync protocol details.

docs/sql/fulltext-search.rdx

Lines changed: 0 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -20,13 +20,6 @@ ORDER BY score DESC LIMIT 20;
2020
SELECT title FROM articles WHERE text_match(title, 'databse', { fuzzy: true, distance: 2 });
2121
```
2222

23-
## Synonyms
24-
25-
```sql
26-
CREATE SYNONYM GROUP db_terms AS ('database', 'db', 'datastore');
27-
-- Searching for 'db performance' now also matches 'database performance'
28-
```
29-
3023
## CJK Search
3124

3225
CJK text is automatically tokenized via character bigrams:

docs/sql/overview.rdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ NodeDB supports standard SQL with high PostgreSQL compatibility:
3434

3535
Beyond standard SQL, NodeDB extends the language for each engine:
3636

37-
- **Vector** — `SEARCH ... USING VECTOR()`, `vector_distance()`
37+
- **Vector** — `vector_distance()` with `<->` / `<=>` / `<#>` operators, ANN-aware `ORDER BY ... LIMIT k`
3838
- **Graph** — `GRAPH TRAVERSE`, `GRAPH ALGO`, `MATCH` patterns
3939
- **Full-Text** — `text_match()`, `bm25_score()`, `search_score()`
4040
- **Spatial** — `ST_DWithin()`, `ST_Contains()`, `ST_Distance()`, etc.

docs/sql/vector-search.rdx

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8,18 +8,20 @@ description: Nearest neighbor search, filtered vector search, and multi-vector q
88
## Nearest Neighbor
99

1010
```sql
11-
SEARCH articles USING VECTOR(embedding, ARRAY[0.1, 0.3, -0.2, ...], 10);
11+
SELECT id, vector_distance(embedding, ARRAY[0.1, 0.3, -0.2, ...]) AS dist
12+
FROM articles
13+
ORDER BY dist LIMIT 10;
1214
```
1315

14-
Returns the 10 nearest neighbors by the metric configured on the index (l2, cosine, inner_product, manhattan, chebyshev, hamming, jaccard, or pearson).
16+
Returns the 10 nearest neighbors by the metric configured on the index (l2, cosine, inner_product, manhattan, chebyshev, hamming, jaccard, or pearson). The planner detects `ORDER BY vector_distance(...) LIMIT k` and rewrites it into an ANN top-k plan; no special `SEARCH` keyword is required.
1517

1618
## Filtered Vector Search
1719

1820
```sql
1921
SELECT title, vector_distance(embedding, ARRAY[0.1, 0.3, ...]) AS score
2022
FROM articles
2123
WHERE category = 'machine-learning'
22-
AND id IN (SEARCH articles USING VECTOR(embedding, ARRAY[0.1, 0.3, ...], 10));
24+
ORDER BY score LIMIT 10;
2325
```
2426

2527
The engine builds a Roaring Bitmap of matching IDs and selects the optimal strategy: pre-filter (selective filters), post-filter (broad filters), or brute-force (very selective).

docs/storage-engines/array.rdx

Lines changed: 17 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,7 @@ CREATE ARRAY spatial_grid
6060
| `cell_order` | No | `'Z-ORDER'` | `'Z-ORDER'` (Hilbert curve) or `'ROW-MAJOR'`. Affects spatial cache locality. |
6161
| `audit_retain_ms` | No | `NULL` | Tiles older than `now - audit_retain_ms` (system time) become eligible for purge. `NULL` = keep all. |
6262

63-
`ALTER NDARRAY <name> SET (audit_retain_ms = ...)` updates retention; `DROP ARRAY <name>` is two-phase like `DROP COLLECTION`.
63+
`ALTER ARRAY <name> SET (audit_retain_ms = ...)` updates retention; `DROP ARRAY <name>` is two-phase like `DROP COLLECTION`.
6464

6565
## Insert
6666

@@ -79,17 +79,17 @@ INSERT INTO ARRAY elevation_map (lon, lat, height) VALUES
7979
(-73.7, 40.6, 8.9);
8080

8181
-- Force the in-memory tiles to durable storage
82-
SELECT NDARRAY_FLUSH('elevation_map');
82+
SELECT ARRAY_FLUSH('elevation_map');
8383
```
8484

8585
## Query Functions
8686

8787
Array queries are expressed as table-valued functions in `FROM`. System time and valid time apply via `AS OF` clauses.
8888

89-
### `NDARRAY_SLICE` — multi-dimensional range
89+
### `ARRAY_SLICE` — multi-dimensional range
9090

9191
```sql
92-
SELECT * FROM NDARRAY_SLICE(
92+
SELECT * FROM ARRAY_SLICE(
9393
'elevation_map',
9494
{lon: [-74.0, -73.0), lat: [40.0, 41.0)},
9595
['height'], -- attribute projection (optional)
@@ -104,37 +104,37 @@ SELECT * FROM NDARRAY_SLICE(
104104
| `attrs` | No | `ARRAY[STRING]`| Attributes to project. `NULL` = all attributes. |
105105
| `limit` | No | `INT64` | Max cells returned. `NULL` = no limit. |
106106

107-
### `NDARRAY_PROJECT` — attribute projection
107+
### `ARRAY_PROJECT` — attribute projection
108108

109109
```sql
110-
SELECT * FROM NDARRAY_PROJECT('spatial_grid', ['temperature', 'pressure']);
110+
SELECT * FROM ARRAY_PROJECT('spatial_grid', ['temperature', 'pressure']);
111111
```
112112

113-
### `NDARRAY_AGG` — reduce a dimension
113+
### `ARRAY_AGG` — reduce a dimension
114114

115115
Aggregates an attribute over a dimension, reducing dimensionality:
116116

117117
```sql
118118
-- Sum temperature over x; result keeps y and z
119-
SELECT * FROM NDARRAY_AGG('spatial_grid', 'temperature', 'SUM', 'x');
119+
SELECT * FROM ARRAY_AGG('spatial_grid', 'temperature', 'SUM', 'x');
120120
```
121121

122122
Reducers: `'SUM'`, `'AVG'`, `'MIN'`, `'MAX'`, `'COUNT'`.
123123

124-
### `NDARRAY_ELEMENTWISE` — between two arrays of the same shape
124+
### `ARRAY_ELEMENTWISE` — between two arrays of the same shape
125125

126126
```sql
127-
SELECT * FROM NDARRAY_ELEMENTWISE('current_grid', 'baseline_grid', 'SUBTRACT', 'temperature');
127+
SELECT * FROM ARRAY_ELEMENTWISE('current_grid', 'baseline_grid', 'SUBTRACT', 'temperature');
128128
```
129129

130130
## Maintenance
131131

132132
```sql
133-
SELECT NDARRAY_FLUSH('spatial_grid'); -- force memtable flush
134-
SELECT NDARRAY_COMPACT('spatial_grid'); -- merge tile versions, reclaim space
133+
SELECT ARRAY_FLUSH('spatial_grid'); -- force memtable flush
134+
SELECT ARRAY_COMPACT('spatial_grid'); -- merge tile versions, reclaim space
135135
```
136136

137-
`NDARRAY_FLUSH` always returns `{result: true}` on success; failure raises. Compaction also runs automatically in the background.
137+
`ARRAY_FLUSH` always returns `{result: true}` on success; failure raises. Compaction also runs automatically in the background.
138138

139139
## Bitemporal Queries
140140

@@ -145,15 +145,15 @@ Every array cell carries two times:
145145

146146
```sql
147147
-- Read cells as the array existed in the past
148-
SELECT * FROM NDARRAY_SLICE('data', {x: [0, 100), y: [0, 100)}, ['value'])
148+
SELECT * FROM ARRAY_SLICE('data', {x: [0, 100), y: [0, 100)}, ['value'])
149149
AS OF SYSTEM TIME 1700000000000;
150150

151151
-- Read cells whose valid-time interval includes a given moment
152-
SELECT * FROM NDARRAY_SLICE('forecast', {x: [0, 100), y: [0, 100)}, ['temp'])
152+
SELECT * FROM ARRAY_SLICE('forecast', {x: [0, 100), y: [0, 100)}, ['temp'])
153153
AS OF VALID TIME 1700000000000;
154154

155155
-- Both clauses combined
156-
SELECT * FROM NDARRAY_SLICE('forecast', {x: [0, 100), y: [0, 100)}, ['temp'])
156+
SELECT * FROM ARRAY_SLICE('forecast', {x: [0, 100), y: [0, 100)}, ['temp'])
157157
AS OF SYSTEM TIME 1700000000000 AS OF VALID TIME 1700000001000;
158158
```
159159

@@ -165,7 +165,7 @@ Array cells participate in surrogate-identity bitmaps with the rest of the engin
165165

166166
```sql
167167
SELECT *
168-
FROM NDARRAY_SLICE('spatial_data', {x: [0, 1000), y: [0, 1000)}, ['attr1', 'attr2'])
168+
FROM ARRAY_SLICE('spatial_data', {x: [0, 1000), y: [0, 1000)}, ['attr1', 'attr2'])
169169
WHERE id IN (
170170
SEARCH vectors USING VECTOR(embedding, $query, 100)
171171
);

docs/temporal/bitemporal.rdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -107,7 +107,7 @@ CREATE ARRAY climate_grid
107107
WITH (audit_retain_ms = 7776000000); -- 90 days
108108

109109
-- Cells as committed yesterday
110-
SELECT lon, lat, temp_c FROM NDARRAY_SLICE(
110+
SELECT lon, lat, temp_c FROM ARRAY_SLICE(
111111
'climate_grid',
112112
{lon: [-10, 10), lat: [0, 20)},
113113
['temp_c']

0 commit comments

Comments
 (0)