Skip to content

Commit 2c03a7a

Browse files
committed
feat(array): add sparse N-dimensional array engine
Introduces a bi-temporal sparse array engine backed by the nodedb-array crate. Coordinates and payloads are arbitrary-dimensional; valid-time bounds are carried per-cell and system-time history is preserved so point-in-time reads and slice queries work at any past timestamp. Core library (nodedb-lite): - `engine/array/` — engine, catalog, memtable, segment, manifest, and retention modules - `nodedb/array.rs` — `NodeDbLite` public API methods (create, put_cell, slice, read_coord, delete_cell, flush, history, compact) - Integration test suite in `tests/array_lite.rs` C/JNI FFI (nodedb-lite-ffi): - `ffi_array.rs` — full C API: ndb_array_create, ndb_array_put_cell, ndb_array_slice, ndb_array_read_coord, ndb_array_delete_cell, ndb_array_flush, ndb_array_history, ndb_array_compact - `nodedb_free_buf` added to lib.rs and the cbindgen header - Smoke tests in `tests/array_smoke.rs` WASM (nodedb-lite-wasm): - `src/array.rs` — wasm-bindgen bindings mirroring the C API surface - Tests in `tests/array.rs` Adds `nodedb-array` and `zerompk` to all three crate manifests; adds `getrandom` with the `wasm_js` feature to the WASM crate to satisfy the `getrandom 0.3` requirement on `wasm32-unknown-unknown`. Adjusts the tokio feature set in the core crate: library code no longer needs `rt-multi-thread`; tests opt in via dev-dependencies.
1 parent 019ef8c commit 2c03a7a

22 files changed

Lines changed: 3508 additions & 2 deletions

File tree

Cargo.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ nodedb-fts = { version = "0.0.6" }
3030
nodedb-strict = { version = "0.0.6" }
3131
nodedb-columnar = { version = "0.0.6" }
3232
nodedb-sql = { version = "0.0.6" }
33+
nodedb-array = { version = "0.0.6" }
3334

3435
# Async
3536
tokio = { version = "1" }

nodedb-lite-ffi/Cargo.toml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,8 @@ crate-type = ["lib", "staticlib", "cdylib"]
1616
nodedb-lite = { workspace = true }
1717
nodedb-client = { workspace = true }
1818
nodedb-types = { workspace = true }
19+
nodedb-array = { workspace = true }
20+
zerompk = { workspace = true }
1921
serde_json = { workspace = true }
2022
sonic-rs = { workspace = true }
2123
tokio = { workspace = true, features = ["rt-multi-thread"] }

nodedb-lite-ffi/include/nodedb_lite.h

Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -107,6 +107,128 @@ int32_t nodedb_generate_id_typed(const char *id_type, char **out);
107107
*/
108108
void nodedb_free_string(char *ptr);
109109

110+
/**
111+
* Free a byte buffer returned by nodedb_* functions (e.g. `ndb_array_slice`).
112+
*
113+
* `len` must be the exact length originally written to `*out_len`.
114+
*
115+
* # Safety
116+
* `ptr` must be a buffer previously returned by a nodedb function, or NULL.
117+
*/
118+
void nodedb_free_buf(uint8_t *ptr, uintptr_t len);
119+
120+
/**
121+
* Create a new ND sparse array.
122+
*
123+
* `schema_msgpack` — zerompk-encoded `ArraySchema`.
124+
* `dims_msgpack` — reserved for future use; pass NULL / 0 today.
125+
*
126+
* Returns `NODEDB_OK` on success, `NODEDB_ERR_*` on failure.
127+
*
128+
* # Safety
129+
* All pointer parameters must be valid. `name` must be a null-terminated UTF-8 string.
130+
*/
131+
int32_t ndb_array_create(struct NodeDbNodeDbHandle *handle,
132+
const char *name,
133+
const uint8_t *schema_msgpack,
134+
uintptr_t schema_len);
135+
136+
/**
137+
* Write a cell into array `name` at `coord`.
138+
*
139+
* `coord_msgpack` — zerompk-encoded `Vec<CoordValue>`.
140+
* `payload_msgpack` — zerompk-encoded `Vec<CellValue>` (attribute values).
141+
* `valid_from_ms` — valid-time lower bound (milliseconds since epoch).
142+
* `valid_until_ms` — valid-time upper bound (`i64::MAX` = open-ended).
143+
*
144+
* # Safety
145+
* All pointer parameters must be valid. `name` must be a null-terminated UTF-8 string.
146+
*/
147+
int32_t ndb_array_put_cell(struct NodeDbNodeDbHandle *handle,
148+
const char *name,
149+
const uint8_t *coord_msgpack,
150+
uintptr_t coord_len,
151+
const uint8_t *payload_msgpack,
152+
uintptr_t payload_len,
153+
int64_t valid_from_ms,
154+
int64_t valid_until_ms);
155+
156+
/**
157+
* Slice query: return all live cells whose coordinates fall within `ranges`.
158+
*
159+
* `ranges_msgpack` — zerompk-encoded `Vec<Option<DimRange>>`.
160+
* `as_of_system_ms` — system-time AS-OF (ignored when `has_as_of == 0`).
161+
* `has_as_of` — set to 1 to use `as_of_system_ms`, 0 for current live snapshot.
162+
*
163+
* On success `*out_buf` points to a zerompk-encoded `Vec<CellPayload>`.
164+
* Caller must free with `nodedb_free_buf`.
165+
*
166+
* # Safety
167+
* All pointer parameters must be valid. `name` must be a null-terminated UTF-8 string.
168+
* `out_buf` and `out_len` must not be null.
169+
*/
170+
int32_t ndb_array_slice(struct NodeDbNodeDbHandle *handle,
171+
const char *name,
172+
const uint8_t *ranges_msgpack,
173+
uintptr_t ranges_len,
174+
int64_t as_of_system_ms,
175+
uint8_t has_as_of,
176+
uint8_t **out_buf,
177+
uintptr_t *out_len);
178+
179+
/**
180+
* Read the most recent live payload for `coord` at or before `as_of_system_ms`.
181+
*
182+
* `coord_msgpack` — zerompk-encoded `Vec<CoordValue>`.
183+
* `has_as_of` — set to 1 to use `as_of_system_ms`, 0 for current live snapshot.
184+
*
185+
* On success and if a cell exists, `*out_buf` points to a zerompk-encoded
186+
* `CellPayload` (length > 0). If the cell does not exist, `*out_len` is 0
187+
* and `*out_buf` is NULL.
188+
* Caller must free with `nodedb_free_buf` when `*out_len > 0`.
189+
*
190+
* # Safety
191+
* All pointer parameters must be valid. `name` must be a null-terminated UTF-8 string.
192+
* `out_buf` and `out_len` must not be null.
193+
*/
194+
int32_t ndb_array_read_coord(struct NodeDbNodeDbHandle *handle,
195+
const char *name,
196+
const uint8_t *coord_msgpack,
197+
uintptr_t coord_len,
198+
int64_t as_of_system_ms,
199+
uint8_t has_as_of,
200+
uint8_t **out_buf,
201+
uintptr_t *out_len);
202+
203+
/**
204+
* Soft-delete a cell by writing a tombstone at the current system time.
205+
*
206+
* `coord_msgpack` — zerompk-encoded `Vec<CoordValue>`.
207+
*
208+
* # Safety
209+
* All pointer parameters must be valid. `name` must be a null-terminated UTF-8 string.
210+
*/
211+
int32_t ndb_array_delete_cell(struct NodeDbNodeDbHandle *handle,
212+
const char *name,
213+
const uint8_t *coord_msgpack,
214+
uintptr_t coord_len);
215+
216+
/**
217+
* GDPR erasure: permanently remove cell content at `coord`.
218+
*
219+
* `coord_msgpack` — zerompk-encoded `Vec<CoordValue>`.
220+
*
221+
* After this call `ndb_array_read_coord` returns empty for the coordinate
222+
* at any system time >= the erasure system timestamp.
223+
*
224+
* # Safety
225+
* All pointer parameters must be valid. `name` must be a null-terminated UTF-8 string.
226+
*/
227+
int32_t ndb_array_gdpr_erase_cell(struct NodeDbNodeDbHandle *handle,
228+
const char *name,
229+
const uint8_t *coord_msgpack,
230+
uintptr_t coord_len);
231+
110232
/**
111233
* Get a document by ID. Result written as JSON to `out_json`.
112234
*

0 commit comments

Comments
 (0)