Skip to content

Commit 107e53a

Browse files
pioner92pioner92
committed
chore: update README
1 parent 24f9293 commit 107e53a

1 file changed

Lines changed: 112 additions & 16 deletions

File tree

‎README.md‎

Lines changed: 112 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -2,11 +2,56 @@
22

33
# react-native-columnar
44

5-
A utility for high-performance data transport from JSI C++ to JavaScript.
5+
High-performance columnar `ArrayBuffer` transport from JSI C++ to JavaScript.
66

7-
JSI native modules typically return data as an array of objects — one JS object per row, with a key for every field. This works fine for small amounts of data, but becomes slow at scale: the JS engine has to allocate thousands of objects, box every value, and put pressure on the GC.
7+
JSI modules often return large datasets as arrays of objects. That is easy to use, but expensive at scale: every row becomes a JS object, every value is boxed, and the result puts pressure on the GC.
88

9-
`react-native-columnar` replaces the array of objects with a single binary `ArrayBuffer` in columnar layout. The C++ side writes all values directly into a pre-allocated buffer, passes it to JS via JSI with zero copies, and the JS side reads it through typed array views (`Int32Array`, `Float64Array`, etc.) over the same memory — no allocation, no parsing, no overhead.
9+
`react-native-columnar` writes fixed-width native values into one binary columnar buffer and exposes it to JS as typed array views (`Int32Array`, `Float64Array`, etc.). No per-row objects, no parsing, and no payload copy.
10+
11+
---
12+
13+
## Best use cases
14+
15+
- SQLite result sets
16+
- Frame processor outputs
17+
- Sensor streams
18+
- Analytics events
19+
- Game / physics data
20+
- Large JSI payloads
21+
- Realtime charts
22+
23+
---
24+
25+
## Benchmark
26+
27+
**Test:** 10 000 iterations — each call transfers N rows (5 columns) from C++ to JS and reads one row.
28+
29+
```
30+
Schema: id (int32) | status (uint8) | isActive (uint8) | createdAt (double) | updatedAt (double)
31+
Iterations: 10 000
32+
```
33+
34+
| Rows | Array of objects | react-native-columnar | Speedup |
35+
|------|------------------|-----------------------|----------|
36+
| 100 | ~418.81 ms | **~14.96 ms** | **27×** |
37+
| 500 | ~2079.81 ms | **~22.06 ms** | **94×** |
38+
| 1000 | ~4360.11 ms | **~35.89 ms** | **121×** |
39+
| 2000 | ~9444.47 ms | **~45.39 ms** | **208×** |
40+
41+
**Array of objects** — each call allocates a JS array of objects with 5 keys each, boxes every value, and puts pressure on the GC — multiplied across 10 000 iterations.
42+
43+
**react-native-columnar** — one binary buffer is allocated in C++, all rows are written in a single loop, and the buffer pointer is handed to the JS engine as an `ArrayBuffer`. The JS side creates five typed array views (`Int32Array`, `Uint8Array`, `Float64Array`) over the same memory — **zero copies, zero parsing, no per-row object allocation**.
44+
45+
> Measured on iPhone 16 Pro. Results will vary by device and data shape.
46+
47+
---
48+
49+
## Requirements
50+
51+
- React Native with JSI native modules
52+
- C++20 or newer (`std::span` is used by the C++ helper)
53+
- iOS via CocoaPods or Android via CMake
54+
- Fixed-width numeric data (`int8_t`, `uint32_t`, `double`, etc.)
1055

1156
---
1257

@@ -71,6 +116,13 @@ The binary layout:
71116
──── 8-byte header ──── ──────────── data, 8-byte aligned ───────
72117
```
73118

119+
The contract is intentionally small:
120+
121+
- The JS schema must match the C++ schema exactly.
122+
- Column order and type width must be the same on both sides.
123+
- Each column stores one fixed-width primitive type.
124+
- The buffer owns the payload; JS typed arrays are views over that same memory.
125+
74126
---
75127

76128
## C++ side
@@ -164,6 +216,19 @@ const updatedAt = updatedAtColumn[rowIndex];
164216

165217
`createBufferReader` returns zero-copy typed array views — the `ArrayBuffer` is not copied.
166218

219+
### Reader API
220+
221+
```ts
222+
createBufferReader<TSchema extends readonly ColumnType[]>(
223+
buffer: ArrayBuffer,
224+
schema: TSchema
225+
): [header: Int32Array, columns: ColumnsResult<TSchema>]
226+
```
227+
228+
- `header[0]` — row count
229+
- `header[1]` — column count
230+
- `columns` — typed array views inferred from the schema order
231+
167232
### ColumnType mapping
168233

169234
| `ColumnType` | C++ type | JS typed array | Bytes | Align | Tip |
@@ -179,27 +244,58 @@ const updatedAt = updatedAtColumn[rowIndex];
179244

180245
---
181246

182-
## Benchmark
247+
## Limitations
183248

184-
**Test:** 10 000 iterations — each call transfers N rows (5 columns) from C++ to JS and reads one row.
249+
`react-native-columnar` is optimized for dense numeric payloads. It does not encode
250+
strings, nested objects, nullable values, or variable-length fields by itself.
185251

252+
For those cases, keep metadata separately or encode it into fixed-width columns
253+
with your own conventions, such as enum ids, offsets, masks, or sentinel values.
254+
255+
---
256+
257+
## Troubleshooting
258+
259+
### `react-native-columnar.h` not found
260+
261+
Make sure the package is added to your native build and linked to your JSI library.
262+
263+
On Android, check that `react-native-columnar/android` is added via `add_subdirectory`
264+
and that your native target links against `react-native-columnar`:
265+
266+
```cmake
267+
target_link_libraries(${YOUR_LIBRARY_NAME} react-native-columnar)
186268
```
187-
Schema: id (int32) | status (uint8) | isActive (uint8) | createdAt (double) | updatedAt (double)
188-
Iterations: 10 000
269+
270+
On iOS, make sure CocoaPods has been installed after adding the package:
271+
272+
```sh
273+
cd ios && pod install
189274
```
190275

191-
| Rows | Array of objects | react-native-columnar | Speedup |
192-
|------|------------------|-----------------------|----------|
193-
| 100 | ~418.81 ms | **~14.96 ms** | **27×** |
194-
| 500 | ~2079.81 ms | **~22.06 ms** | **94×** |
195-
| 1000 | ~4360.11 ms | **~35.89 ms** | **121×** |
196-
| 2000 | ~9444.47 ms | **~45.39 ms** | **208×** |
276+
### `std::span` is not available
197277

198-
**Array of objects** — each call allocates a JS array of objects with 5 keys each, boxes every value, and puts pressure on the GC — multiplied across 10 000 iterations.
278+
The C++ helper uses `std::span`, so your native target must compile with C++20 or
279+
newer. If your build fails with errors around `std::span`, enable C++20 for the
280+
target that includes `react-native-columnar.h`.
199281

200-
**react-native-columnar** — one binary buffer is allocated in C++, all rows are written in a single loop, and the buffer pointer is handed to the JS engine as an `ArrayBuffer`. The JS side creates five typed array views (`Int32Array`, `Uint8Array`, `Float64Array`) over the same memory — **zero copies, zero parsing, zero object allocation**.
282+
### `RangeError` while creating typed arrays
201283

202-
> Measured on iPhone 16 Pro. Results will vary by device and data shape.
284+
This usually means the JavaScript schema does not match the native schema, or the
285+
buffer was not created by `ColumnarWriterBuilder`.
286+
287+
Check that:
288+
289+
- The JS `ColumnType` list has the same order as the C++ schema.
290+
- Each JS type matches the exact C++ type size (`int32_t` → `ColumnType.Int32`,
291+
`double` → `ColumnType.Float64`, etc.).
292+
- The buffer is the `ArrayBuffer` returned by `writer.toArrayBuffer(rt)`.
293+
294+
### Values look shifted or incorrect
295+
296+
The reader and writer must agree on both column order and type width. A single
297+
wrong type can shift all following columns. Start by comparing the C++ schema with
298+
the JS schema line by line.
203299

204300
---
205301

0 commit comments

Comments
 (0)