Skip to content

Commit f7f141d

Browse files
committed
chore: updated README
1 parent 30bd221 commit f7f141d

1 file changed

Lines changed: 59 additions & 201 deletions

File tree

README.md

Lines changed: 59 additions & 201 deletions
Original file line numberDiff line numberDiff line change
@@ -2,33 +2,18 @@
22

33
# react-native-columnar
44

5-
High-performance columnar `ArrayBuffer` transport from JSI C++ to JavaScript.
5+
Zero-copy columnar `ArrayBuffer` transport from JSI C++ to JavaScript.
66

7-
JSI modules often return large datasets as arrays of objects. That is easy to use, but expensive at scale: every row becomes a JS object, every value is boxed, and the result puts pressure on the GC.
8-
9-
`react-native-columnar` writes fixed-width native values into one binary columnar buffer and exposes it to JS as typed array views (`Int32Array`, `Float64Array`, etc.). No per-row objects, no parsing, and no payload copy.
7+
JSI modules often return datasets as arrays of objects — every row becomes a JS object, every value gets boxed, GC pressure grows. `react-native-columnar` packs all values into one binary buffer and exposes each column as a typed array view over the same memory. No objects, no parsing, no copy.
108

119
---
1210

13-
## Best use cases
14-
15-
- SQLite result sets
16-
- Frame processor outputs
17-
- Sensor streams
18-
- Analytics events
19-
- Game / physics data
20-
- Large JSI payloads
21-
- Realtime charts
22-
23-
---
11+
## ⚡ Benchmark
2412

25-
## Benchmark
26-
27-
**Test:** 10 000 iterations — each call transfers N rows (5 columns) from C++ to JS and reads one row.
13+
10 000 iterations · 5 columns · iPhone 16 Pro
2814

2915
```
30-
Schema: id (int32) | status (uint8) | isActive (uint8) | createdAt (double) | updatedAt (double)
31-
Iterations: 10 000
16+
id (int32) | status (uint8) | isActive (uint8) | createdAt (double) | updatedAt (double)
3217
```
3318

3419
| Rows | Array of objects | react-native-columnar | Speedup |
@@ -38,24 +23,15 @@ Iterations: 10 000
3823
| 1000 | ~4360.11 ms | **~35.89 ms** | **121×** |
3924
| 2000 | ~9444.47 ms | **~45.39 ms** | **208×** |
4025

41-
**Array of objects** — each call allocates a JS array of objects with 5 keys each, boxes every value, and puts pressure on the GC — multiplied across 10 000 iterations.
42-
43-
**react-native-columnar** — one binary buffer is allocated in C++, all rows are written in a single loop, and the buffer pointer is handed to the JS engine as an `ArrayBuffer`. The JS side creates five typed array views (`Int32Array`, `Uint8Array`, `Float64Array`) over the same memory — **zero copies, zero parsing, no per-row object allocation**.
44-
45-
> Measured on iPhone 16 Pro. Results will vary by device and data shape.
46-
4726
---
4827

49-
## Requirements
28+
## 🎯 Best use cases
5029

51-
- React Native with JSI native modules
52-
- C++20 or newer (`std::span` is used by the C++ helper)
53-
- iOS via CocoaPods or Android via CMake
54-
- Fixed-width numeric data (`int8_t`, `uint32_t`, `double`, etc.)
30+
SQLite result sets · Frame processor outputs · Sensor streams · Analytics events · Realtime charts · Large JSI payloads
5531

5632
---
5733

58-
## Installation
34+
## 📦 Installation
5935

6036
```sh
6137
npm install react-native-columnar
@@ -65,99 +41,49 @@ yarn add react-native-columnar
6541

6642
**iOS** — headers are picked up automatically via CocoaPods.
6743

68-
**Android — inside an app project**
44+
**Android — app project**
6945

70-
Autolinking registers the package automatically. Enable Prefab in your `android/app/build.gradle`:
46+
Autolinking registers the package automatically. Add to `android/app/build.gradle`:
7147

7248
```groovy
7349
android {
74-
buildFeatures {
75-
prefab true
76-
}
50+
buildFeatures { prefab true }
7751
}
7852
```
7953

80-
Then in your `CMakeLists.txt`:
54+
Then in `CMakeLists.txt`:
8155

8256
```cmake
8357
find_package(react-native-columnar REQUIRED CONFIG)
84-
85-
target_link_libraries(
86-
${YOUR_LIBRARY_NAME}
87-
react-native-columnar::react-native-columnar
88-
)
58+
target_link_libraries(${YOUR_LIBRARY_NAME} react-native-columnar::react-native-columnar)
8959
```
9060

91-
**Android — inside a standalone library**
92-
93-
Autolinking does not run in library projects. Use `add_subdirectory` instead — it resolves headers directly from `node_modules` without needing a Gradle dependency.
61+
**Android — standalone library**
9462

95-
Make sure `NODE_MODULES_DIR` is passed from your `build.gradle` (any JSI library already does this):
63+
Add to `package.json`:
9664

97-
```groovy
98-
externalNativeBuild {
99-
cmake {
100-
arguments "-DNODE_MODULES_DIR=${nodeModules}"
101-
}
102-
}
65+
```json
66+
{ "dependencies": { "react-native-columnar": "*" } }
10367
```
10468

105-
Then in your `CMakeLists.txt`:
69+
Then in `CMakeLists.txt` (`NODE_MODULES_DIR` is already passed by any JSI library):
10670

10771
```cmake
108-
if(NOT TARGET react-native-columnar)
109-
add_subdirectory(
110-
${NODE_MODULES_DIR}/react-native-columnar/android
111-
${CMAKE_BINARY_DIR}/react-native-columnar
112-
)
113-
endif()
114-
115-
target_link_libraries(${YOUR_LIBRARY_NAME} react-native-columnar)
72+
include_directories(${NODE_MODULES_DIR}/react-native-columnar/cpp)
11673
```
11774

118-
Then in your C++ files:
75+
In your C++ files:
11976

12077
```cpp
12178
#include "react-native-columnar.h"
12279
```
12380

12481
---
12582

126-
## How it works
127-
128-
```
129-
C++ (JSI) JavaScript
130-
────────────────────────────── ──────────────────────────────
131-
ColumnarWriterBuilder createBufferReader()
132-
→ allocates buffer → wraps buffer with typed views
133-
→ writes columns (int32/ → Int32Array / Float64Array /
134-
double / uint8) by value Uint8Array pointing into the
135-
→ zero-copy ArrayBuffer same memory — no copy
136-
transfer via JSI
137-
```
138-
139-
The binary layout:
140-
141-
```
142-
[ rows: u32 | columns: u32 ][ col0 data ][ padding ][ col1 data ] ...
143-
──── 8-byte header ──── ──────────── data, 8-byte aligned ───────
144-
```
145-
146-
The contract is intentionally small:
147-
148-
- The JS schema must match the C++ schema exactly.
149-
- Column order and type width must be the same on both sides.
150-
- Each column stores one fixed-width primitive type.
151-
- The buffer owns the payload; JS typed arrays are views over that same memory.
152-
153-
---
154-
155-
## C++ side
83+
## 🔧 C++ side
15684

15785
### 1. Define a schema
15886

159-
Use `DECLARE_BINARY_SCHEMA` to declare a schema. The second argument is an X-macro that lists `(CppType, fieldName)` pairs:
160-
16187
```cpp
16288
#include "react-native-columnar.h"
16389

@@ -171,13 +97,11 @@ Use `DECLARE_BINARY_SCHEMA` to declare a schema. The second argument is an X-mac
17197
DECLARE_BINARY_SCHEMA(UserSchema, USER_COLUMNS)
17298
```
17399
174-
This generates a `UserSchema` struct with `columnCount`, `byteSize()`, and a `Columns` struct holding `std::span` views for each field.
100+
Generates `UserSchema` with `columnCount`, `byteSize()`, and a `Columns` struct of `std::span` views.
175101
176-
### 2. Write data and return an ArrayBuffer
102+
### 2. Write and return an ArrayBuffer
177103
178104
```cpp
179-
#include "react-native-columnar.h"
180-
181105
jsi::Value getUsers(jsi::Runtime& rt, const jsi::Value*, const jsi::Value* args, size_t) {
182106
const uint32_t rows = static_cast<uint32_t>(args[0].asNumber());
183107
@@ -192,19 +116,17 @@ jsi::Value getUsers(jsi::Runtime& rt, const jsi::Value*, const jsi::Value* args,
192116
cols.updatedAt[i] = getUser(i).updatedAt;
193117
}
194118
195-
return writer.toArrayBuffer(rt);
119+
return writer.toArrayBuffer(rt); // zero-copy move into JSI
196120
}
197121
```
198122

199-
`toArrayBuffer` moves the buffer into a JSI `ArrayBuffer`**no copy**.
200-
201123
---
202124

203-
## JS side
125+
## 🟦 JS side
204126

205-
### Read the buffer
127+
### 1. Define the schema
206128

207-
Define the schema once — it must match the column order and types declared in C++:
129+
Must match column order and types from C++:
208130

209131
```ts
210132
import { createBufferReader, ColumnType } from 'react-native-columnar';
@@ -218,126 +140,62 @@ const USER_SCHEMA = [
218140
] as const;
219141
```
220142

221-
Then call it on every buffer you receive from JSI:
143+
### 2. Read the buffer
222144

223145
```ts
224-
const buffer: ArrayBuffer = __getUsers(); // your JSI function
146+
const buffer: ArrayBuffer = __getUsers();
225147

226148
const [header, columns] = createBufferReader(buffer, USER_SCHEMA);
227-
const [idColumn, statusColumn, isActiveColumn, createdAtColumn, updatedAtColumn] = columns;
228-
229-
// Each column is a zero-copy typed array view into the original buffer:
230-
// idColumn — Int32Array
231-
// statusColumn — Uint8Array
232-
// isActiveColumn — Uint8Array
233-
// createdAtColumn — Float64Array
234-
// updatedAtColumn — Float64Array
235-
236-
const rowIndex = 0;
237-
const id = idColumn[rowIndex];
238-
const status = statusColumn[rowIndex];
239-
const isActive = isActiveColumn[rowIndex];
240-
const createdAt = createdAtColumn[rowIndex];
241-
const updatedAt = updatedAtColumn[rowIndex];
149+
const [idCol, statusCol, isActiveCol, createdAtCol, updatedAtCol] = columns;
150+
// idCol — Int32Array | statusCol — Uint8Array | createdAtCol — Float64Array
151+
152+
const id = idCol[0];
153+
const status = statusCol[0];
154+
const isActive = isActiveCol[0];
155+
const createdAt = createdAtCol[0];
156+
const updatedAt = updatedAtCol[0];
242157
```
243158

244-
`createBufferReader` returns zero-copy typed array views — the `ArrayBuffer` is not copied.
159+
All columns are zero-copy typed array views — the buffer is never copied.
245160

246-
### Reader API
161+
### API
247162

248163
```ts
249-
createBufferReader<TSchema extends readonly ColumnType[]>(
250-
buffer: ArrayBuffer,
251-
schema: TSchema
252-
): [header: Int32Array, columns: ColumnsResult<TSchema>]
164+
createBufferReader(buffer: ArrayBuffer, schema: readonly ColumnType[])
165+
// → [header: Int32Array, columns: TypedArray[]]
166+
// header[0] = row count, header[1] = column count
253167
```
254168

255-
- `header[0]` — row count
256-
- `header[1]` — column count
257-
- `columns` — typed array views inferred from the schema order
258-
259169
### ColumnType mapping
260170

261-
| `ColumnType` | C++ type | JS typed array | Bytes | Align | Tip |
262-
|------------------------|-------------|-------------------|-------|-------|----------------------------------|
263-
| `ColumnType.Int8` | `int8_t` | `Int8Array` | 1 | 1 | |
264-
| `ColumnType.Uint8` | `uint8_t` | `Uint8Array` | 1 | 1 | bool, flags |
265-
| `ColumnType.Int16` | `int16_t` | `Int16Array` | 2 | 2 | |
266-
| `ColumnType.Uint16` | `uint16_t` | `Uint16Array` | 2 | 2 | |
267-
| `ColumnType.Int32` | `int32_t` | `Int32Array` | 4 | 4 | integers — id, count, enum |
268-
| `ColumnType.Uint32` | `uint32_t` | `Uint32Array` | 4 | 4 | |
269-
| `ColumnType.Float32` | `float` | `Float32Array` | 4 | 4 | ratio, normalized value, screen coord (~7 sig. digits) |
270-
| `ColumnType.Float64` | `double` | `Float64Array` | 8 | 8 | timestamp (ms), price, high-precision decimals |
171+
| `ColumnType` | C++ type | JS view | Bytes | Tip |
172+
|-------------------|-------------|----------------|-------|-------------------------------|
173+
| `Int8` | `int8_t` | `Int8Array` | 1 | |
174+
| `Uint8` | `uint8_t` | `Uint8Array` | 1 | bool, flags |
175+
| `Int16` | `int16_t` | `Int16Array` | 2 | |
176+
| `Uint16` | `uint16_t` | `Uint16Array` | 2 | |
177+
| `Int32` | `int32_t` | `Int32Array` | 4 | id, count, enum |
178+
| `Uint32` | `uint32_t` | `Uint32Array` | 4 | |
179+
| `Float32` | `float` | `Float32Array` | 4 | screen coords (~7 sig. digits)|
180+
| `Float64` | `double` | `Float64Array` | 8 | timestamp, price |
271181

272182
---
273183

274-
## Limitations
184+
## ⚠️ Limitations
275185

276-
`react-native-columnar` is optimized for dense numeric payloads. It does not encode
277-
strings, nested objects, nullable values, or variable-length fields by itself.
278-
279-
For those cases, keep metadata separately or encode it into fixed-width columns
280-
with your own conventions, such as enum ids, offsets, masks, or sentinel values.
186+
Designed for dense numeric data only. Strings, nullable values, nested objects, and variable-length fields are not supported natively — encode them as fixed-width columns using ids, offsets, or sentinel values.
281187

282188
---
283189

284-
## Troubleshooting
285-
286-
### `react-native-columnar.h` not found
287-
288-
Make sure the package is added to your native build and linked to your JSI library.
289-
290-
On Android, check that Prefab is enabled in the Gradle module that builds your
291-
native target:
292-
293-
```groovy
294-
android {
295-
buildFeatures {
296-
prefab true
297-
}
298-
}
299-
```
300-
301-
Then make sure CMake finds and links the Prefab package:
302-
303-
```cmake
304-
find_package(react-native-columnar REQUIRED CONFIG)
305-
306-
target_link_libraries(
307-
${YOUR_LIBRARY_NAME}
308-
react-native-columnar::react-native-columnar
309-
)
310-
```
311-
312-
On iOS, make sure CocoaPods has been installed after adding the package:
313-
314-
```sh
315-
cd ios && pod install
316-
```
317-
318-
### `std::span` is not available
319-
320-
The C++ helper uses `std::span`, so your native target must compile with C++20 or
321-
newer. If your build fails with errors around `std::span`, enable C++20 for the
322-
target that includes `react-native-columnar.h`.
323-
324-
### `RangeError` while creating typed arrays
325-
326-
This usually means the JavaScript schema does not match the native schema, or the
327-
buffer was not created by `ColumnarWriterBuilder`.
190+
## 🛠 Troubleshooting
328191

329-
Check that:
192+
**`react-native-columnar.h` not found on Android** — check that `prefab true` is enabled and CMake links the package correctly.
330193

331-
- The JS `ColumnType` list has the same order as the C++ schema.
332-
- Each JS type matches the exact C++ type size (`int32_t``ColumnType.Int32`,
333-
`double``ColumnType.Float64`, etc.).
334-
- The buffer is the `ArrayBuffer` returned by `writer.toArrayBuffer(rt)`.
194+
**`std::span` errors** — set C++20 on the target that includes the header.
335195

336-
### Values look shifted or incorrect
196+
**`RangeError` in JS** — JS and C++ schemas are out of sync. Check column order and types match exactly (`int32_t``Int32`, `double``Float64`).
337197

338-
The reader and writer must agree on both column order and type width. A single
339-
wrong type can shift all following columns. Start by comparing the C++ schema with
340-
the JS schema line by line.
198+
**Values look shifted** — one wrong type shifts all following columns. Compare schemas line by line.
341199

342200
---
343201

0 commit comments

Comments
 (0)