You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
1.**Clean API surface** — usage for common cases should be straightforward and well-documented. Avoid breaking changes unless they significantly improve usability.
6
+
2.**Understandable code** — cyclomatic complexity per method should stay around or below current levels. Higher-level methods should delegate to clearly-named helpers so the logic reads like pseudo-code. Use utility functions (e.g. `kWayMerge`, binary search) for generic algorithms.
7
+
3.**Performance** — maintain good performance across all code paths. The Index and Partition layers are most performance-sensitive and may prefer performance over readability at the margin. Elsewhere, prefer simplicity; simpler code is often faster.
8
+
9
+
**Keep this file up-to-date**: after any code review that surfaces new architectural insights or a new principle, compact this file and integrate what was learned.
10
+
11
+
## Architecture
12
+
13
+
```
14
+
EventStore → Storage → Partition (append-only data files)
15
+
→ Index → per-stream index files
16
+
→ EventStream / JoinEventStream (iterators over indexes)
17
+
→ Consumer (durable position tracking)
18
+
```
19
+
20
+
-**Storage/Index/Partition** each follow a 3-tier class hierarchy under `src/<Component>/`: `Readable*` → `ReadOnly*` → `Writable*`. The facade file `src/<Component>.js` re-exports the Writable + ReadOnly variants.
21
+
-**EventStore** (`src/EventStore.js`) is the main entry point — owns a `Storage` instance, manages stream indexes in a `streams/` subdirectory, and provides `commit()` / `getEventStream()` / `query()` / `createConsumer()`.
22
+
-**DCB concurrency**: `query()` returns a `CommitCondition` capturing the global log position + type/matcher filter. Passing it as `expectedVersion` to `commit()` rejects only when matching events were appended since the query.
23
+
- Streams are named indexes over a shared storage file; events are partitioned by `event.stream`. Category queries use `<category>-<id>` naming convention.
24
+
25
+
## Lifecycle & I/O Patterns
26
+
27
+
-**Constructor stays sync** — all file I/O is deferred to `open()`.
28
+
-**`open()` is the entry point for I/O**: scanning partitions, scanning index files, acquiring locks, and opening file descriptors.
29
+
-**`'opened'` event** — emitted by Storage once the first async scan + primary index open completes. EventStore listens to `'opened'` to emit its own `'ready'`.
30
+
-**`'index-created'` event** — emitted by Storage during `scanFiles()` for each existing secondary index file found. EventStore uses this to register streams without its own directory scan.
31
+
-**Secondary indexes open on demand** — only the primary index is opened eagerly; secondary indexes are opened lazily on first access.
32
+
-**`initialized` three-state**: `null` = not started, `false` = scan in progress, `true` = scan done. Re-opens after `close()` are synchronous.
33
+
-**`open(callback)` hook** — fires after `openIndexes()` and before `'opened'`. Used by `WritableStorage` for torn-write repair.
34
+
-**LOCK_RECLAIM in `open()`** — orphaned lock removal lives in `WritableStorage.open()`, directly before `lock()`; torn-write repair runs via the `open(callback)` hook.
The package exposes NDJSON stream endpoints, durable consumer management, and an `HttpEventStream` client helper for consuming event streams over fetch.
96
+
97
+
---
98
+
78
99
## Documentation
79
100
80
101
The full documentation is hosted at **<https://node-event-storage.readthedocs.io/en/latest/>** and covers:
Return an `EventStream` for the named stream, or `false` if no such stream exists. `minRevision` and `maxRevision` are 1-based and inclusive; negative values count from the end.
86
+
Return an `EventStream` for the named stream, or `false` if no such stream exists. `minRevision` and `maxRevision` are 1-based and inclusive; negative values count from the end.
87
+
88
+
-`predicate` supports function and object matchers.
89
+
-`raw=true` returns newline-delimited JSON `Buffer` chunks (NDJSON) for direct network streaming.
- Function matcher: `(payload, metadata) => boolean`
513
+
- Object matcher: evaluated against `{ stream, payload, metadata }`
514
+
-**Raw mode (`raw=true`)**
515
+
- Function matcher: `(buffer) => boolean` where `buffer` is one compact JSON document
516
+
- Object matcher: byte-level raw matcher over compact JSON bytes (lazy-built at first stream consumption)
517
+
518
+
The raw object matcher requires the default compact JSON serializer format. If you use a custom serializer (including pretty-printed or transformed JSON), object matchers in raw mode are not guaranteed to work; use a function matcher in that case.
519
+
520
+
---
521
+
495
522
## Storage
496
523
497
524
`Storage` is the low-level append-only document store used internally by `EventStore`. It can be used directly for advanced use cases.
The optional `matcher` narrows the boundary to exactly the events that would affect the decision — unrelated events of the same type (e.g. a different order) never cause spurious conflicts.
38
38
39
+
`query()` also supports raw mode (`query(types, matcher, minRevision, true)`), but raw streaming itself is a general stream-reading feature, not DCB-specific. See [Event Streams -> Reading Streams](streams.md#reading-streams) for the full raw-mode semantics and matcher behavior.
0 commit comments