Skip to content

Commit 96e8658

Browse files
Merge pull request #20 from elliotchenzichang/update-read-me
update the archtecture flow of this project
2 parents ffa9c16 + ff066e3 commit 96e8658

1 file changed

Lines changed: 61 additions & 10 deletions

File tree

README.md

Lines changed: 61 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,67 @@ The same source lives at [`cmd/demo/main.go`](cmd/demo/main.go). Could clone thi
7272

7373
---
7474

75+
## Architecture
76+
77+
High-level layout: the **`DB`** type ties together **options**, an in-memory **keydir**, **segment files** (active + sealed), optional **hint** sidecars for recovery, and an **advisory lock**. **`entity`** defines the on-disk record layout shared by writes, reads, hints, and merge.
78+
79+
```mermaid
80+
flowchart TB
81+
subgraph Client["Client"]
82+
App["Application / cmd/demo"]
83+
end
84+
85+
subgraph API["Package root"]
86+
DB["DB · Get / Set / Delete / Merge / recovery"]
87+
Opt["Options"]
88+
Lk["Advisory lock · lock_unix / lock_other"]
89+
end
90+
91+
subgraph Idx["index/"]
92+
KD["KeyDir · key → DataPosition"]
93+
end
94+
95+
subgraph St["storage/"]
96+
DF["DataFiles"]
97+
Act["ActiveFile · append-only segment"]
98+
Old["OldFile · sealed segments"]
99+
Hnt["Hint files · read/write"]
100+
end
101+
102+
subgraph Ent["entity/"]
103+
Rec["Entry · Meta · CRC · tombstone"]
104+
end
105+
106+
subgraph Disk["Data directory on disk"]
107+
Dat["*.dat"]
108+
Hin["*.hint"]
109+
Lf[".tiny-bitcask.lock"]
110+
end
111+
112+
App --> DB
113+
DB --> Opt
114+
DB --> KD
115+
DB --> DF
116+
DB --> Lk
117+
Lk --> Lf
118+
119+
DF --> Act
120+
DF --> Old
121+
DF --> Hnt
122+
123+
Act --> Dat
124+
Old --> Dat
125+
Hnt --> Hin
126+
127+
Rec -.->|encode / decode / verify| Act
128+
Rec -.->|encode / decode / verify| Old
129+
Rec -.->|compact index rows| Hnt
130+
```
131+
132+
**Typical paths (mental model).** **Write:** append an `Entry` to the active `.dat`, then update `KeyDir`. **Read:** `KeyDir` lookup → `ReadAt` on the segment identified by `fid`. **Open:** scan segments (or load hints) to rebuild `KeyDir`. **Merge:** scan sealed segments for live rows only, rewrite into the active file, drop old `.dat` / `.hint` pairs.
133+
134+
---
135+
75136
## What is implemented
76137

77138
- **Open / create**: `NewDB` — empty directory creates a new store; existing directory **recovers** the keydir by scanning `*.dat` files in order (or hints for sealed segments). **`Options`**: `VerifyCRC` (default on), `ReadOnly` (open existing store read-only), `ExclusiveLock` (Unix advisory `flock` on `.tiny-bitcask.lock`; shared lock when `ReadOnly`).
@@ -112,13 +173,3 @@ Some items from [bitcask-intro.pdf](https://riak.com/assets/bitcask-intro.pdf) a
112173
| `options.go` | `Dir`, `SegmentSize`, `VerifyCRC`, `ReadOnly`, `ExclusiveLock` |
113174

114175
---
115-
116-
## Todo (paper-aligned and robustness)
117-
118-
- [x] **Hint files**: format, write on rotation, load hint instead of full scan on open when present.
119-
- [x] **Recovery**: populate old-file ID list from on-disk segments so `Merge` works after restart; timestamps from full scan recovery.
120-
- [x] **Merge**: compare keydir `(fid, offset)` to the **start** offset of each scanned entry; test `TestDB_Merge_LiveKeyOnlyInOldSegment`.
121-
- [x] **Delete / tombstones**: valid meta + key for tombstone records; recovery applies tombstones; merge skips tombstone bodies.
122-
- [x] **CRC verification** on read (`Options.VerifyCRC`, default `true`).
123-
- [x] **`Sync` / `Close`**: flush and fsync active file; close FDs and lock.
124-
- [x] **API**: `ListKeys`, `Fold`, read-only open (`Options.ReadOnly`), Unix advisory lock (`Options.ExclusiveLock`).

0 commit comments

Comments
 (0)