You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: src/content/docs/reference/building.md
+50-6Lines changed: 50 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -79,15 +79,21 @@ TidesDB is tested and supported on the following platforms:
79
79
- Clang 6.0+ (macOS/Linux)
80
80
- MSVC 2019 16.8+ (Windows with `/experimental:c11atomics`)
81
81
82
-
### Required Dependencies
83
-
TidesDB requires the following libraries for compression and threading:
82
+
### Dependencies
83
+
84
+
The only hard requirements are a C11 toolchain, CMake, and threading support:
84
85
85
-
-**[Snappy](https://github.com/google/snappy)** · Fast compression/decompression (not available on SunOS/Illumos)
86
-
-**[LZ4](https://github.com/lz4/lz4)** · Extremely fast compression
87
-
-**[Zstandard](https://github.com/facebook/zstd)** · High compression ratio
88
86
-**pthreads** · POSIX threads (Linux/macOS/BSD: built-in, Windows: PThreads4W via vcpkg)
89
87
-**libatomic** · Atomic operations library (required for 32-bit architectures like PowerPC)
90
-
-**C++ standard library** · Required by Snappy (automatically linked on Linux)
88
+
89
+
#### Compression backends (optional, enabled by default)
90
+
91
+
Each of the three compression libraries is optional. All three are built in by default; drop any with `-DTIDESDB_WITH_<NAME>=OFF`. A build with all three disabled has no compression dependencies at all and supports only `TDB_COMPRESS_NONE`, so TidesDB can be installed with nothing beyond libc and a thread library. See [Compression Backends](#compression-backends) under Build Options.
92
+
93
+
-**[Snappy](https://github.com/google/snappy)** · Fast compression/decompression (`TIDESDB_WITH_SNAPPY`; not packaged on SunOS/Illumos, so disabled there by default)
94
+
-**[LZ4](https://github.com/lz4/lz4)** · Extremely fast compression (`TIDESDB_WITH_LZ4`)
95
+
-**[Zstandard](https://github.com/facebook/zstd)** · High compression ratio (`TIDESDB_WITH_ZSTD`)
96
+
-**C++ standard library** · Pulled in only when Snappy is enabled (Snappy's implementation is C++); a build without Snappy carries no libstdc++ dependency
91
97
92
98
## Installing Dependencies
93
99
@@ -463,12 +469,50 @@ TidesDB provides several CMake options to customize the build:
The on-disk compression algorithm IDs are stable regardless of which backends are compiled in, so an SSTable written with an algorithm a given build lacks is reported with a clear error rather than misread. The default column family compression resolves to the best available backend at runtime (LZ4, then Zstandard, then Snappy, then none), so a default configuration always works against whatever was built.
497
+
498
+
:::note[Consumers need no compression headers]
499
+
The third-party compression headers are included only inside the library, so an application that links `libtidesdb` and includes `<tidesdb/tidesdb.h>` does not need the Snappy/LZ4/Zstandard development headers itself.
500
+
:::
501
+
502
+
#### Linking against a vendored compression library
503
+
504
+
By default each backend links a known CMake target if one exists in the build (for example one provided by a parent project via `FetchContent` or `add_subdirectory`), and otherwise falls back to a system `-l<name>`. If you vendor a compression library under a target name the autodetect cannot guess, point TidesDB at it explicitly:
This avoids the forced system `-l` link that otherwise fails when the library is only available as an in-tree target rather than installed system-wide. The same applies to `TIDESDB_SNAPPY_TARGET` and `TIDESDB_LZ4_TARGET`.
515
+
472
516
### Mimalloc Memory Allocator
473
517
474
518
TidesDB optionally supports [mimalloc](https://github.com/microsoft/mimalloc), Microsoft's high-performance memory allocator. When enabled, mimalloc replaces the standard malloc/free at link time, providing improved allocation performance for memory-intensive workloads.
The output includes the total number of keys, data size, memtable size, number of LSM levels, read amplification factor, cache hit rate, average key and value sizes, and per-level SSTable counts and sizes. For tables with secondary indexes, each index CF's statistics are reported separately.
1186
1186
1187
+
When the column family has logged any user bytes, a per-CF write-amplification note is also emitted:
`user` is the logical bytes the plugin wrote through tidesdb's API, `wal` is the per-CF WAL bytes for that family (zero under unified memtable mode, which uses a single shared WAL accounted for separately in `SHOW ENGINE TIDESDB STATUS`), `flush` and `compact_write` are bytes emitted to SSTables by flush jobs and compaction jobs respectively, along with how many SSTables each produced, `compact_read` is bytes pulled in as compaction input, and `ratio` is `(wal + flush + compact_write) / user` rounded to two decimals. A high ratio with a low SSTable count tends to point at oversized flush memtables, a high ratio with many compactions points at unnecessary L0 churn or an under-sized `L1_FILE_COUNT_TRIGGER`.
1194
+
1187
1195
This information is useful for understanding the physical layout of your data and diagnosing performance characteristics.
1188
1196
1189
1197
@@ -1197,6 +1205,12 @@ SHOW ENGINE TIDESDB STATUS\G
1197
1205
1198
1206
The output includes the data directory path, number of column families, global sequence number, memory usage (total system memory, resolved memory limit, memory pressure level, memtable bytes, transaction memory bytes), storage metrics (total SSTables, open SSTable handles, total data size, immutable memtable count), background queue sizes (flush pending, flush queue, compaction queue), and block cache statistics (enabled, entries, size, hits, misses, hit rate, partitions).
1199
1207
1208
+
When `tidesdb_unified_memtable=ON` an additional **Unified Memtable** block reports the active skip-list bytes, the number of immutables queued, whether a flush is currently in flight, the current WAL generation, and the round-robin next-CF index that the library uses when it scans for CF state to fold into the shared memtable.
1209
+
1210
+
A **Write Amplification** block reports cumulative user bytes written through the plugin alongside unified-WAL bytes, per-CF WAL bytes, flush bytes written, compaction bytes written and read, and a derived total WA ratio computed as `(unified_wal + per_cf_wal + flush + compaction_write) / user_bytes_written`. The flush and compaction rows also include their SSTable counts, so the ratio between bytes and SSTable count gives a coarse view of average output-SSTable size at each stage.
1211
+
1212
+
When an object-store backend is configured an **Object Store** block is included, and the last successfully uploaded unified-WAL generation is reported alongside the local cache footprint, upload queue depth, and lifetime upload counters.
1213
+
1200
1214
When `tidesdb_print_all_conflicts` is enabled, the last conflict event is also displayed under a Conflicts section.
1201
1215
1202
1216
### Status Variables
@@ -1209,8 +1223,8 @@ SHOW GLOBAL STATUS LIKE 'tidesdb%';
1209
1223
1210
1224
| Variable | Description |
1211
1225
|----------|-------------|
1212
-
|`Tidesdb_version`| TideSQL plugin version string (e.g. `4.5.4`) |
1213
-
|`Tidesdb_version_hex`| TideSQL plugin version as integer (e.g. `263940` = `0x40504`) |
1226
+
|`Tidesdb_version`| TideSQL plugin version string (e.g. `4.5.5`) |
1227
+
|`Tidesdb_version_hex`| TideSQL plugin version as integer (e.g. `263941` = `0x40505`) |
1214
1228
|`Tidesdb_column_families`| Number of active column families |
1215
1229
|`Tidesdb_global_sequence`| Global MVCC sequence number |
1216
1230
|`Tidesdb_memtable_bytes`| Total memtable memory usage in bytes |
@@ -1236,6 +1250,27 @@ SHOW GLOBAL STATUS LIKE 'tidesdb%';
1236
1250
|`Tidesdb_max_sst_tombstone_density_level`| LSM level of the SSTable with the maximum density |
1237
1251
|`Tidesdb_backpressure_waits`| Number of times a writer blocked on TidesDB back-pressure inside the plugin |
1238
1252
|`Tidesdb_backpressure_wait_us`| Total microseconds spent blocked on back-pressure |
1253
+
|`Tidesdb_unified_memtable_enabled`| 1 when the unified memtable is active, 0 when each column family carries its own memtable and WAL |
1254
+
|`Tidesdb_unified_memtable_bytes`| Live byte usage of the unified memtable's active skip list |
1255
+
|`Tidesdb_unified_immutable_count`| Number of unified-memtable immutables queued for flush |
1256
+
|`Tidesdb_unified_is_flushing`| 1 while a unified-memtable flush is in flight, 0 otherwise |
1257
+
|`Tidesdb_unified_wal_generation`| Current unified-WAL generation number; advances each time the active WAL segment rolls over |
1258
+
|`Tidesdb_object_store_enabled`| 1 when an object-store backend is configured (S3 or remote), 0 for purely local storage |
1259
+
|`Tidesdb_replica_mode_active`| 1 when this instance is running as a read-only object-store replica |
1260
+
|`Tidesdb_local_cache_bytes`| Bytes currently held in the local object-store cache directory |
1261
+
|`Tidesdb_local_cache_files`| Number of files in the local object-store cache |
1262
+
|`Tidesdb_upload_queue_depth`| Pending uploads waiting on the object-store uploader threads |
1263
+
|`Tidesdb_total_uploads`| Lifetime count of successful uploads to the object store |
1264
+
|`Tidesdb_upload_failures`| Lifetime count of upload attempts that failed and were retried |
1265
+
|`Tidesdb_last_uploaded_generation`| Highest unified-WAL generation acknowledged by the object store (lags `Tidesdb_unified_wal_generation` by the upload queue) |
1266
+
|`Tidesdb_uwal_bytes_written`| Bytes written by the plugin to the unified WAL over the process lifetime |
1267
+
|`Tidesdb_wal_bytes_written`| Bytes written to per-CF WALs (sum across column families when unified memtable is OFF) |
1268
+
|`Tidesdb_flush_bytes_written`| Bytes emitted to L0 SSTables by flush jobs across all column families |
1269
+
|`Tidesdb_compaction_bytes_written`| Bytes written by compaction jobs across all column families |
1270
+
|`Tidesdb_compaction_bytes_read`| Bytes read by compaction jobs as input across all column families |
1271
+
|`Tidesdb_user_bytes_written`| Logical user bytes ingested via the plugin's write path (key + value, before any WAL/SSTable framing) |
1272
+
|`Tidesdb_flush_count`| Number of flush operations completed across all column families |
1273
+
|`Tidesdb_compaction_count`| Number of compaction operations completed across all column families |
1239
1274
|`Tidesdb_lock_waits`| Number of times any thread blocked in the pessimistic lock-wait loop |
1240
1275
|`Tidesdb_lock_wait_us`| Total microseconds spent in the pessimistic lock-wait loop |
1241
1276
|`Tidesdb_lock_deadlocks`| Number of wait-for-graph cycle detections that returned `ER_LOCK_DEADLOCK`|
0 commit comments