You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
**Pebblify** is a high-performance migration tool that converts LevelDB databases to PebbleDB format, specifically designed for Cosmos SDK and CometBFT (formerly Tendermint) blockchain nodes.
9
-
10
-
PebbleDB offers significant performance improvements over LevelDB, including better write throughput, more efficient compaction, and reduced storage overhead. Pebblify makes it easy to migrate your existing node data without manual intervention.
**Pebblify** is a high-performance migration tool that converts LevelDB databases to PebbleDB format, specifically designed for Cosmos SDK and CometBFT blockchain nodes.
13
17
14
-
> [!WARNING]
15
-
> This tool is still in the early stages of development and may contain bugs or be unstable. If you notice any unusual behavior, please open an issue.
18
+
PebbleDB offers significant performance improvements over LevelDB, including better write throughput, more efficient compaction, and reduced storage overhead. Pebblify outperforms existing tools: in a real-world production benchmark it completed a full snapshot conversion in **6m48** versus 7m13 for Level2Pebble, while producing a **20.34 GiB** output compared to 26 GiB, with no data loss. It adds crash recovery, disk space checks, verification, Prometheus metrics, health probes, notifications, and daemon mode that Level2Pebble does not provide.
16
19
17
-
## Features
20
+
**[Install now](#installation)** or read the [full documentation](https://docs.dockermint.io/pebblify/).
18
21
19
-
-**Fast parallel conversion** — Process multiple databases concurrently with configurable worker count
20
-
-**Crash recovery** — Resume interrupted migrations from the last checkpoint
21
-
-**Adaptive batching** — Automatically adjusts batch sizes based on memory constraints
22
-
-**Real-time progress** — Live progress bar with throughput metrics and ETA
23
-
-**Data verification** — Verify converted data integrity with configurable sampling
24
-
-**Disk space checks** — Pre-flight validation to ensure sufficient storage
For daemon and container installation options, see the [installation guide](https://docs.dockermint.io/pebblify/).
134
+
88
135
## Daemon mode
89
136
137
+
> [!WARNING]
138
+
> The daemon mode is currently in **beta**. If you encounter a bug, please [open an issue](https://github.com/Dockermint/pebblify/issues/new/choose).
139
+
90
140
`pebblify daemon` is a long-running HTTP service that accepts snapshot archive URLs, converts them from LevelDB to PebbleDB format, repacks the output, and saves it to one or more storage targets (local directory, SCP, S3). Jobs are submitted via a REST API and processed serially.
91
141
92
142
**Platform:** The `daemon` subcommand is Linux-only at runtime. On macOS, use Docker Compose or Podman.
@@ -109,92 +159,60 @@ curl -s -X POST http://127.0.0.1:2324/v1/jobs \
109
159
110
160
Full guide: [docs/markdown/daemon-quickstart.md](docs/markdown/daemon-quickstart.md)
111
161
112
-
## Quick Start
162
+
## Systemd daemon (Linux only)
113
163
114
-
### Convert
164
+
Installs the daemon as a system service. Requires root. Not supported on macOS.
> For full command reference and all available flags, see the [documentation](https://docs.dockermint.io/pebblify/).
146
-
147
-
## Artifact attestation
148
-
149
-
Starting with v0.4.0, every release binary and Docker image is published with SLSA provenance and SBOM attestations. Verify before running:
187
+
After installation:
150
188
151
189
```bash
152
-
# Verify a release binary
153
-
gh attestation verify pebblify-linux-amd64 \
154
-
--repo Dockermint/Pebblify
155
-
156
-
# Verify the Docker image
157
-
gh attestation verify \
158
-
oci://ghcr.io/dockermint/pebblify:v0.4.0 \
159
-
--repo Dockermint/Pebblify
190
+
systemctl --user daemon-reload
191
+
systemctl --user start pebblify
160
192
```
161
193
162
-
Available attestation targets: `pebblify-linux-amd64`, `pebblify-linux-arm64`, `pebblify-darwin-amd64`, `pebblify-darwin-arm64`, and the multi-arch Docker image.
163
-
164
-
Full guide: [docs/markdown/release-automation.md](docs/markdown/release-automation.md)
165
-
166
-
## Benchmark
167
-
168
-
Real-world conversion on a production Cosmos node dataset:
169
-
170
-
| Metric | Value |
171
-
|---|---|
172
-
| Total keys | 217,895,735 |
173
-
| Duration | 4m 11s |
174
-
| Throughput |~865k keys/s · ~160 MB/s |
175
-
| Data processed | 39.54 GiB read / 39.54 GiB written |
0 commit comments