Skip to content

Commit 8cc3cb5

Browse files
nayrosknayrosk
authored
Merge pull request #45 from Dockermint/docs/release-v0.4.0
docs: v0.4.0 release documentation refresh
2 parents d87e8ac + 80cbb4a commit 8cc3cb5

14 files changed

Lines changed: 2279 additions & 3 deletions

‎README.md‎

Lines changed: 81 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -35,13 +35,48 @@ PebbleDB offers significant performance improvements over LevelDB, including bet
3535

3636
## Installation
3737

38-
### From Source
38+
### CLI (all platforms)
39+
40+
Cross-platform. Works on `linux/amd64`, `linux/arm64`, `darwin/amd64`, and `darwin/arm64`.
3941

4042
```bash
4143
git clone https://github.com/Dockermint/pebblify.git
4244
cd pebblify
43-
make build # build for current platform
44-
make install # build and install to PATH
45+
make install-cli # build and install pebblify binary to PATH
46+
```
47+
48+
Or download a pre-built binary from [releases](https://github.com/Dockermint/pebblify/releases).
49+
50+
### Systemd daemon — **Linux only**
51+
52+
Installs the daemon as a system service. Requires root. Not supported on macOS.
53+
54+
```bash
55+
sudo make install-systemd-daemon
56+
```
57+
58+
After installation, fill in `/etc/pebblify/.env`, then:
59+
60+
```bash
61+
sudo systemctl daemon-reload
62+
sudo systemctl enable --now pebblify
63+
```
64+
65+
See [daemon quickstart](docs/markdown/daemon-quickstart.md) for the full setup guide.
66+
67+
### Podman Quadlet (rootless)
68+
69+
Deploys the daemon as a rootless Podman container managed by the systemd user session. Linux-native; macOS users need Podman Desktop.
70+
71+
```bash
72+
make install-podman
73+
```
74+
75+
After installation:
76+
77+
```bash
78+
systemctl --user daemon-reload
79+
systemctl --user start pebblify
4580
```
4681

4782
### Docker
@@ -50,6 +85,30 @@ make install # build and install to PATH
5085
make build-docker
5186
```
5287

88+
## Daemon mode
89+
90+
`pebblify daemon` is a long-running HTTP service that accepts snapshot archive URLs, converts them from LevelDB to PebbleDB format, repacks the output, and saves it to one or more storage targets (local directory, SCP, S3). Jobs are submitted via a REST API and processed serially.
91+
92+
**Platform:** The `daemon` subcommand is Linux-only at runtime. On macOS, use Docker Compose or Podman.
93+
94+
```bash
95+
# Set the API token
96+
export PEBBLIFY_BASIC_AUTH_TOKEN="your-secret-token-here"
97+
98+
# Start the daemon (Linux only)
99+
pebblify daemon
100+
101+
# Submit a job
102+
curl -s -X POST http://127.0.0.1:2324/v1/jobs \
103+
-u "ignored:${PEBBLIFY_BASIC_AUTH_TOKEN}" \
104+
-H "Content-Type: application/json" \
105+
-d '{"url": "https://snapshots.example.com/gaia-snapshot.tar.lz4"}'
106+
```
107+
108+
> **macOS users:** Run the daemon via Docker Compose (`docker compose -f docker-compose.daemon.yml up -d`) or Podman Desktop. See [daemon quickstart](docs/markdown/daemon-quickstart.md).
109+
110+
Full guide: [docs/markdown/daemon-quickstart.md](docs/markdown/daemon-quickstart.md)
111+
53112
## Quick Start
54113

55114
### Convert
@@ -85,6 +144,25 @@ docker run --rm \
85144

86145
> For full command reference and all available flags, see the [documentation](https://docs.dockermint.io/pebblify/).
87146
147+
## Artifact attestation
148+
149+
Starting with v0.4.0, every release binary and Docker image is published with SLSA provenance and SBOM attestations. Verify before running:
150+
151+
```bash
152+
# Verify a release binary
153+
gh attestation verify pebblify-linux-amd64 \
154+
--repo Dockermint/Pebblify
155+
156+
# Verify the Docker image
157+
gh attestation verify \
158+
oci://ghcr.io/dockermint/pebblify:v0.4.0 \
159+
--repo Dockermint/Pebblify
160+
```
161+
162+
Available attestation targets: `pebblify-linux-amd64`, `pebblify-linux-arm64`, `pebblify-darwin-amd64`, `pebblify-darwin-arm64`, and the multi-arch Docker image.
163+
164+
Full guide: [docs/markdown/release-automation.md](docs/markdown/release-automation.md)
165+
88166
## Benchmark
89167

90168
Real-world conversion on a production Cosmos node dataset:

‎docs/docusaurus/configuration-reference.mdx‎

Lines changed: 265 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,265 @@
1+
---
2+
id: configuration-reference
3+
title: Configuration Reference
4+
sidebar_label: Configuration Reference
5+
sidebar_position: 5
6+
description: Full reference for the Pebblify daemon config.toml schema and environment variables.
7+
---
8+
9+
Full reference for all `config.toml` keys and environment variables accepted by the Pebblify daemon.
10+
11+
The config file path resolves in this order:
12+
1. `PEBBLIFY_CONFIG_PATH` environment variable
13+
2. `./config.toml` (default)
14+
15+
Source: `internal/daemon/config/config.go`
16+
17+
## [general]
18+
19+
<details>
20+
<summary>Required. Schema version gate.</summary>
21+
22+
| Key | Type | Default | Description |
23+
| :--------------- | :--- | :------ | :------------------------------------------------ |
24+
| `config_version` | int | — | Must be `0` in v0.4.0. Absent or higher value = fatal startup error. |
25+
26+
```toml
27+
[general]
28+
config_version = 0
29+
```
30+
31+
</details>
32+
33+
## [api]
34+
35+
<details open>
36+
<summary>HTTP control plane. Always active — no enable flag.</summary>
37+
38+
| Key | Type | Default | Valid values | Notes |
39+
| :---------------------- | :----- | :------------- | :------------------------ | :------------------------------------------------------------ |
40+
| `host` | string | `"127.0.0.1"` | Any bind address | Bind address for the API listener. |
41+
| `port` | int | `2324` | `1–65535` | TCP port. |
42+
| `authentification_mode` | string | `"basic_auth"` | `basic_auth`, `unsecure` | `basic_auth` requires `PEBBLIFY_BASIC_AUTH_TOKEN`. `unsecure` emits a WARN log. |
43+
44+
:::warning
45+
`authentification_mode = "unsecure"` is not recommended for production. The daemon logs a warning on every startup.
46+
:::
47+
48+
```toml
49+
[api]
50+
host = "127.0.0.1"
51+
port = 2324
52+
authentification_mode = "basic_auth"
53+
```
54+
55+
</details>
56+
57+
## [notify]
58+
59+
<details>
60+
<summary>Job lifecycle notifications. Disabled by default.</summary>
61+
62+
| Key | Type | Default | Valid values | Notes |
63+
| :----------- | :----- | :----------- | :-------------- | :------------------------------------------------------------------------- |
64+
| `enable` | bool | `false` | `true`, `false` | Toggles notifications. |
65+
| `mode` | string | `"telegram"` | `telegram` | Only `telegram` is valid in v0.4.0. |
66+
| `channel_id` | string | `""` | Chat ID string | Required when `enable = true`. Telegram chat or channel ID (may be negative). |
67+
68+
When `notify.enable = true`:
69+
- `PEBBLIFY_TELEGRAM_BOT_TOKEN` environment variable is required.
70+
- `channel_id` must be non-empty.
71+
- Missing either is a fatal startup error.
72+
73+
```toml
74+
[notify]
75+
enable = true
76+
mode = "telegram"
77+
channel_id = "-1001234567890"
78+
```
79+
80+
</details>
81+
82+
## [telemetry]
83+
84+
<details>
85+
<summary>Prometheus metrics endpoint.</summary>
86+
87+
| Key | Type | Default | Notes |
88+
| :------- | :----- | :------------- | :------------------------------------------------ |
89+
| `enable` | bool | `true` | Toggles the `/metrics` listener. |
90+
| `mode` | string | `"prometheus"` | Only `prometheus` is valid. |
91+
| `host` | string | `"127.0.0.1"` | Bind address. |
92+
| `port` | int | `2323` | TCP port. |
93+
94+
```toml
95+
[telemetry]
96+
enable = true
97+
mode = "prometheus"
98+
host = "127.0.0.1"
99+
port = 2323
100+
```
101+
102+
</details>
103+
104+
## [health]
105+
106+
<details>
107+
<summary>Liveness and readiness probes.</summary>
108+
109+
| Key | Type | Default | Notes |
110+
| :------- | :----- | :------------ | :------------------------------------------------ |
111+
| `enable` | bool | `true` | Toggles the `/healthz` and `/readyz` listener. |
112+
| `host` | string | `"127.0.0.1"` | Bind address. |
113+
| `port` | int | `2325` | TCP port. |
114+
115+
- `GET /healthz` — liveness, always `200 OK` while running.
116+
- `GET /readyz` — readiness, `200 OK` only when queue is empty and no job is running.
117+
118+
```toml
119+
[health]
120+
enable = true
121+
host = "127.0.0.1"
122+
port = 2325
123+
```
124+
125+
</details>
126+
127+
## [convertion]
128+
129+
<details>
130+
<summary>Temporary directory and source cleanup settings.</summary>
131+
132+
:::note Spelling
133+
The TOML key is `convertion` (matching the Go struct tag). This spelling is intentionally preserved to avoid a breaking schema change.
134+
:::
135+
136+
| Key | Type | Default | Notes |
137+
| :----------------------- | :----- | :------- | :----------------------------------------------------------------- |
138+
| `temporary_directory` | string | `"/tmp"` | Scratch directory for download, extraction, conversion, repacking. Supports `~` prefix. |
139+
| `delete_source_snapshot` | bool | `true` | Remove the source LevelDB directory after conversion. |
140+
141+
```toml
142+
[convertion]
143+
temporary_directory = "/tmp"
144+
delete_source_snapshot = true
145+
```
146+
147+
</details>
148+
149+
## [save]
150+
151+
<details open>
152+
<summary>Output archive compression and storage targets.</summary>
153+
154+
| Key | Type | Default | Valid values | Notes |
155+
| :------------ | :----- | :------ | :-------------------------------- | :--------------------------------- |
156+
| `compression` | string | `"lz4"` | `none`, `lz4`, `zstd`, `gzip`, `zip` | Codec for the output archive. |
157+
158+
```toml
159+
[save]
160+
compression = "lz4"
161+
```
162+
163+
</details>
164+
165+
### [save.local]
166+
167+
<details>
168+
<summary>Local filesystem storage target.</summary>
169+
170+
| Key | Type | Default | Notes |
171+
| :--------------------- | :----- | :--------------- | :----------------------------------------------------- |
172+
| `enable` | bool | `true` | Toggles the local storer. |
173+
| `local_save_directory` | string | `"~/.snapshots"` | Output directory. Required when `enable = true`. Supports `~` prefix. |
174+
175+
```toml
176+
[save.local]
177+
enable = true
178+
local_save_directory = "~/.snapshots"
179+
```
180+
181+
</details>
182+
183+
### [save.scp]
184+
185+
<details>
186+
<summary>SCP/SSH storage target. Disabled by default.</summary>
187+
188+
| Key | Type | Default | Valid values | Env override | Notes |
189+
| :---------------------- | :----- | :------ | :------------------------ | :-------------------- | :--------------------------------- |
190+
| `enable` | bool | `false` | `true`, `false` | — | Toggles the SCP storer. |
191+
| `authentification_mode` | string | `"key"` | `key`, `password`, `none` | — | SSH authentication method. |
192+
| `host` | string | `""` | Hostname or IP | — | Required when `enable = true`. |
193+
| `port` | int | `0` | `1–65535` | — | Required when `enable = true`. |
194+
| `username` | string | `""` | SSH login name | — | Required when `enable = true`. |
195+
196+
Secret env vars:
197+
- `PEBBLIFY_SCP_KEY_PATH` — required when `authentification_mode = key`
198+
- `PEBBLIFY_SCP_PASSWORD` — required when `authentification_mode = password`
199+
200+
```toml
201+
[save.scp]
202+
enable = true
203+
authentification_mode = "key"
204+
host = "backup.example.com"
205+
port = 22
206+
username = "pebblify"
207+
```
208+
209+
</details>
210+
211+
### [save.s3]
212+
213+
<details>
214+
<summary>AWS S3 storage target. Disabled by default.</summary>
215+
216+
| Key | Type | Default | Notes |
217+
| :------------- | :----- | :------ | :--------------------------------------------------------------------------------- |
218+
| `enable` | bool | `false` | Toggles the S3 storer. |
219+
| `bucket_name` | string | `""` | Destination bucket. Required when `enable = true`. |
220+
| `s3_access_key`| string | `""` | AWS access key ID. Required when `enable = true`. Safe to store in config.toml. |
221+
| `save_path` | string | `""` | Key prefix. Final S3 key: `<save_path>/<filename>`. May be empty. |
222+
223+
Secret env var: `PEBBLIFY_S3_SECRET_KEY` — required when `enable = true`.
224+
225+
Region resolution order: `AWS_REGION` → `AWS_DEFAULT_REGION` → SDK default config → `us-east-1` (with WARN log).
226+
227+
```toml
228+
[save.s3]
229+
enable = true
230+
bucket_name = "pebblify-snapshots"
231+
s3_access_key = "AKIAIOSFODNN7EXAMPLE"
232+
save_path = "converted"
233+
```
234+
235+
</details>
236+
237+
## [queue]
238+
239+
<details>
240+
<summary>Job queue capacity. Optional section.</summary>
241+
242+
| Key | Type | Default | Valid values | Notes |
243+
| :------------ | :--- | :------ | :----------- | :------------------------------------------------------------------ |
244+
| `buffer_size` | int | `64` | `>= 1` | Maximum pending jobs in FIFO. Full buffer = new submissions get `503`. |
245+
246+
```toml
247+
[queue]
248+
buffer_size = 64
249+
```
250+
251+
</details>
252+
253+
## Environment variables
254+
255+
| Variable | Required when | Description |
256+
| :---------------------------- | :------------------------------------------------------ | :----------------------------------------- |
257+
| `PEBBLIFY_CONFIG_PATH` | Never | Config file path override. |
258+
| `PEBBLIFY_LOG_LEVEL` | Never | `trace`, `debug`, `info`, `warn`, `error`. Default: `info`. |
259+
| `PEBBLIFY_BASIC_AUTH_TOKEN` | `api.authentification_mode = basic_auth` | API bearer token. |
260+
| `PEBBLIFY_TELEGRAM_BOT_TOKEN` | `notify.enable = true` and `notify.mode = telegram` | Telegram Bot API token. Never logged. |
261+
| `PEBBLIFY_SCP_KEY_PATH` | `save.scp.enable = true` and `authentification_mode = key` | SSH private key path. |
262+
| `PEBBLIFY_SCP_PASSWORD` | `save.scp.enable = true` and `authentification_mode = password` | SSH login password. |
263+
| `PEBBLIFY_S3_SECRET_KEY` | `save.s3.enable = true` | AWS secret access key. |
264+
| `AWS_REGION` | `save.s3.enable = true` (recommended) | AWS region for S3 uploads. |
265+
| `AWS_DEFAULT_REGION` | `save.s3.enable = true` (alternative) | Fallback AWS region. |

0 commit comments

Comments
 (0)