Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
[workspace]
members = [
"apps/convex-inspect",
"apps/convex-sync",
"crates/convex-sync-core",
"crates/convex-export-s3",
Expand Down
21 changes: 18 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,12 +23,14 @@ extraction model:
```mermaid
flowchart TD
Root[convex-streaming-olap-export]
Inspect[apps/convex-inspect]
CLI[apps/convex-sync]
Core[crates/convex-sync-core]
S3[crates/convex-export-s3]
AWS[platform/aws]
DBS3[platform/databricks/s3]
DBN[platform/databricks/native]
Root --> Inspect
Root --> CLI
Root --> Core
Root --> S3
Expand All @@ -39,6 +41,7 @@ flowchart TD

Read the repo by layer:

- [`apps/convex-inspect/README.md`](apps/convex-inspect/README.md): direct source inspection commands
- [`apps/convex-sync/README.md`](apps/convex-sync/README.md): CLI surface and S3/export runtime commands
- `crates/convex-sync-core/`: shared Convex client, checkpoint FSM, event normalization, sync engine
- `crates/convex-export-s3/`: raw parquet sink, staging materialization, S3 publish flow
Expand Down Expand Up @@ -68,6 +71,12 @@ Current release coverage:
- `convex-sync-dev` is checkout-linked and rebuilds incrementally via Cargo
- release installs go to `~/.local/share/convex-sync/<version>/convex-sync`
- command symlinks go in `~/.local/bin`
- `convex-inspect` is repo-local today and not part of the release artifact

## Operator Binaries

- `convex-inspect`: inspect Convex schemas, snapshot pages, and delta pages directly
- `convex-sync`: run the maintained parquet -> staging -> S3 export workflow

## Supported Variations

Expand Down Expand Up @@ -95,17 +104,23 @@ The maintained Rust runtime path:

CLI:

- `cargo run -p convex-sync -- schemas`
- `cargo run -p convex-sync -- snapshot --table-name users`
- `cargo run -p convex-sync -- deltas --cursor 0`
- `cargo run -p convex-sync -- sync-once`
- `cargo run -p convex-sync -- materialize-staging`
- `cargo run -p convex-sync -- publish-s3 --bucket your-bucket`
- `cargo run -p convex-sync -- run --bucket your-bucket`

Inspection:

- `cargo run -p convex-inspect -- schemas`
- `cargo run -p convex-inspect -- snapshot --table-name users`
- `cargo run -p convex-inspect -- deltas --cursor 0`

Or via `just`:

- `just dev-cli --help`
- `just schemas`
- `just snapshot --table-name users`
- `just deltas --cursor 0`
- `just sync-once`
- `just materialize-staging`
- `just publish-s3 --bucket your-bucket`
Expand Down
17 changes: 17 additions & 0 deletions apps/convex-inspect/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
[package]
name = "convex-inspect"
edition.workspace = true
license.workspace = true
publish.workspace = true
version = "0.0.2"

[[bin]]
name = "convex-inspect"
path = "src/main.rs"

[dependencies]
clap.workspace = true
convex-sync-core = { path = "../../crates/convex-sync-core" }
dotenvy.workspace = true
tokio.workspace = true
url.workspace = true
30 changes: 30 additions & 0 deletions apps/convex-inspect/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# `convex-inspect` CLI

Direct source-inspection CLI for Convex snapshot and delta APIs.

```mermaid
flowchart LR
C[Convex]
S[schemas]
Snap[snapshot]
D[deltas]
C --> S
C --> Snap
C --> D
```

## Commands

- `schemas`: fetch Convex schema metadata
- `snapshot`: inspect snapshot pages directly
- `deltas`: inspect delta pages directly

## Help

```bash
cargo run -p convex-inspect -- --help
cargo run -p convex-inspect -- snapshot --help
```

This binary is intentionally repo-local for now. The released operator-facing
artifact remains `convex-sync`.
Loading
Loading