Skip to content

CONSUME-RUST.md

Latest commit

 

History

History
259 lines (199 loc) · 12.7 KB

CONSUME-RUST.md

File metadata and controls

259 lines (199 loc) · 12.7 KB
Raw

Using beastdb from a Rust project

Audience: app authors who want the heed-inspired crate as a dependency. Product name: always beastdb. In-process library name: libbeastdb. License: Unlicense (SPDX: Unlicense); crate license = "Unlicense" in ffi/rust/beastdb/Cargo.toml. ref/ submodules keep upstream licenses. Honesty: engine is Lean 4; this crate wraps the full C ABI v9 over the temporary export+runtime package (nix build .#beastdb-lib) - export default = @[export] + libleanshared in your process. That is not freestanding R7-RT close. Freestanding product library package is separately nix build .#libbeastdb (Systems Lean append-log spine; no runtime; size_t open/put/get/sync - not this crate's beastdb.h surface yet). Goal (E): freestanding libbeastdb depth that can serve full embed without managed runtime (LIMITS.md Embed TCB). Force CLI: BEASTDB_USE_CLI=1. R7 deeper partial; R7-SNAP: idle gets need reopen or successful Env::sync (not LMDB MVCC).

Dual package paths (Rust crate uses export residual)

Flake package What you get Use with this crate?
nix build .#beastdb-lib libbeastdb.so + CLI-only .a + beastdb.h + Lean runtime Yes - required for heed-shaped v9 today
nix build .#libbeastdb Freestanding libbeastdb.a spine only (no libleanshared) No - wrong ABI for this crate; C smoke/demo/thr only
nix build .#beastdb AOT product CLI CLI emergency / BEASTDB_BIN

Do not treat .#libbeastdb as an alias of .#beastdb-lib.

What you need at runtime (export residual path)

Artifact Role
libbeastdb.so (from beastdb-lib) C ABI + dual-backend + Lean runtime in-process (export package default). NEEDED: libleanshared
libbeastdb.a (from beastdb-lib) CLI-only bridge today - no Lean engine in the archive; ops fork/exec beastdb
libleanshared Lean 4 runtime (export path). Not optional for in-process export
beastdb-fsync On PATH for export open (requireFsync := true); same helper the product CLI wrap uses
beastdb AOT CLI Required for CLI emergency (BEASTDB_USE_CLI=1); also the only backend behind static .a
Export backend Same-process compiled @[export] + Lean runtime (native Lean code + runtime in the caller; not a source interpreter; not "no language runtime")
Rust crate ffi/rust/beastdb Heed-shaped API (Env, Database, dual write modes, ...) over the C ABI

The bridge resolves the CLI (emergency backend) as:

  1. Process environment BEASTDB_BIN (absolute path) - preferred when forcing CLI
  2. Else compile-time baked path (Nix beastdb-lib sets this)

Recommended: Nix + path/git dependency

From the beastdb monorepo (or a flake input):

# 1) Product CLI + temporary export+runtime library (NOT freestanding .#libbeastdb)
nix build .#beastdb -o /tmp/beastdb-cli
nix build .#beastdb-lib -o /tmp/beastdb-lib

export BEASTDB_BIN=/tmp/beastdb-cli/bin/beastdb
export BEASTDB_LIB_DIR=/tmp/beastdb-lib/lib
export BEASTDB_INCLUDE_DIR=/tmp/beastdb-lib/include
# optional: pkg-config
export PKG_CONFIG_PATH=/tmp/beastdb-lib/lib/pkgconfig${PKG_CONFIG_PATH:+:$PKG_CONFIG_PATH}

In your app Cargo.toml:

[dependencies]
# Path (local checkout of the full monorepo):
beastdb = { path = "../beastdb/ffi/rust/beastdb" }

# Or git (full monorepo - not a crates.io-only crate yet):
# beastdb = { git = "https://github.com/<org>/beastdb", package = "beastdb" }
use beastdb::{Database, EnvOpenOptions, Str};

fn main() -> beastdb::Result<()> {
    // BEASTDB_BIN must point at the AOT CLI if the library was not Nix-baked.
    let env = EnvOpenOptions::new().create(true).open("/tmp/my-store")?;
    let db = env.database::<Str, Str>();
    {
        let mut w = env.write_txn()?;
        db.put(&mut w, "hello", "world")?;
        w.commit()?; // durable sync
    }
    assert_eq!(db.get(&env, "hello")?, Some("world".into()));
    Ok(())
}

Build your app:

cargo build --release
# ensure BEASTDB_BIN is set in the environment when you run the binary
./target/release/my-app

Flake consumer (optional)

If your app is also a Nix flake, depend on this flake's packages.<system>.beastdb-lib and beastdb, set BEASTDB_LIB_DIR / rpath in the derivation, and wrap the app with BEASTDB_BIN.

Alternative: bundled-bridge (compile C from monorepo sources)

When you depend on a full monorepo checkout (path or git) and do not want a prebuilt libbeastdb.so:

[dependencies]
beastdb = { path = "../beastdb/ffi/rust/beastdb", features = ["bundled-bridge"] }
# Still need the Lean AOT product binary (Nix is the supported way to build it):
nix build github:<org>/beastdb#beastdb -o /tmp/beastdb-cli
export BEASTDB_BIN=/tmp/beastdb-cli/bin/beastdb

# Optional: bake a default path into the bridge at compile time
export BEASTDB_BIN_DEFAULT=$BEASTDB_BIN

cargo build --release

bundled-bridge compiles ffi/src/beastdb_bridge.c with the cc crate. It does not build Lean or replace the product engine.

What is committed vs regenerated

AGENTS.md policy (updated): Lean remains semantic SSOT. Lean's C backend may emit C; that output is not where store behavior is designed. For distribution / embed completeness, regenerated Lean C may be committed under ffi/generated/ only (never hand-edit; regenerate via the flake).

Artifact In git? Why
ffi/src/beastdb_bridge.c Yes Only hand-written bridge translation unit (agent-guarded; no extra glue .c)
ffi/include/beastdb.h Yes Only public C header (agent-guarded)
ffi/generated/** Yes (when vendored) Regenerated Lean C / link snapshots; not semantic SSOT; not hand-written glue
target/, ad-hoc result/ No Local build products

Today the tree may not yet contain a populated ffi/generated/ (first vendor lands via a documented nix run .#vendor-generated-c; production dual-backend already ships via Nix beastdb-lib with export opt-in). Until then, use Nix packages (beastdb + beastdb-lib) + this crate as above.

"Repo feels complete" for embedders = header + hand-written bridge + optional vendored generated C + Rust crate + Nix packages.

API surface (thin v0 + Phase 6 growth + D1 + D2)

See crate README and HEED-MAP.md:

  • Open / put / get / contains / delete / iter / prefix_iter / rev_iter / range / first / last / sync / close
  • Mode A immediate + Mode B batch + BatchFlushPolicy (size/count/max_delay)
  • Ordered scan via product C ABI v6/v8 cursor (RoIter; snapshot; tombstones hidden; internal-key order)
  • D1 (C ABI v9): Database::len / is_empty / clear / delete_range
  • D2: Env::read_txn / RoTxn (heed-shaped read view; not LMDB MVCC reader table); pure codecs U32BE / U64BE (no IntegerComparator); selective T2 suite
  • Not: mut cursors, multi-DB names, LMDB ACID, heed3 crypto, power-fail claims; order is not always unsigned external-byte; not full heed

Migration cookbook (heed non-encrypted -> beastdb): MIGRATE-HEED.md - side-by-side patterns + Skip walls.

D1: len / is_empty / clear / delete_range (honesty)

Heed-shaped convenience over product live-index + tombstone batch (API v9).

Method Semantics Not claimed
len / is_empty Count live keys only (tombs hidden) on the current product path (same class as get/contains; not cursor snapshot-at-open) Not LMDB page stats; Mode B unflushed buffer is invisible until flush/commit
clear Tombstone all live keys in one multi-key atomic batch; empty store is idempotent success Not a hard epoch wipe / free-page reclaim; WAL history remains until compact
delete_range Tombstone live keys whose internal keys fall in the bound range (Bound::{Unbounded,Included,Excluded}) Not unsigned external-byte / memcmp range; order follows bytesToKey (LSB-first bits per byte)

maxEntries headroom (G10): clear / delete_range go through product applyAtomicBatch, which refuses when totalEntries + batchSize > maxEntries (default maxEntries = 1_000_000). totalEntries is the sum of shard log lengths (historical puts and tombs), not the live-key count. So a store can still accept single puts yet fail closed on clear / delete_range once one tomb per matched live key no longer fits under the remaining budget. Compact (or otherwise reduce log length) to free headroom; fail-closed is intentional - not a silent partial clear.

use beastdb::{Bound, Database, EnvOpenOptions, Str};

fn d1_example(path: &str) -> beastdb::Result<()> {
    let env = EnvOpenOptions::new().create(true).open(path)?;
    let db = env.database::<Str, Str>();
    {
        let mut w = env.write_txn()?;
        db.put(&mut w, "a", "1")?;
        db.put(&mut w, "b", "2")?;
        w.commit()?;
    }
    assert_eq!(db.len(&env)?, 2);
    assert!(!db.is_empty(&env)?);
    // Internal-key range (not memcmp); empty match is Ok(())
    db.delete_range(&env, Bound::Included(b"a"), Bound::Included(b"a"))?;
    assert_eq!(db.len(&env)?, 1);
    db.clear(&env)?;
    assert!(db.is_empty(&env)?);
    env.sync()?; // multi-file checkpoint publish after clear
    Ok(())
}

Embed note: after clear / delete_range, prefer Env::sync when you need a durable multi-file checkpoint publish (same class as other immediate mutators). Plan for maxEntries headroom before large clear/range-delete batches (see table note above). This is not full heed (stat, mut cursors, named multi-DB, ...).

D2: RoTxn / read_txn (honesty)

use beastdb::{Database, EnvOpenOptions, Str};

fn d2_read(path: &str) -> beastdb::Result<()> {
    let env = EnvOpenOptions::new().create(true).open(path)?;
    let db = env.database::<Str, Str>();
    {
        let mut w = env.write_txn()?;
        db.put(&mut w, "k", "v")?;
        w.commit()?;
    }
    let rtxn = env.read_txn()?; // heed-shaped; not LMDB MVCC
    assert_eq!(db.get(&rtxn, "k")?, Some("v".into()));
    assert_eq!(db.len(&rtxn)?, 1);
    rtxn.commit(); // no-op end-of-view
    Ok(())
}
Op class Visibility
Point reads (get / contains / len) Current product path (last durable publish / Mode A put / Mode B flush)
Cursors (iter / range / ...) Product scan snapshot at open
Mode B unflushed buffer Invisible until flush/commit

Not claimed: LMDB reader table, max_readers, frozen RO snapshot id for point gets. Full migration table: MIGRATE-HEED.md.

Troubleshooting

Symptom Fix
undefined reference to beastdb_* Set BEASTDB_LIB_DIR or enable bundled-bridge
BEASTDB_BIN unset at runtime export BEASTDB_BIN=/path/to/beastdb
exec exit 127 Wrong BEASTDB_BIN path or non-executable
Corrupt open Store dir missing MANIFEST / bad control plane - fail closed
Want export backend (default) No env needed (PR2 package default); or BEASTDB_USE_EXPORT=1
Want CLI backend (emergency) BEASTDB_USE_CLI=1 at open only
required fsync failed on export Put beastdb-fsync on PATH (nix build .#beastdb wraps it)
Export peer visibility (R7-SNAP) Long-lived EXPORT Env holds an in-process product handle snapshot. Idle point reads do not auto-follow peer multi-process puts. Load a newer durable snapshot via: (1) reopen (drop/close then EnvOpenOptions::open again), or (2) successful Env::sync (beastdb_sync / product Api.sync refreshes the handle from disk truth). There is no dedicated beastdb_env_refresh in C ABI v9. Not LMDB MVCC / reader table. CLI emergency (BEASTDB_USE_CLI=1) reopens per call, so the next op sees peers without a long-lived snapshot. Same-process puts through this env update the handle as usual.

Related