Skip to content

QUICKSTART.md

Latest commit

 

History

History
438 lines (348 loc) · 17.9 KB

QUICKSTART.md

File metadata and controls

438 lines (348 loc) · 17.9 KB
Raw

Quickstart

10 minutes from zero to a working Benten handler.

1. Install

npx create-benten-app my-app
cd my-app
npm install
npm test

The scaffolder drops a minimal TypeScript project:

  • @benten/engine — the napi-rs-wrapped engine
  • A handler file with a crud('post') one-liner
  • A smoke test exercising create / get / list / update / delete

2. Your first handler

The zero-config path:

import { crud } from "@benten/engine";

export const postHandlers = crud("post");

crud("post") exposes create, get, list, update, and delete actions with sensible defaults: properties inferred from input, no authentication required, local storage.

3. Use it

import { Engine } from "@benten/engine";
import { postHandlers } from "./handlers.js";

const engine = await Engine.open(".benten/my-app.redb");
const handler = await engine.registerSubgraph(postHandlers);
// `handler.id` is "crud:post" — the engine derives it as `crud:<label>` for
// crud()-registered handlers. Capture the returned handle and pass
// `handler.id` to `engine.call` (as below) so future label-format tweaks
// don't break call sites. The action is the second argument to `engine.call`.

// Create
const created = await engine.call(handler.id, "post:create", {
  title: "Hello Benten",
  body: "First post.",
});
console.log(created.cid);

// List
const listed = await engine.call(handler.id, "post:list", {});
console.log(listed.items);

// Update
await engine.call(handler.id, "post:update", {
  cid: created.cid,
  patch: { body: "Edited body." },
});

// Delete
await engine.call(handler.id, "post:delete", { cid: created.cid });

Adding capabilities

When you need authentication, open the engine with the grant-backed policy and stamp a capability on the handler:

import { Engine, PolicyKind, crud } from "@benten/engine";

const engine = await Engine.openWithPolicy(
  ".benten/my-app.redb",
  PolicyKind.GrantBacked,
);
const handler = await engine.registerSubgraph(
  crud("post", { capability: "store:post:*" }),
);

// Grant the wildcard capability — permits create/update/delete under the
// `post` label because `store:post:*` attenuates to `store:post:write`.
await engine.grantCapability({ actor: "alice", scope: "store:post:*" });

// `callAs` accepts either a real CID or a friendly principal string.
await engine.callAs(handler.id, "post:create", { title: "x" }, "alice");

The default PolicyKind.NoAuth permits everything (the embedded / single-user model). Swap in PolicyKind.GrantBacked for the revocation-aware Phase-1 policy. Phase 3 added PolicyKind.Ucan — a durable UCAN-grant policy backend (Rust-side struct UCANBackend in benten-caps) over benten-id's claim envelope + chain validation surface (DID-based actor identity, nbf/exp time-windows, attenuation along delegation chains, constant-time signature comparison). See the Atrium walkthrough below.

Diagnosing denied reads

Under a grant-backed policy, a denied read returns null — byte-identical with a genuine miss. That's deliberate: an unauthorized caller cannot distinguish existence from permission by probing CIDs.

This symmetric-None surface covers more than just Engine::get_node: Phase 2a threaded the same collapse into the evaluator dispatch itself (the Option C contract — denied-vs-miss collapse at the evaluator's pre-read hook), so a READ primitive inside a user subgraph observes the same behaviour (denied → null, backend miss → null) through PrimitiveHost::check_read_capability. Handlers running through engine.call(...) honour the same honest-no boundary end-to-end under Option C — there is no evaluator-side backdoor around the public-API contract.

If you're the operator and need to tell "denied" apart from "not found" (debugging a missing grant, for example), grant yourself the store:debug:read capability and call engine.diagnoseRead:

await engine.grantCapability({ actor: "alice", scope: "store:debug:read" });

const info = await engine.diagnoseRead(cid);
if (info.notFound) {
  console.log("never written (or deleted)");
} else if (info.deniedByPolicy) {
  console.log(`exists, missing grant for ${info.deniedByPolicy}`);
} else {
  console.log("exists and is readable");
}

Without store:debug:read, diagnoseRead throws E_CAP_DENIED — ordinary callers still cannot distinguish the two cases. Under PolicyKind.NoAuth the method is open.

Suspending and resuming (Phase 2a)

Some workflows wait for an external event — a webhook confirming payment, a human approval, an AI assistant's next turn. WAIT suspends execution and hands back a SuspendedHandle you persist:

import { promises as fs } from "node:fs";

const paymentHandler = subgraph("checkout")
  .action("charge")
  .read({ label: "cart", by: "id", value: "$input.cart_id" })
  .wait({
    signal: "external:payment_confirmed",
    signal_shape: "{ amount: Int, currency: Text }",
  })
  .write({ label: "order", properties: { status: "paid" } })
  .respond({ body: "$result" });

await engine.registerSubgraph(paymentHandler.build());

const result = await engine.callWithSuspension("checkout", "charge", {
  cart_id: "c-42",
});
if (result.kind === "suspended") {
  const bytes = result.handle;
  await fs.writeFile(".benten/suspended/checkout-c-42.cbor", bytes);
}

// Later, in a different process, after restart:
const bytes = await fs.readFile(".benten/suspended/checkout-c-42.cbor");
const outcome = await engine.resumeFromBytes(bytes, {
  amount: 19900,
  currency: "USD",
});

Tampered bytes, the wrong principal, or a grant revoked between suspend and resume all surface as typed errors before any write runs (E_EXEC_STATE_TAMPERED, E_RESUME_ACTOR_MISMATCH, E_RESUME_SUBGRAPH_DRIFT, E_CAP_REVOKED_MID_EVAL). The timed form wait({ duration: "5m" }) fires E_WAIT_TIMEOUT if no resume arrives in time.

Inspecting handlers

Handlers are data. You can visualize them:

console.log(handler.toMermaid());
// Mermaid flowchart you can paste into any Markdown viewer.

And trace a call:

const trace = await engine.trace(handler.id, "post:create", {
  title: "Test",
  body: "Trace me",
});
console.log(trace.steps);
// Array of { nodeCid, primitive, durationUs, inputs?, outputs? } — one entry
// per OperationNode executed. `engine.trace` does not persist the outcome or
// fire a ChangeEvent; it's safe to run repeatedly.

Streaming results back to the client (STREAM — Phase 2b)

Long-running queries (large list pulls, log tailers, aggregate exports) shouldn't materialise the full result before responding. The STREAM primitive yields chunks as they're produced; the JS-side engine.callStream returns an AsyncIterable<Chunk> you can for await ... of:

import { Engine, subgraph } from "@benten/engine";

const exportHandler = subgraph("export-feed")
  .read({ label: "post", as: "rows" })
  .iterate({ over: "$result.rows", max: 100_000 })
  .stream({ source: "$loop.row", chunkSize: 64 })
  .respond({ body: "{ status: \"streamed\" }" })
  .build();

await engine.registerSubgraph(exportHandler);

for await (const chunk of engine.callStream("export-feed", "default", {})) {
  process.stdout.write(chunk);          // chunks arrive as `Buffer`
}
// `for await` calls `return()` on early break → underlying mpsc
// receiver closes promptly; no leaked tasks.

Back-pressure is handled Rust-side; the JS iterator is a thin shell. If the consumer slows, the producer (the IVM-driven evaluator) blocks on the mpsc bound and stops doing work. Closing the iterator tears down the producer in O(1).

Reacting to changes (SUBSCRIBE — Phase 2b)

SUBSCRIBE is the reactive primitive: a handler subscribes to a label's ChangeEvent stream and runs each event through a downstream DAG (commonly TRANSFORM + WRITE to project a derived view).

const projectHandler = subgraph("post-summary-view")
  .subscribe({ event: "post:changed" })
  .transform({ expr: "{ id: $event.cid, title: $event.body.title }" })
  .write({ label: "post-summary" })
  .emit({ event: "post-summary:built" })
  .build();

await engine.registerSubgraph(projectHandler);

// Engine wires the SUBSCRIBE node into the change-event bus on
// register. Each post:changed event runs the downstream DAG once.
// SUBSCRIBE handlers do NOT need an explicit `engine.call` — the
// runtime drives them.

A SUBSCRIBE handler that errors fires an E_SUBSCRIBE_PROJECTION on the engine error channel; the handler stays subscribed (an erroring event doesn't poison the subscription) and the next event drives a fresh frame. Cursor durability — the "where in the change-event stream did this subscriber leave off" — is persisted by the generalised SuspensionStore.

Calling out to WASM (SANDBOX — Phase 2b)

When you need pure-CPU compute over arbitrary bytes (text summarisation, image resampling, format conversion, hashing beyond BLAKE3) the SANDBOX primitive runs a precompiled WASM module under the wasmtime host with capability-derived host-fn manifest:

import { Engine, subgraph } from "@benten/engine";

await engine.installModule({ name: "example.summarizer", version: "0.1.0",
  modules: [{ name: "summarize-v1", cid: moduleCid,
    requires: ["host:compute:log", "host:compute:time", "host:compute:kv:read"] }],
}, manifestCid);

const handler = subgraph("summarize")
  .read({ label: "doc", by: "id", value: "$input.doc_id" })
  .sandbox({ module: "example.summarizer:summarize-v1",
    fuel: 1_000_000, wallclockMs: 30_000, outputLimitBytes: 1_048_576 })
  .write({ label: "summary" })
  .respond({ body: "$result" })
  .build();
await engine.registerSubgraph(handler);
const out = await engine.call("summarize", "default", { doc_id: "d-42" });

A SANDBOX call with a manifest the caller's policy doesn't grant fires E_CAP_DENIED at SANDBOX entry; an in-module host-fn call to a host-fn outside the manifest fires E_SANDBOX_HOST_FN_NOT_ON_MANIFEST. Inv-4 (nest depth) and Inv-7 (cumulative output) both fire as E_INV_SANDBOX_DEPTH and E_INV_SANDBOX_OUTPUT respectively. See HOST-FUNCTIONS.md for the full host-fn surface and SANDBOX-LIMITS.md for limit defaults.

SANDBOX is composition-only — there is no top-level engine.sandbox(...) API. A SANDBOX node always lives inside a handler (so capability resolution, Inv-4 nest-depth, and Inv-14 attribution chaining all flow through the evaluator).

Runnable example handlers ship at packages/engine/examples/ covering all three Phase-2b primitives plus four Phase-3 Atrium examples (see below).

Joining an Atrium (Phase 3)

Note (Phase-3-close state). The durable Phase-3 UCAN backend ships in Rust at crates/benten-caps/src/backends/ucan.rs (struct UCANBackend). The TS-surface PolicyKind.Ucan enum variant is wired in the napi binding at bindings/napi/src/lib.rs via EngineBuilder::capability_policy_ucan_durable() — runtime end-to-end is LIVE. The four end-to-end Atrium runner examples below (atrium-peer-mgmt.ts, atrium-sync-trigger.ts, ucan-grant-flow.ts, did-resolution.ts) execute against the durable backend; the companion Vitest pin packages/engine/test/atrium_examples.test.ts verifies the SHAPE half. Phase-3-close exit criteria 1 (two full-peer iroh sync), 15 (3+-peer concurrent-write convergence), and 16 (multi-device cryptographic-attestation closure) are all GREEN at tag phase-3-close.

An Atrium is a peer-to-peer-synchronized graph shared by a small set of full-peer engines (a user's laptop + phone-OS app + desktop; or a community of cooperating users). The engine.atrium({...}) factory call returns an Atrium handle whose .join() initiates peer discovery + handshake; the same handle exposes trustPeer(), listPeers(), subscribe(), and declareDeviceAttestation():

import { Engine, PolicyKind } from "@benten/engine";

// Phase 3's durable UCANBackend is selected via PolicyKind.Ucan.
// (Phase-2b: PolicyKind.GrantBacked is the revocation-aware
// per-actor policy; Phase 3 layered UCAN attenuation + chain
// validation on top via benten-id.)
const engine = await Engine.openWithPolicy(
  ".benten/my-app.redb",
  PolicyKind.Ucan,
);

// `engine.atrium` is a callable factory; each call returns a fresh
// handle.
const family = engine.atrium({ atriumId: "family" });
await family.join();

// Trust a peer DID — extends the trust set this handle uses for
// per-session subscriptions + device-attestation walking.
await family.trustPeer("did:key:z6MkrJVnaZkeFzdQyMZu1c...");

const peers = family.listPeers();
console.log(peers); // ["did:key:z6Mk...", ...]

// Subscribe to a path; handler fires on each ChangeEvent the full
// peer routes through the cap-recheck pipeline.
const sub = await family.subscribe("/zone/posts", (event) => {
  console.log("post changed", event);
});

// Tear down the per-session state when done.
await sub.unsubscribe();
await family.leave();

// The handle survives leave: rejoin re-establishes peer discovery
// against the surviving trust-store + device-attestation table, so
// AttributionFrame.peer_did_set continuity is preserved across the
// leave/rejoin cycle. Use `isActive()` to gate calls that require
// the handle's iroh endpoint to be running.
if (!family.isActive()) {
  await family.rejoin();
}

// Multi-device attestation: declare this device's signed envelope
// before peers will accept its writes as originating from this
// identity. The envelope is an Ed25519-signed structure that binds
// device-DID to identity-DID with a payload-hash + session-nonce
// for replay defense.
await family.declareDeviceAttestation({
  envelope: signedDeviceAttestationEnvelope, // produced by
                                              // `Acceptor::accept_at`
                                              // on the identity side
});

A full peer is the durable Atrium participant — the Rust crate set includes benten-id (DID + UCAN claim envelope) and benten-sync (the iroh + Loro + Merkle Search Tree sync runtime). Browser engines do not join Atriums directly; they consume engine state via the thin-client protocol (fetch GET for snapshot reads + POST with a device-DID-signed auth header for writes + Server-Sent-Events for ChangeEvent streams). See the four runnable examples shipped at packages/engine/examples/:

  • atrium-peer-mgmt.ts — join, trustPeer, listPeers, revokePeer, peer-lifecycle hooks (onPeerJoin / onPeerLeave), leave.
  • atrium-sync-trigger.ts — manual + scheduled sync; observe MST diff stats.
  • ucan-grant-flow.ts — minting, attenuating, revoking UCAN grants end-to-end against the durable UCANBackend.
  • did-resolution.ts — did:key round-trip + (Phase-9-future) did:plc placeholder.

Compromise #22 (peer-DID + connection-metadata leakage to public iroh relays) is honestly disclosed in SECURITY-POSTURE.md; operators with stricter metadata threat models should self-host an iroh relay or wait for Phase 7 Garden-controlled relays.

What works today

Phase 1 shipped, Phase 2a closed at tag phase-2a-close, Phase 2b closed at tag phase-2b-close, and Phase 3 closes with this commit. Live:

  • crud('post') zero-config path
  • All 12 primitives (READ, WRITE, TRANSFORM, BRANCH, ITERATE, WAIT, CALL, RESPOND, EMIT, SANDBOX, SUBSCRIBE, STREAM) — Phase-2b added the last three.
  • WAIT with signal + duration variants and the full 4-step resume protocol
  • Capability enforcement via PolicyKind.GrantBacked, grantCapability, revokeCapability, and Phase-3's durable PolicyKind.Ucan (TS-surface variant; Rust-side struct named UCANBackend) over benten-id (DID-based actors, attenuation, revocation, time-window validation, constant-time signature comparison)
  • handler.toMermaid() visualization
  • engine.trace() step-by-step evaluation records
  • engine.diagnoseRead() operator introspection
  • engine.callStream() AsyncIterable + engine.openStream()
  • Reactive SUBSCRIBE handlers driven off the change-event bus
  • SANDBOX primitive with wasmtime host + named-manifest registry + durable BlobBackend (Phase 3 closed Compromise #17)
  • Durable handler-version chain via core::version::Anchor (Phase 3 closed Compromise #18)
  • Algorithm B generalized in Phase 3 — user-defined view IDs run under Strategy::B with their actual label patterns rather than being coerced to ContentListingView semantics (Compromise #11 closed)
  • Atriums — peer-to-peer sync between full peers (laptop / phone-OS app / desktop) over iroh QUIC + Merkle Search Tree diff + Loro CRDT merge
  • Browser-as-thin-client — wasm32 engines participate as views into full peers via fetch/POST + SSE; no iroh / Loro / SANDBOX in the wasm32 bundle; optional IndexedDB cache for snapshot data
  • Verify-on-read at every Node-bytes surface — RedbBackend::get_node routes through Node::load_verified(cid, &bytes) so on-disk tamper fires E_INV_CONTENT_HASH rather than returning the wrong-but-decodable Node

Not yet live:

  • Marketplace / dynamic manifest registration (register_runtime reserved with E_SANDBOX_MANIFEST_REGISTRATION_DEFERRED; Phase 8)
  • Garden-controlled iroh relays (Phase 7) — until then, public iroh relays leak peer-DID + connection-metadata at the transport layer (Compromise #22)
  • Inv-4 runtime depth-threading — both arms fully active. One adversarial integration test stays #[ignore]'d pending a testing_call_engine_dispatch host-fn helper — the runtime defense is wired; only the adversarial-test driver is paper-only. See docs/INVARIANT-COVERAGE.md "Inv-4 + Inv-7 runtime arm status" for the wiring trace.

If something in the "live" list doesn't behave as documented, file an issue.

Next: