docs(kernel-utils): Improve sheaf documentation

Author: grypez

packages/kernel-utils/src/sheaf/LIFT.md

@@ -0,0 +1,139 @@
+# Lift
+
+The lift is the caller-supplied selection policy in the sheaf dispatch
+pipeline. It runs when the stalk at an invocation point contains more than one
+germ and the sheaf has no data to resolve the ambiguity on its own. The caller
+is responsible for writing a lift that is correct for the sections it will
+receive.
+
+## Coroutine protocol
+
+The lift is an `async function*` generator, not a plain async function:
+
+```ts
+type Lift<M> = (
+  germs: EvaluatedSection<Partial<M>>[],
+  context: LiftContext<M>,
+) => AsyncGenerator<EvaluatedSection<Partial<M>>, void, unknown[]>;
+```
+
+The sheaf drives it with the following protocol:
+
+1. **Prime** — `gen.next([])` starts the coroutine. The empty array is
+   discarded; it exists only to satisfy the generator type.
+2. **Yield** — the coroutine yields a candidate germ to try next.
+3. **Attempt** — the sheaf calls the candidate's exo method.
+4. **Success** — the result is returned; the generator is abandoned.
+5. **Failure** — the sheaf calls `gen.next(errors)`, passing the ordered list
+   of every error thrown so far (cumulative, not just the last). The coroutine
+   receives this as the resolved value of its `yield` expression.
+6. **Exhausted** — if the generator returns without yielding, the sheaf
+   rethrows the last error.
+
+Most lifts express a fixed priority order and can ignore the error input:
+
+```ts
+const awayLift: Lift<AwayMeta> = async function* (germs) {
+  yield* germs.filter((g) => g.metadata?.mode === 'delegation');
+  yield* germs.filter((g) => g.metadata?.mode === 'call-home');
+};
+```
+
+A lift that inspects failure history can read the errors from yield:
+
+```ts
+const cautious: Lift<Meta> = async function* (germs) {
+  for (const germ of germs) {
+    const errors: unknown[] = yield germ;
+    // errors is the cumulative list of all failures so far, including the one
+    // just returned for this germ. Inspect to decide whether to continue.
+    if (errors.some(isUnrecoverable)) return;
+  }
+};
+```
+
+## LiftContext
+
+The second argument to the lift is a `LiftContext`:
+
+```ts
+type LiftContext<M> = {
+  method: string; // the method being dispatched
+  args: unknown[]; // the invocation arguments
+  constraints: Partial<M>; // metadata keys identical across every germ
+};
+```
+
+**`constraints`** are metadata keys whose values are the same on every germ in
+the stalk. Because all candidates agree on these keys, they carry no
+information useful for choosing between them — the sheaf strips them from each
+germ and delivers them separately. A lift that needs to know, say, the agreed
+`protocol` version reads it from `context.constraints.protocol` rather than
+from any individual germ.
+
+**`args`** is available for cases where the lift itself must inspect the call.
+Most of the time, however, arg-dependent selection is better expressed as
+`callable` metadata on the sections than as conditional logic in the lift.
+
+Consider a swap where each provider has a different cost curve over volume.
+Encode each provider's cost as `callable` metadata evaluated at dispatch time:
+
+```ts
+const sections: PresheafSection<SwapMeta>[] = [
+  {
+    exo: providerAExo,
+    metadata: callable((args) => ({ cost: providerACost(Number(args[0])) })),
+  },
+  {
+    exo: providerBExo,
+    metadata: callable((args) => ({ cost: providerBCost(Number(args[0])) })),
+  },
+];
+```
+
+By the time the lift runs, `germ.metadata.cost` already holds the concrete
+cost for this specific invocation — the swap amount has been applied. A lift
+that sorts by cost needs no knowledge of `args` at all:
+
+```ts
+const cheapestFirst: Lift<SwapMeta> = async function* (germs) {
+  yield* [...germs].sort(
+    (a, b) => (a.metadata?.cost ?? 0) - (b.metadata?.cost ?? 0),
+  );
+};
+```
+
+This is why evaluable metadata exists: the arg-dependent logic lives with the
+sections that own it, and the lift stays a pure selection policy.
+
+## Semantic equivalence assumption
+
+Two sections may differ in real ways — one might use TCP and the other UDP; one
+might be a Rust implementation and the other JavaScript. The semantic
+equivalence contract does not require that two sections be identical. It
+requires only that **if two sections are indistinguishable by metadata, their
+differences are immaterial to the authority invoker**.
+
+The sheaf relies on the following separation of responsibilities:
+
+- **Section constructors** are responsible for advertising every feature that
+  matters to callers. If transport protocol, latency tier, cost curve, or
+  freshness guarantee could affect the invoker's decision, it belongs in the
+  section's metadata. Omitting a distinguishing feature is a declaration that
+  callers need not care about it.
+
+- **Lift constructors** are responsible for selecting among the features that
+  section constructors have chosen to expose. The lift cannot see what was not
+  advertised.
+
+This is a semantic contract, not a runtime enforcement — the sheaf cannot
+verify it. When a section constructor omits a feature from metadata, they are
+asserting: for any authority invoker using this sheaf, that feature is
+irrelevant. If the assertion is wrong, the collapse step may silently discard a
+candidate that the lift would have ranked differently.
+
+> One `getBalance` provider uses a fully-synced node; another uses a lagging
+> replica. If both are tagged `{ cost: 1 }` with no freshness field, the
+> section constructors are asserting that freshness is immaterial to callers of
+> this sheaf. If that is not true, `{ cost: 1, freshness: 'lagging' }` vs
+> `{ cost: 1, freshness: 'live' }` would let the lift choose.

packages/kernel-utils/src/sheaf/README.md

@@ -7,6 +7,9 @@ over a presheaf of capabilities. The sheaf grants revocable dispatch sections
 via `getSection`, tracks all delegated authority, and supports point-wise
 revocation.
 
+See [USAGE.md](./USAGE.md) for annotated examples and [LIFT.md](./LIFT.md) for
+the lift coroutine protocol and semantic equivalence assumption.
+
 ## Concepts
 
 **Presheaf section** (`PresheafSection`) — The input data: a capability (exo)
@@ -33,15 +36,21 @@ entries.
 > Stalk at `("getBalance", "alice")` might contain two germs (cost 1 vs 100);
 > stalk at `("transfer", ...)` might contain one.
 
-**Lift** — An async function that selects one germ from a multi-germ stalk.
+**Lift** — An `async function*` coroutine that yields candidates from a
+multi-germ stalk in preference order. See [LIFT.md](./LIFT.md) for the
+coroutine protocol, `LiftContext`, and the semantic equivalence assumption
+required of all lifts.
+
 At dispatch time, metadata is decomposed into **constraints** (keys with the
 same value across every germ — topologically determined, not a choice) and
 **options** (the remaining keys — the lift's actual decision space). The lift
 receives only options on each germ; constraints arrive separately in the
 context.
 
 > `argmin` by cost, `argmin` by latency, or any custom selection logic. The
-> lift is never invoked for single-germ stalks.
+> lift is never invoked when the stalk resolves to a single germ — either
+> because only one section matched, or because all matching sections had
+> identical metadata and collapsed to one representative.
 
 **Sheaf** — The authority manager returned by `sheafify`. Holds the presheaf
 data (captured at construction time) and a registry of all granted sections.
@@ -50,7 +59,8 @@ data (captured at construction time) and a registry of all granted sections.
 const sheaf = sheafify({ name: 'Wallet', sections });
 ```
 
-- `sheaf.getSection({ guard?, lift })` — produce a revocable dispatch exo
+- `sheaf.getSection({ guard, lift })` — produce a revocable dispatch exo
+- `sheaf.getDiscoverableSection({ guard, lift, schema })` — same, but the exo exposes its guard
 - `sheaf.revokePoint(method, ...args)` — revoke every granted section whose
   guard covers the point
 - `sheaf.getExported()` — union guard of all active (non-revoked) sections
@@ -62,13 +72,25 @@ At each invocation point `(method, args)` within a granted section:
 
 ```
 getStalk(sections, method, args)     presheaf → stalk (filter by guard)
+evaluateMetadata(stalk, args)        metadata specs → concrete values
 collapseEquivalent(stalk)            locality condition (quotient by metadata)
 decomposeMetadata(collapsed)         restriction map (constraints / options)
 lift(stripped, { method, args,       operational selection (extra-theoretic)
                 constraints })
-dispatch to collapsed[index].exo     evaluation
+dispatch to chosen.exo               evaluation
 ```
 
+The pipeline short-circuits at two points: if only one section matches the
+guard, it is invoked directly without evaluate/collapse/lift; if all matching
+sections collapse to an identical germ, the single representative is invoked
+without calling the lift.
+
+`callable` and `source` metadata specs make the stalk shape depend on the
+invocation arguments. A `swap(amount)` section can produce `{ cost: 'low' }`
+for small amounts and `{ cost: 'high' }` for large ones, yielding a different
+set of germs — and potentially a different lift outcome — for the same method
+called with different arguments.
+
 ## Design choices
 
 **Germ identity is metadata identity.** The collapse step quotients by

packages/kernel-utils/src/sheaf/USAGE.md

@@ -0,0 +1,177 @@
+# Usage
+
+## Single provider
+
+When there is only one section per invocation point, no lift is needed — the
+dispatch short-circuits before the lift is ever called. Provide a no-op lift
+as a placeholder:
+
+```ts
+import { M } from '@endo/patterns';
+import { makeDefaultExo } from '<internal>';
+import { sheafify } from '@metamask/kernel-utils';
+
+const noop = async function* (germs) {
+  yield* germs;
+};
+
+const priceGuard = M.interface('PriceService', {
+  getPrice: M.callWhen(M.await(M.string())).returns(M.await(M.number())),
+});
+
+const priceExo = makeDefaultExo('PriceService', priceGuard, {
+  async getPrice(token) {
+    return fetchPrice(token);
+  },
+});
+
+const sheaf = sheafify({
+  name: 'PriceService',
+  sections: [{ exo: priceExo }],
+});
+
+const section = sheaf.getSection({ guard: priceGuard, lift: noop });
+// section is a revocable dispatch exo; call it like any capability
+const price = await E(section).getPrice('ETH');
+```
+
+## Multiple providers with a lift
+
+When the stalk at a given invocation point contains more than one germ, the
+sheaf calls the lift to choose. The lift is an `async function*` coroutine that
+yields candidates in preference order; it receives accumulated errors as the
+argument to each subsequent `.next()` so it can adapt its ranking.
+
+The idiomatic pattern is a generator that `yield*`s candidates filtered by
+metadata, expressing priority tiers in source order:
+
+```ts
+import { sheafify, constant } from '@metamask/kernel-utils';
+import type { Lift } from '@metamask/kernel-utils';
+
+type WalletMeta = { mode: 'fast' | 'reliable' };
+
+const preferFast: Lift<WalletMeta> = async function* (germs) {
+  yield* germs.filter((g) => g.metadata?.mode === 'fast');
+  yield* germs.filter((g) => g.metadata?.mode === 'reliable');
+};
+
+const sheaf = sheafify<WalletMeta>({
+  name: 'Wallet',
+  sections: [
+    { exo: fastExo, metadata: constant({ mode: 'fast' }) },
+    { exo: reliableExo, metadata: constant({ mode: 'reliable' }) },
+  ],
+});
+
+// guard restricts which methods callers may invoke
+const section = sheaf.getSection({ guard: clientGuard, lift: preferFast });
+```
+
+The sheaf drives the generator: it primes it with `gen.next([])`, calls the
+chosen candidate, then passes any thrown errors back as `gen.next(errors)` so
+the lift can adapt before yielding the next candidate.
+
+Use the `constant`, `source`, or `callable` helpers to build metadata specs:
+
+```ts
+import { constant, source, callable } from '@metamask/kernel-utils';
+
+// static value known at construction time
+constant({ mode: 'fast' });
+
+// JS source string compiled once in the sheaf's compartment at construction time
+source(`(args) => ({ cost: args[0] > 9000 ? 'high' : 'low' })`);
+
+// live function evaluated at each dispatch — useful when cost varies by argument,
+// e.g. a swap whose metadata encodes volume-based cost tiers
+callable((args) => ({ cost: Number(args[0]) > 9000 ? 'high' : 'low' }));
+```
+
+## Discoverable sections
+
+`getDiscoverableSection` works like `getSection` but the returned exo exposes
+its guard — it can be introspected by the caller to discover what methods and
+argument shapes it accepts. Use this when the recipient needs to advertise
+capability to a third party. It requires a `schema` map describing each method:
+
+```ts
+import type { MethodSchema } from '@metamask/kernel-utils';
+
+const schema: Record<string, MethodSchema> = {
+  getPrice: { description: 'Get the current price of a token.' },
+};
+
+const section = sheaf.getDiscoverableSection({
+  guard: clientGuard,
+  lift,
+  schema,
+});
+```
+
+`getSection` is the non-discoverable variant (no `schema` required).
+
+`getGlobalSection` and `getDiscoverableGlobalSection` derive the guard
+automatically from the union of all presheaf sections. They are `@deprecated`
+as a nudge toward explicit guards once the caller knows the section set —
+explicit guards make the capability's scope visible at the call site. When
+sections are assembled dynamically (e.g., rebuilt at runtime from a set of
+grants that changes) and the union guard isn't known until after `sheafify`
+runs, the global variants are the right choice.
+
+## Revocation
+
+```ts
+// revoke every granted section whose guard covers this invocation point
+sheaf.revokePoint('getPrice', 'ETH');
+
+// revoke all granted sections at once
+sheaf.revokeAll();
+
+// union guard of all currently active (non-revoked) sections
+const exported = sheaf.getExported();
+```
+
+## Remote sections
+
+`makeRemoteSection` wraps a CapTP remote reference as a `PresheafSection`,
+fetching the remote's guard once at construction and forwarding all calls via
+`E()`. This lets you mix local exos and remote capabilities in the same sheaf:
+
+```ts
+import { makeRemoteSection, constant } from '@metamask/kernel-utils';
+
+const remoteSection = await makeRemoteSection(
+  'RemoteWallet', // name for the wrapper exo
+  remoteCapRef, // CapTP reference
+  constant({ mode: 'remote' }), // optional metadata
+);
+
+const sheaf = sheafify({
+  name: 'Mixed',
+  sections: [localSection, remoteSection],
+});
+```
+
+## Lift composition
+
+`@metamask/kernel-utils` exports helpers for building lifts from composable
+parts, useful when lift logic would otherwise be duplicated across callers:
+
+```ts
+import {
+  proxyLift,
+  withFilter,
+  withRanking,
+  fallthrough,
+} from '@metamask/kernel-utils';
+```
+
+- **`withRanking(comparator, inner)`** — sort germs by comparator before
+  passing to `inner`
+- **`withFilter(predicate, inner)`** — remove germs that fail `predicate`
+  before passing to `inner`
+- **`fallthrough(liftA, liftB)`** — try all candidates from `liftA` first;
+  if all fail, try `liftB`
+- **`proxyLift(inner)`** — forward yielded candidates up and error arrays down;
+  useful when wrapping a lift in middleware