Skip to content

feat(data): db.observe.derive + Database.Read read projection#146

Open
krisnye wants to merge 6 commits into
mainfrom
krisnye/observe-derive
Open

feat(data): db.observe.derive + Database.Read read projection#146
krisnye wants to merge 6 commits into
mainfrom
krisnye/observe-derive

Conversation

@krisnye

@krisnye krisnye commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator

Auto-tracking reactive derivation for the ECS. db.observe.derive(compute) runs compute against a read-only projection of the database, records exactly the reads it performs, and re-emits only when a recorded dependency changes — so the whole class of "read a field but forgot to subscribe to it" staleness bugs (e.g. the frameRate gap found in the FFP avTimelinePayload refactor) becomes impossible by construction.

Still a review PR — no version bump; not necessarily merging as-is.

Database.Read<DB>

The read-only projection the compute callback receives. Keeps get / read / select, resources, index lookups (find / findRange / get), and archetype identity (components / id). Structurally omits observe (a derive subscribes for you), transactions/writes, queryArchetypes, locate, and per-archetype columns / rowCount / insert. Misuse is a compile error, not a runtime throw. Supporting type Database.Index.ReadHandle<C,I> = a handle minus observe.

read overloads

  • read(entity, archetype) narrowed to Readonly<T> (was Readonly<T> & EntityReadValues<C>); minArchetypearchetype. Blast radius across adobe/data: zero — no caller was leaning on the whole-entity union.
  • new read(entity, components[]) — a projection (present-or-undefined; null only when the entity is absent). Names the exact fields touched so a derive can scope its dependencies to them.

derive engine (observe-derive.ts)

  • One read-recording wrapper per database, shared across all derives; its recorded-dep set is reset per synchronous compute run. Safe to share because compute is synchronous and the read surface exposes no writes/observers/nested-derives, so runs can't overlap or recurse. (No Proxy — the read surface is a finite, explicit wrapper.)
  • Dependencies reduce to two kinds, both resolved from the transaction result:
    • entity depsget / read(entity, archetype|components[]) → recompute when that entity is in changedEntities and (for scoped reads) a watched component changed on it, or it was deleted;
    • column depsselect include/exclude/where/order, an index's readColumns, and resource names → recompute when the transaction's changedComponents intersects them.
  • One transaction subscription per derive; recompute coalesced on a microtask; emit gated by structural equals (no re-emit on an equal result).
  • Component-scoped granularity is opt-in via which read you call: read(e) = whole-entity dep, read(e, ["a"]) / get(e,"a") = just those fields.

Tests

  • observe-derive.test.ts: initial synchronous emit; re-emit on a read component; no re-emit on an unread component (projection scoping) vs whole-entity coarseness; structural dedupe; select membership, index-bucket, and resource deps; coalescing; disposal.
  • database-read.type-test.ts: positive read surface + @ts-expect-error for every omitted API (an unused expect-error fails tsc, so these prove each footgun is blocked).
  • typecheck + lint + 911 ecs/observe tests green.

Notes / follow-ups

  • Per-commit cost is O(Σ recorded deps across live derives) — the intended fine-grained trade.
  • A no-op update records the entity in changedEntities, so it triggers a recompute that structural dedupe then suppresses (correct, mildly wasteful).
  • Indexes/resources added by a later extend aren't in the shared recorder (built lazily on first use); fine for the create-time plugin case, a follow-up for dynamic extends.

krisnye and others added 2 commits July 14, 2026 16:40
…nly)

Design-review PR — the type surface for a reactive-derivation primitive, no
runtime implementation yet.

- `Database.Read<DB>`: the read-only projection a derive callback receives.
  Keeps value reads (`get`/`read`/`select`), resource reads, index LOOKUPS
  (`find`/`findRange`/`get`), and archetype identity (`components`/`id`).
  Structurally OMITS observers, transactions/writes, `queryArchetypes`,
  `locate`, and per-archetype table/column/`insert` access — so touching a
  non-derivable API is a compile error, not a runtime throw.
- `Database.Index.ReadHandle<C, I>`: an index handle minus `observe`.
- `db.observe.derive<T>(compute: (db: Database.Read<…>) => T, options?)
  : Observe<T>` — signature only; the intent is that a derive records the
  reads `compute` performs and re-emits when any could have changed.

Compile-time tests (`database-read.type-test.ts`) pin the positive surface
and, via `@ts-expect-error`, that `observe` / `transactions` /
`queryArchetypes` / `locate` / `columns` / `rowCount` / `insert` /
`indexes.*.observe` are all rejected. typecheck + lint clean.

No implementation and no version bump — for reviewing the types; not to merge.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implements the spec from the prior commit.

read overloads (core.ts):
- read(entity, archetype) narrowed to `Readonly<T>` (was
  `& EntityReadValues<C>`); `minArchetype` renamed to `archetype`. The
  narrowing's blast radius across adobe/data was zero.
- new read(entity, components[]) projection — present-or-undefined, `null`
  only when the entity is absent — enabling component-scoped derive deps.

derive engine (observe-derive.ts):
- ONE read-recording wrapper per database, shared across all derives; its
  recorded-dep set is reset per synchronous compute run. Safe to share because
  compute is synchronous and the read surface exposes no writes / observers /
  nested derives, so runs cannot overlap or recurse.
- deps reduce to entity deps (gated on changedEntities + changed-component
  intersection) and column deps (select include/exclude/where/order, index
  readColumns, resource names) checked against changedComponents. One
  transaction subscription per derive; recompute coalesced on a microtask;
  emit gated by structural `equals`.

Tests: observe-derive.test.ts — initial emit, component-scoped re-emit vs
whole-entity coarseness, structural dedupe, select / index / resource deps,
coalescing, disposal. Type tests updated for the projection read.
typecheck + lint + 911 ecs/observe tests green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@krisnye krisnye changed the title spec(data): Database.Read + db.observe.derive types (design review, no impl) feat(data): db.observe.derive + Database.Read read projection Jul 15, 2026
krisnye and others added 4 commits July 14, 2026 19:41
…sced

`observeTransactions` fires once per committed transaction, so a derive already
recomputes at most once per commit — deferring the emit to a microtask only
coalesced *across* commits in one turn, which starves consumers that route per
commit (e.g. a burst of ephemeral drag commits, where each must produce its own
downstream effect). Emit synchronously at the commit boundary instead — the same
cadence as `observe.entity` / the raw component observers a hand-written computed
would use. Structural dedupe still suppresses unchanged results.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`read(entity, components[])` built the full row via `getRowData` and then
discarded the columns it didn't need. Read only the requested components'
columns directly (`archetype.columns[c].get(row)`), skipping absent ones — no
whole-row allocation or reads of columns the caller didn't ask for.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ructural

Redefine `Database.Read<DB>` by mapping over `DB`'s own members instead of
`DB extends Database<infer …>`. The `infer` form collapsed an intersection
`A & B` to a single matched member; the structural form distributes, so
`Read<A & B>` merges both read surfaces.

Move `derive` off `db.observe.derive` to a top-level `db.derive` whose callback
receives `Database.Read<this>`. On an intersection receiver `this` resolves to
`A & B`, so a derive over a composed database sees indexes + resources +
archetypes from both members — consumers of an `A & B` database no longer cast.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ip-level select

The derive read-recorder previously reduced index lookups and `select`s to a
coarse global column set: any write to a backing column, anywhere, re-ran the
whole compute body (suppressed only downstream by structural dedupe). For a
per-key fan-out of expensive derives that is a real footgun — an unrelated
row's change recomputes every body.

Track set-valued reads precisely instead, reusing observeIndexEntities'
technique: record the result sequence + a recompute thunk, cheaply gate, then
recompute just that read and compare. An index lookup gates on
`changedComponents ∩ readColumns` then compares its bucket sequence; a presence
`select` gates on `changedArchetypes` (membership is archetype-shaped, so a
value write can never move it). Only a genuinely different result re-runs the
body — a false positive costs one O(bucket) recompute, not a body run.

Also drop the value-dependent `where` / `order` options from the derive's
`select` surface (`Database.Read`): they can only be tracked coarsely, so a
value-keyed or ordered reactive read must go through a declared index. And
invert the entity-dep scan to O(changedEntities) rather than O(watched).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant