feat(data): db.observe.derive + Database.Read read projection#146
Open
krisnye wants to merge 6 commits into
Open
feat(data): db.observe.derive + Database.Read read projection#146krisnye wants to merge 6 commits into
krisnye wants to merge 6 commits into
Conversation
…nly) Design-review PR — the type surface for a reactive-derivation primitive, no runtime implementation yet. - `Database.Read<DB>`: the read-only projection a derive callback receives. Keeps value reads (`get`/`read`/`select`), resource reads, index LOOKUPS (`find`/`findRange`/`get`), and archetype identity (`components`/`id`). Structurally OMITS observers, transactions/writes, `queryArchetypes`, `locate`, and per-archetype table/column/`insert` access — so touching a non-derivable API is a compile error, not a runtime throw. - `Database.Index.ReadHandle<C, I>`: an index handle minus `observe`. - `db.observe.derive<T>(compute: (db: Database.Read<…>) => T, options?) : Observe<T>` — signature only; the intent is that a derive records the reads `compute` performs and re-emits when any could have changed. Compile-time tests (`database-read.type-test.ts`) pin the positive surface and, via `@ts-expect-error`, that `observe` / `transactions` / `queryArchetypes` / `locate` / `columns` / `rowCount` / `insert` / `indexes.*.observe` are all rejected. typecheck + lint clean. No implementation and no version bump — for reviewing the types; not to merge. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implements the spec from the prior commit. read overloads (core.ts): - read(entity, archetype) narrowed to `Readonly<T>` (was `& EntityReadValues<C>`); `minArchetype` renamed to `archetype`. The narrowing's blast radius across adobe/data was zero. - new read(entity, components[]) projection — present-or-undefined, `null` only when the entity is absent — enabling component-scoped derive deps. derive engine (observe-derive.ts): - ONE read-recording wrapper per database, shared across all derives; its recorded-dep set is reset per synchronous compute run. Safe to share because compute is synchronous and the read surface exposes no writes / observers / nested derives, so runs cannot overlap or recurse. - deps reduce to entity deps (gated on changedEntities + changed-component intersection) and column deps (select include/exclude/where/order, index readColumns, resource names) checked against changedComponents. One transaction subscription per derive; recompute coalesced on a microtask; emit gated by structural `equals`. Tests: observe-derive.test.ts — initial emit, component-scoped re-emit vs whole-entity coarseness, structural dedupe, select / index / resource deps, coalescing, disposal. Type tests updated for the projection read. typecheck + lint + 911 ecs/observe tests green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…sced `observeTransactions` fires once per committed transaction, so a derive already recomputes at most once per commit — deferring the emit to a microtask only coalesced *across* commits in one turn, which starves consumers that route per commit (e.g. a burst of ephemeral drag commits, where each must produce its own downstream effect). Emit synchronously at the commit boundary instead — the same cadence as `observe.entity` / the raw component observers a hand-written computed would use. Structural dedupe still suppresses unchanged results. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`read(entity, components[])` built the full row via `getRowData` and then discarded the columns it didn't need. Read only the requested components' columns directly (`archetype.columns[c].get(row)`), skipping absent ones — no whole-row allocation or reads of columns the caller didn't ask for. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ructural Redefine `Database.Read<DB>` by mapping over `DB`'s own members instead of `DB extends Database<infer …>`. The `infer` form collapsed an intersection `A & B` to a single matched member; the structural form distributes, so `Read<A & B>` merges both read surfaces. Move `derive` off `db.observe.derive` to a top-level `db.derive` whose callback receives `Database.Read<this>`. On an intersection receiver `this` resolves to `A & B`, so a derive over a composed database sees indexes + resources + archetypes from both members — consumers of an `A & B` database no longer cast. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ip-level select The derive read-recorder previously reduced index lookups and `select`s to a coarse global column set: any write to a backing column, anywhere, re-ran the whole compute body (suppressed only downstream by structural dedupe). For a per-key fan-out of expensive derives that is a real footgun — an unrelated row's change recomputes every body. Track set-valued reads precisely instead, reusing observeIndexEntities' technique: record the result sequence + a recompute thunk, cheaply gate, then recompute just that read and compare. An index lookup gates on `changedComponents ∩ readColumns` then compares its bucket sequence; a presence `select` gates on `changedArchetypes` (membership is archetype-shaped, so a value write can never move it). Only a genuinely different result re-runs the body — a false positive costs one O(bucket) recompute, not a body run. Also drop the value-dependent `where` / `order` options from the derive's `select` surface (`Database.Read`): they can only be tracked coarsely, so a value-keyed or ordered reactive read must go through a declared index. And invert the entity-dep scan to O(changedEntities) rather than O(watched). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Auto-tracking reactive derivation for the ECS.
db.observe.derive(compute)runscomputeagainst a read-only projection of the database, records exactly the reads it performs, and re-emits only when a recorded dependency changes — so the whole class of "read a field but forgot to subscribe to it" staleness bugs (e.g. theframeRategap found in the FFPavTimelinePayloadrefactor) becomes impossible by construction.Still a review PR — no version bump; not necessarily merging as-is.
Database.Read<DB>The read-only projection the
computecallback receives. Keepsget/read/select,resources, index lookups (find/findRange/get), and archetype identity (components/id). Structurally omitsobserve(a derive subscribes for you), transactions/writes,queryArchetypes,locate, and per-archetypecolumns/rowCount/insert. Misuse is a compile error, not a runtime throw. Supporting typeDatabase.Index.ReadHandle<C,I>= a handle minusobserve.readoverloadsread(entity, archetype)narrowed toReadonly<T>(wasReadonly<T> & EntityReadValues<C>);minArchetype→archetype. Blast radius across adobe/data: zero — no caller was leaning on the whole-entity union.read(entity, components[])— a projection (present-or-undefined;nullonly when the entity is absent). Names the exact fields touched so a derive can scope its dependencies to them.derive engine (
observe-derive.ts)computerun. Safe to share becausecomputeis synchronous and the read surface exposes no writes/observers/nested-derives, so runs can't overlap or recurse. (NoProxy— the read surface is a finite, explicit wrapper.)get/read(entity, archetype|components[])→ recompute when that entity is inchangedEntitiesand (for scoped reads) a watched component changed on it, or it was deleted;selectinclude/exclude/where/order, an index'sreadColumns, and resource names → recompute when the transaction'schangedComponentsintersects them.equals(no re-emit on an equal result).read(e)= whole-entity dep,read(e, ["a"])/get(e,"a")= just those fields.Tests
observe-derive.test.ts: initial synchronous emit; re-emit on a read component; no re-emit on an unread component (projection scoping) vs whole-entity coarseness; structural dedupe;selectmembership, index-bucket, and resource deps; coalescing; disposal.database-read.type-test.ts: positive read surface +@ts-expect-errorfor every omitted API (an unused expect-error failstsc, so these prove each footgun is blocked).typecheck+lint+ 911 ecs/observe tests green.Notes / follow-ups
changedEntities, so it triggers a recompute that structural dedupe then suppresses (correct, mildly wasteful).extendaren't in the shared recorder (built lazily on first use); fine for the create-time plugin case, a follow-up for dynamic extends.