feat: Support transforming primitive types using a codec (#73)

* feat(core): add bidirectional transforms for domain types

Enable conversion between CRDT primitives (strings, numbers, booleans)
and application domain types (Date, Temporal, BigInt, custom objects).

- Add TransformDefinition interface with decode/encode functions
- Add EqualityStrategy for configurable diff behavior
- Integrate transforms into diff, event application, and Mirror
- Add comprehensive test suite (9 test files, ~2,651 lines)

* fix(core): tighten transform equality checks

* fix(core): harden transform schema helpers

* fix(core): decode ephemeral transform values

* refactor(api): remove throwOnValidationError option

* fix(types): reflect missing transformed startup values

* refactor(schema): centralize child schema resolution

* fix(core): restore schema.String overload compatibility

---------

Co-authored-by: Zixuan Chen <remch183@outlook.com>

Author: bentefay

AGENTS.md

@@ -42,7 +42,7 @@
 ## Public API (loro-mirror)
 
 - `Mirror(options: MirrorOptions<S>)`
-    - `doc` (required), `schema?`, `initialState?`, `validateUpdates?`, `throwOnValidationError?`, `debug?`, `checkStateConsistency?`, `inferOptions?`.
+    - `doc` (required), `schema?`, `initialState?`, `validateUpdates?`, `debug?`, `checkStateConsistency?`, `inferOptions?`.
     - Methods: `getState()`, `setState(updater, options?)`, `subscribe(cb)`, `dispose()`, `checkStateConsistency()`, `getContainerIds()`.
     - `SetStateOptions` supports `{ tags?: string | string[] }`; subscriber metadata includes `{ source: UpdateSource; tags?: string[] }`.
 - `schema(definition, options?)` plus builders: `.String()`, `.Number()`, `.Boolean()`, `.Ignore()`, `.LoroMap()`, `.LoroMapRecord()`, `.LoroList()`, `.LoroMovableList()`, `.LoroText()`, `.LoroTree()`.

README.md

@@ -10,6 +10,7 @@ A TypeScript state management library that syncs application state with [loro-cr
 - 🔍 **Selective Updates**: Subscribe to specific parts of your state
 - 🛠️ **Developer Friendly**: Familiar API inspired by popular state management libraries
 - 📱 **React Integration**: Hooks and context providers for React applications
+- 🔀 **Transforms**: Work with rich domain types (Date, BigInt, custom objects) while Loro stores primitives
 
 ## Packages
 
@@ -113,6 +114,49 @@ Loro Mirror provides a declarative schema system that enables:
     - `schema.Any(options?)` - Runtime-inferred value/container type for JSON-like dynamic fields
     - `schema.Ignore(options?)` - Field that won't sync with Loro, useful for local computed fields
 
+- **Transforms** (on primitive types):
+
+    Primitive schemas support `.transform()` to convert between CRDT primitives and rich domain types:
+
+    ```typescript
+    const dateTransform = {
+        decode: (s: string) => new Date(s),
+        encode: (d: Date) => d.toISOString(),
+    };
+
+    const taskSchema = schema({
+        task: schema.LoroMap({
+            title: schema.String(),
+            dueDate: schema.String().transform(dateTransform), // Type infers as Date
+        }),
+    });
+
+    // Now you can use Date objects directly
+    mirror.setState({ task: { title: "Review PR", dueDate: new Date() } });
+    mirror.getState().task.dueDate.getTime(); // Already a Date
+    ```
+
+    The transform definition has this shape:
+
+    ```typescript
+    interface TransformDefinition<CRDTType, DomainType> {
+        decode: (value: CRDTType & {}) => DomainType & {};
+        encode: (value: DomainType & {}) => CRDTType & {};
+        validate?: (value: DomainType & {}) => boolean | string;
+        isEqual?:
+            | "reference-equality"
+            | "encoded-value-equality"
+            | "deep-equality"
+            | ((a: DomainType, b: DomainType) => boolean);
+    }
+    ```
+
+    **Note:** The encode and decode functions will never receive `null` or `undefined` values and must not return them. Nullish values are handled before transforms are applied.
+
+    **Important:** `DomainType` should be immutable or [supported by Immer](https://immerjs.github.io/immer/complex-objects/), otherwise changes to transformed values may not be detected and converted to CRDT operations. Never mutate `DomainType` instances outside of Loro Mirror's `setState` function.
+
+    For optional fields, just use the transform directly - the type automatically includes `| undefined`:
+
 `schema.Any` options:
 
 - `defaultLoroText?: boolean` (defaults to `false` for `Any` when omitted)
@@ -334,9 +378,9 @@ For detailed documentation, see the README files in each package:
 
 Loro Mirror uses a typed schema to map your app state to Loro containers. Common schema constructors:
 
-- `schema.String(options?)`: string
-- `schema.Number(options?)`: number
-- `schema.Boolean(options?)`: boolean
+- `schema.String(options?)`: string (supports `.transform()` for domain types)
+- `schema.Number(options?)`: number (supports `.transform()` for domain types)
+- `schema.Boolean(options?)`: boolean (supports `.transform()` for domain types)
 - `schema.Any(options?)`: JSON-like unknown (runtime-inferred)
 - `schema.Ignore(options?)`: exclude from sync (app-only)
 - `schema.LoroText(options?)`: rich text (`LoroText`)
@@ -363,7 +407,6 @@ const mySchema = schema({ outline: schema.LoroTree(node) });
     - **`schema`**: root schema – optional but recommended for strong typing and validation.
     - **`initialState`**: partial state – merged with schema defaults and current doc JSON.
     - **`validateUpdates`**: boolean (default `true`) – validate new state against schema.
-    - **`throwOnValidationError`**: boolean (default `false`) – throw on invalid updates.
     - **`debug`**: boolean (default `false`) – log diffs and applied changes.
     - **`checkStateConsistency`**: boolean (default `false`) – after non-ephemeral `setState` calls, assert the doc-backed base state equals the normalized `LoroDoc` snapshot.
     - **`inferOptions`**: `{ defaultLoroText?: boolean; defaultMovableList?: boolean }` – influence container-type inference when inserting containers from plain values.

packages/core/README.md

@@ -70,12 +70,11 @@ Trees are advanced usage; see Advanced: Trees at the end.
 
 ### Mirror
 
-- Constructor: `new Mirror({ doc, schema?, initialState?, validateUpdates?=true, throwOnValidationError?=false, debug?=false, checkStateConsistency?=false, inferOptions? })`
+- Constructor: `new Mirror({ doc, schema?, initialState?, validateUpdates?=true, debug?=false, checkStateConsistency?=false, inferOptions? })`
     - doc: LoroDoc to sync with
     - schema: Root schema; enables validation and typed defaults
     - initialState: Shallow-merged over schema defaults and current doc JSON
     - validateUpdates: Validate on `setState`
-    - throwOnValidationError: Throw if validation fails (default false)
     - debug: Verbose logging
     - checkStateConsistency: Extra runtime check that the doc-backed base state still matches `toNormalizedJson(doc)` after non-ephemeral `setState` updates
     - inferOptions: `{ defaultLoroText?: boolean; defaultMovableList?: boolean }` for container inference when schema is missing

packages/core/src/core/diff.ts

@@ -22,6 +22,7 @@ import {
     SchemaType,
     type InferContainerOptions,
 } from "../schema/index.js";
+import { getMapFieldSchema } from "../schema/resolver.js";
 import { ChangeKinds, type Change } from "./mirror.js";
 import { CID_KEY } from "../constants.js";
 
@@ -41,8 +42,10 @@ import {
     isArrayLike,
     isTreeID,
     defineCidProperty,
+    valuesEqual,
+    applyEncode,
+    hasTransform,
 } from "./utils.js";
-import { getMapFieldSchema } from "../schema/resolver.js";
 
 /**
  * Finds the longest increasing subsequence of a sequence of numbers
@@ -628,6 +631,7 @@ export function diffMovableList<S extends ArrayLike>(
     }
 
     // 4) Insertions (items only in new state)
+    const itemSchema = schema?.itemSchema;
     for (const [newIndex, item] of newState.entries()) {
         const id = idSelector(item);
         if (!id || !oldMap.has(id)) {
@@ -640,7 +644,7 @@ export function diffMovableList<S extends ArrayLike>(
                         kind: "insert",
                     },
                     true,
-                    schema?.itemSchema,
+                    itemSchema,
                     inferOptions,
                 ),
             );
@@ -649,7 +653,7 @@ export function diffMovableList<S extends ArrayLike>(
 
     // 5) Updates (for items present in both states)
     for (const info of common) {
-        if (deepEqual(info.oldItem, info.newItem)) continue;
+        if (valuesEqual(itemSchema, info.oldItem, info.newItem, "deep-equality")) continue;
 
         const movableList = doc.getMovableList(containerId);
         const currentItem = movableList.get(info.oldIndex);
@@ -659,7 +663,7 @@ export function diffMovableList<S extends ArrayLike>(
                 info.oldItem,
                 info.newItem,
                 currentItem.id,
-                schema?.itemSchema,
+                itemSchema,
                 inferOptions,
             );
             changes.push(...nested);
@@ -673,7 +677,7 @@ export function diffMovableList<S extends ArrayLike>(
                         kind: "set",
                     },
                     true,
-                    schema?.itemSchema,
+                    itemSchema,
                     inferOptions,
                 ),
             );
@@ -761,6 +765,7 @@ export function diffListWithIdSelector<S extends ArrayLike>(
     }
 
     // 2) Insertions (items that are new or moved).
+    const itemSchema = schema?.itemSchema;
     for (const [newIndex, item] of newState.entries()) {
         const id = idSelector(item);
         if (!id || !keepIdSet.has(id)) {
@@ -773,7 +778,7 @@ export function diffListWithIdSelector<S extends ArrayLike>(
                         kind: "insert",
                     },
                     useContainer,
-                    schema?.itemSchema,
+                    itemSchema,
                     inferOptions,
                 ),
             );
@@ -790,7 +795,7 @@ export function diffListWithIdSelector<S extends ArrayLike>(
 
         const oldItem = oldInfo.item;
         const newItem = newInfo.item;
-        if (deepEqual(oldItem, newItem)) continue;
+        if (valuesEqual(itemSchema, oldItem, newItem, "deep-equality")) continue;
 
         const itemOnLoro = list.get(oldInfo.index);
         if (isContainer(itemOnLoro)) {
@@ -800,7 +805,7 @@ export function diffListWithIdSelector<S extends ArrayLike>(
                     oldItem,
                     newItem,
                     itemOnLoro.id,
-                    schema?.itemSchema,
+                    itemSchema,
                     inferOptions,
                 ),
             );
@@ -820,7 +825,7 @@ export function diffListWithIdSelector<S extends ArrayLike>(
                         kind: "insert",
                     },
                     useContainer,
-                    schema?.itemSchema,
+                    itemSchema,
                     inferOptions,
                 ),
             );
@@ -861,13 +866,14 @@ export function diffList<S extends ArrayLike>(
     const oldLen = oldState.length;
     const newLen = newState.length;
     const list = doc.getList(containerId);
+    const itemSchema = schema?.itemSchema;
 
     // Find common prefix
     let start = 0;
     while (
         start < oldLen &&
         start < newLen &&
-        oldState[start] === newState[start]
+        valuesEqual(itemSchema, oldState[start], newState[start], "reference-equality")
     ) {
         start++;
     }
@@ -877,7 +883,7 @@ export function diffList<S extends ArrayLike>(
     while (
         suffix < oldLen - start &&
         suffix < newLen - start &&
-        oldState[oldLen - 1 - suffix] === newState[newLen - 1 - suffix]
+        valuesEqual(itemSchema, oldState[oldLen - 1 - suffix], newState[newLen - 1 - suffix], "reference-equality")
     ) {
         suffix++;
     }
@@ -889,7 +895,7 @@ export function diffList<S extends ArrayLike>(
     const overlap = Math.min(oldBlockLen, newBlockLen);
     for (let j = 0; j < overlap; j++) {
         const i = start + j;
-        if (oldState[i] === newState[i]) continue;
+        if (valuesEqual(itemSchema, oldState[i], newState[i], "reference-equality")) continue;
 
         const itemOnLoro = list.get(i);
         if (isContainer(itemOnLoro)) {
@@ -898,7 +904,7 @@ export function diffList<S extends ArrayLike>(
                 oldState[i],
                 newState[i],
                 itemOnLoro.id,
-                schema?.itemSchema,
+                itemSchema,
                 inferOptions,
             );
             changes.push(...nestedChanges);
@@ -918,7 +924,7 @@ export function diffList<S extends ArrayLike>(
                         kind: "insert",
                     },
                     true,
-                    schema?.itemSchema,
+                    itemSchema,
                     inferOptions,
                 ),
             );
@@ -948,7 +954,7 @@ export function diffList<S extends ArrayLike>(
                     kind: "insert",
                 },
                 true,
-                schema?.itemSchema,
+                itemSchema,
                 inferOptions,
             ),
         );
@@ -1115,13 +1121,14 @@ export function diffMovableListByIndex(
     const oldLen = oldState.length;
     const newLen = newState.length;
     const list = doc.getMovableList(containerId);
+    const itemSchema = schema?.itemSchema;
 
     // Find common prefix
     let start = 0;
     while (
         start < oldLen &&
         start < newLen &&
-        oldState[start] === newState[start]
+        valuesEqual(itemSchema, oldState[start], newState[start], "reference-equality")
     ) {
         start++;
     }
@@ -1131,7 +1138,7 @@ export function diffMovableListByIndex(
     while (
         suffix < oldLen - start &&
         suffix < newLen - start &&
-        oldState[oldLen - 1 - suffix] === newState[newLen - 1 - suffix]
+        valuesEqual(itemSchema, oldState[oldLen - 1 - suffix], newState[newLen - 1 - suffix], "reference-equality")
     ) {
         suffix++;
     }
@@ -1143,7 +1150,7 @@ export function diffMovableListByIndex(
     const overlap = Math.min(oldBlockLen, newBlockLen);
     for (let j = 0; j < overlap; j++) {
         const i = start + j;
-        if (oldState[i] === newState[i]) continue;
+        if (valuesEqual(itemSchema, oldState[i], newState[i], "reference-equality")) continue;
 
         const itemOnLoro = list.get(i);
         const next = newState[i];
@@ -1157,7 +1164,7 @@ export function diffMovableListByIndex(
                         oldState[i],
                         next,
                         itemOnLoro.id,
-                        schema?.itemSchema,
+                        itemSchema,
                         inferOptions,
                     ),
                 );
@@ -1174,7 +1181,7 @@ export function diffMovableListByIndex(
                     kind: "set",
                 },
                 true,
-                schema?.itemSchema,
+                itemSchema,
                 inferOptions,
             ),
         );
@@ -1203,7 +1210,7 @@ export function diffMovableListByIndex(
                     kind: "insert",
                 },
                 true,
-                schema?.itemSchema,
+                itemSchema,
                 inferOptions,
             ),
         );
@@ -1301,8 +1308,10 @@ export function diffMap<S extends ObjectLike>(
             inferOptions,
         );
         let containerType =
-            childSchema?.getContainerType() ??
-            tryInferContainerType(newItem, childInferOptions);
+            hasTransform(childSchema)
+                ? undefined
+                : (childSchema?.getContainerType() ??
+                  tryInferContainerType(newItem, childInferOptions));
         if (
             childSchema?.getContainerType() &&
             containerType &&
@@ -1336,7 +1345,7 @@ export function diffMap<S extends ObjectLike>(
                 changes.push({
                     container: containerId,
                     key,
-                    value: newItem,
+                    value: applyEncode(childSchema, newItem),
                     kind: "insert",
                 });
             }
@@ -1421,15 +1430,19 @@ export function diffMap<S extends ObjectLike>(
                         ),
                     );
                 }
-                // The type of the child has changed
-                // Either it was previously a container and now it's not
-                // or it was not a container and now it is
             } else {
+                if (valuesEqual(childSchema, oldItem, newItem, "reference-equality")) {
+                    continue; 
+                }
+
+                // The type or value of the child has changed
+                // Either the value has changed, or it was previously a container and now it's not,
+                // or it was not a container and now it is
                 changes.push(
                     insertChildToMap(
                         containerId,
                         key,
-                        newStateObj[key],
+                        applyEncode(childSchema, newItem),
                         applySchemaToInferOptions(childSchema, inferOptions),
                     ),
                 );