@@ -40,6 +40,110 @@ docker run -d --name redis -p 6379:6379 redis:8.0
4040
4141** Note:** Redis 8.0+ is required for INT8/UINT8 vector datatypes. SVS-VAMANA algorithm requires Redis 8.2+ and Intel AVX-512 hardware.
4242
43+
44+ ## How It Works
45+
46+ Every migration follows the same three-phase flow: ** describe what changed** (the patch),
47+ ** generate a plan** (diffing the patch against the live schema), and ** execute the plan** .
48+
49+ ### Single-Index Flow: wizard/plan then apply
50+
51+ ```
52+ wizard (interactive) plan (non-interactive)
53+ | |
54+ v v
55+ SchemaPatch YAML <----or----> SchemaPatch YAML
56+ | |
57+ +------ planner.create_plan() -------+
58+ |
59+ v
60+ MigrationPlan YAML
61+ |
62+ v
63+ executor.apply()
64+ |
65+ v
66+ MigrationReport YAML
67+ ```
68+
69+ ** Phase 1: Build a SchemaPatch.**
70+ A patch is a small YAML file that declares * what you want to change* , not the full target schema.
71+ You can build it interactively with ` rvl migrate wizard ` , or write it by hand. The patch has
72+ five sections, each optional:
73+
74+ | Patch Section | What it does |
75+ | ---| ---|
76+ | ` add_fields ` | Adds new field definitions to the index |
77+ | ` remove_fields ` | Removes fields from the index (document data is kept, just no longer indexed) |
78+ | ` rename_fields ` | Renames fields in both the index schema and all documents (HGET old, HSET new, HDEL old) |
79+ | ` update_fields ` | Modifies field attributes: algorithm, datatype, distance metric, sortable, separator, etc. |
80+ | ` index ` | Changes the index name or key prefix |
81+
82+ ** Phase 2: Generate a MigrationPlan.**
83+ The planner connects to Redis, snapshots the live index schema and stats,
84+ then merges the patch into the source schema to produce a ` merged_target_schema ` .
85+ It classifies every change as supported or blocked and extracts rename operations.
86+
87+ The plan YAML contains:
88+ - ` source ` : frozen snapshot of the live index at planning time (schema, stats, key sample, prefixes)
89+ - ` requested_changes ` : the patch that was applied
90+ - ` merged_target_schema ` : source + patch = what the index will look like after migration
91+ - ` diff_classification ` : whether the migration is supported and any blocked reasons
92+ - ` rename_operations ` : extracted index renames, prefix changes, and field renames
93+ - ` warnings ` : any important notes (downtime required, lossy quantization, etc.)
94+
95+ The same patch produces different plans per index because each index has a different source schema.
96+
97+ ** Phase 3: Apply.**
98+ The executor reads the plan and runs the migration steps:
99+
100+ 1 . Enumerate keys (SCAN with source prefix)
101+ 2 . Field renames (pipelined HGET/HSET/HDEL)
102+ 3 . Dump original vectors to backup file (if quantizing and backup-dir provided)
103+ 4 . Drop index (FT.DROPINDEX, documents are preserved)
104+ 5 . Key prefix renames (RENAME or DUMP/RESTORE for cluster)
105+ 6 . Quantize vectors from backup (pipelined read/convert/write)
106+ 7 . Create index (FT.CREATE with merged target schema)
107+ 8 . Wait for re-indexing to complete
108+ 9 . Validate (doc count, schema match, key sample)
109+
110+ ### Batch Flow: wizard/plan then batch-plan then batch-apply
111+
112+ For applying the same change across multiple indexes:
113+
114+ ```
115+ SchemaPatch YAML (shared, written once)
116+ |
117+ v
118+ batch_planner.create_batch_plan()
119+ for each index:
120+ snapshot live schema
121+ merge patch into source
122+ if applicable: write per-index MigrationPlan
123+ if not: mark skip_reason
124+ |
125+ v
126+ BatchPlan YAML
127+ shared_patch: { ... }
128+ indexes:
129+ - name: idx_a, applicable: true, plan_path: plans/idx_a.yaml
130+ - name: idx_b, applicable: true, plan_path: plans/idx_b.yaml
131+ - name: idx_c, applicable: false, skip_reason: "field not found"
132+ |
133+ v
134+ batch_executor.apply()
135+ for each applicable index (sequentially):
136+ executor.apply(per_index_plan)
137+ ```
138+
139+ The batch planner takes a ** single shared patch** and tests it against every target index.
140+ Indexes where the patch doesn't apply (e.g., it references a field that doesn't exist in that
141+ index, or the change is blocked) are marked ` applicable: false ` with a ` skip_reason ` and skipped
142+ during apply. Each applicable index gets its own full ` MigrationPlan ` written to disk.
143+
144+ This means you can review each per-index plan individually before running ` batch-apply ` .
145+
146+
43147## Step 1: Discover Available Indexes
44148
45149``` bash
0 commit comments