|
1 | 1 | ### Q3 2026 objectives |
2 | 2 |
|
3 | | -#### Objective 1: Ship the first version of the replay vision / self-driving model |
| 3 | +#### Goal: Ship the first version of the replay vision / self-driving model |
4 | 4 |
|
5 | | -**What we'll ship:** |
6 | | -- **Data labeling suite** - tooling to label replay sessions so we have high-quality supervision for training and evaluation. |
7 | | -- **Session replay text renderer** - a programmatic renderer that extracts the objective facts already present in a recording ("user clicked button X, then Y happened") into text, used to ground the model's decoder. There's precedent here in screen-reader work. |
8 | | -- **DOMEncoder** - encoder for DOM state. |
9 | | -- **RRWebEncoder** - encoder for rrweb replay events. |
10 | | -- **Data prep** - pipelines to turn raw replays into training-ready inputs for the encoders and renderer. |
11 | | -- **Sampling pipeline** - select and sample which sessions feed training, so the model learns from a representative, high-signal slice of replays. |
12 | | -- **Full model training** - train the QFormer first (it starts from random weights) to compress per-event replay vectors into a compact representation, and train the language model last to avoid catastrophic forgetting. |
13 | | -- **Model observability suite** - tooling (e.g. MLflow) to visualize neurons and debug problems during training. |
14 | | -- **Eval dataset** - a held-out dataset to measure whether the model actually understands replays. |
| 5 | +**Components:** |
| 6 | +- **Data labeling suite** |
| 7 | +- **Session replay text renderer** |
| 8 | +- **Write data prep pipeline** |
| 9 | +- **Build the sampling pipeline** |
| 10 | +- **Train the Replay encoder model** |
| 11 | +- **Train the end-to-end agent** |
| 12 | +- **Build the model observability suite** |
| 13 | +- **Build an eval dataset** |
15 | 14 |
|
16 | | -**What we're not shipping:** |
| 15 | +**Non-goals:** |
17 | 16 | - **Hand-rolled inference** |
0 commit comments