Skip to content

Commit d325f2d

Browse files
Davids048SolitaryThinkerRandNMR73XOR-opjzhang38
committed
[feat] Activate Dreamverse LTX2 integration
Apply Dreamverse monorepo changes for stack slice 13/13 from the source branch. Source-Branch: will/dreamverse-monorepo Source-SHA: 03d3e61 Dreamverse-Stack: 13/13 Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com> Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com>
1 parent 3747994 commit d325f2d

4 files changed

Lines changed: 125 additions & 10 deletions

File tree

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
# Dreamverse Development
2+
3+
Dreamverse lives under `apps/dreamverse/` as a product app inside the
4+
FastVideo monorepo. Backend code uses the local FastVideo workspace package;
5+
frontend tooling remains standalone under `apps/dreamverse/web/`.
6+
7+
## Backend tests
8+
9+
Run CPU-safe backend tests from the FastVideo repository root:
10+
11+
```bash
12+
uv run --locked --package dreamverse --extra test pytest apps/dreamverse/server/tests/ -m 'not gpu' -q
13+
```
14+
15+
## Backend launch
16+
17+
Launch the migrated backend through the installed console commands:
18+
19+
```bash
20+
dreamverse-server --port 8009
21+
dreamverse-mock-server --port 8009
22+
```
23+
24+
If `dreamverse-server` is missing, install FastVideo with the `dreamverse`
25+
extra from the checkout:
26+
27+
```bash
28+
uv pip install -e ".[dreamverse]"
29+
```
30+
31+
## Frontend build and tests
32+
33+
Run frontend commands from the standalone web app:
34+
35+
```bash
36+
cd apps/dreamverse/web
37+
pnpm install --frozen-lockfile
38+
pnpm run build
39+
pnpm run test
40+
```
41+
42+
Playwright is intentionally run against a live backend as part of the GPU4
43+
manual verification flow, not in the Phase 3 migration gate.
44+
45+
## Local GPU4 verification hook
46+
47+
Use physical GPU 4 for migration smoke tests. `CUDA_VISIBLE_DEVICES=4` makes
48+
that GPU appear as logical GPU 0 inside the process, preserving the previous
49+
Dreamverse deployment behavior.
50+
51+
```bash
52+
CUDA_VISIBLE_DEVICES=4 dreamverse-server --host 0.0.0.0 --port 8009
53+
```
54+
55+
In another shell, verify the service:
56+
57+
```bash
58+
curl -s http://localhost:8009/healthz
59+
```
60+
61+
Phase 4 adds the public `/healthz`, `/readyz`, `/status`,
62+
`/prompt-system-config`, and `/curated-presets` route coverage needed for the
63+
full Playwright suite.
64+
65+
## Phase 0 production-equivalent prerequisites
66+
67+
For the production-equivalent NVFP4 path, install these dependencies
68+
in the FastVideo `.venv` before GPU smoke tests:
69+
70+
```bash
71+
uv pip install --python .venv/bin/python \
72+
flashinfer-python flash-attn cerebras-cloud-sdk openai \
73+
--no-build-isolation
74+
```
75+
76+
| Package | Why |
77+
|---|---|
78+
| `flashinfer-python` | Required for NVFP4 quantization. Without it, model load fails with `ImportError: NVFP4 quantization requires flashinfer`. |
79+
| `flash-attn` | Optional but recommended; without it attention falls back to Torch SDPA (functional but slower). |
80+
| `cerebras-cloud-sdk` | Required by the migrated prompt enhancer for the default `cerebras` provider. |
81+
| `openai` | Required by the prompt enhancer's OpenAI-compatible providers + downstream rewrites. |
82+
83+
### B200 / sm_100a + gcc-15 conda toolchain (flashinfer JIT workaround)
84+
85+
On hosts where the conda toolchain ships gcc-15 (which nvcc rejects with
86+
`#error -- unsupported GNU version! gcc versions later than 14 are not
87+
supported!`), set these env vars before launching anything that triggers
88+
flashinfer's JIT kernel build:
89+
90+
```bash
91+
export CC=/usr/bin/gcc-13
92+
export CXX=/usr/bin/g++-13
93+
export CUDAHOSTCXX=/usr/bin/g++-13
94+
export NVCC_PREPEND_FLAGS="-ccbin /usr/bin/gcc-13 -allow-unsupported-compiler"
95+
```
96+
97+
`dreamverse-server` does NOT set these — they need to come from the launching
98+
shell. The `dreamverse-deploy` skill
99+
([`.agents/skills/dreamverse-deploy/`](../../.agents/skills/dreamverse-deploy/SKILL.md))
100+
sets them for you and is the recommended local-deploy path.

fastvideo/models/registry.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -115,6 +115,8 @@
115115
_UPSAMPLERS = {
116116
"SRTo720pUpsampler": ("upsamplers", "hunyuan15", "SRTo720pUpsampler"),
117117
"SRTo1080pUpsampler": ("upsamplers", "hunyuan15", "SRTo1080pUpsampler"),
118+
"LTX2LatentUpsampler":
119+
("upsamplers", "ltx2_upsampler", "LTX2LatentUpsampler"),
118120
}
119121

120122
_LEGACY_FAST_VIDEO_MODELS = {

fastvideo/registry.py

Lines changed: 17 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -212,35 +212,42 @@ def _get_config_info(
212212

213213

214214
def _register_configs() -> None:
215-
# LTX-2 (base)
215+
# LTX-2 (distilled) — registered FIRST so its detector wins over
216+
# the base detector when both fire. The detector loop in
217+
# ``get_model_name_for_path`` ORs the path-based check with a
218+
# pipeline-name check (``ltx2pipeline``) which the base detector's
219+
# "distilled not in path" predicate matches as True (the
220+
# pipeline_name string contains no "distilled" marker), so the
221+
# less-specific BASE detector would otherwise win when the
222+
# input is the absolute path of the distilled checkpoint.
216223
register_configs(
217224
sampling_param_cls=None,
218225
pipeline_config_cls=LTX2T2VConfig,
219226
workload_types=(WorkloadType.T2V, ),
220227
hf_model_paths=[
221-
"Lightricks/LTX-2",
222-
"FastVideo/LTX2-base",
223-
"FastVideo/LTX2-Diffusers",
228+
"FastVideo/LTX2-Distilled-Diffusers",
224229
],
225230
model_detectors=[
226-
lambda path: ("ltx2" in path.lower() or "ltx-2" in path.lower()) and "distilled" not in path.lower(),
231+
lambda path: ("ltx2" in path.lower() or "ltx-2" in path.lower()) and "distilled" in path.lower(),
227232
],
228233
model_family="ltx2",
229-
default_preset="ltx2_base",
234+
default_preset="ltx2_distilled",
230235
)
231-
# LTX-2 (distilled)
236+
# LTX-2 (base)
232237
register_configs(
233238
sampling_param_cls=None,
234239
pipeline_config_cls=LTX2T2VConfig,
235240
workload_types=(WorkloadType.T2V, ),
236241
hf_model_paths=[
237-
"FastVideo/LTX2-Distilled-Diffusers",
242+
"Lightricks/LTX-2",
243+
"FastVideo/LTX2-base",
244+
"FastVideo/LTX2-Diffusers",
238245
],
239246
model_detectors=[
240-
lambda path: ("ltx2" in path.lower() or "ltx-2" in path.lower()) and "distilled" in path.lower(),
247+
lambda path: ("ltx2" in path.lower() or "ltx-2" in path.lower()) and "distilled" not in path.lower(),
241248
],
242249
model_family="ltx2",
243-
default_preset="ltx2_distilled",
250+
default_preset="ltx2_base",
244251
)
245252

246253
# Stable Audio Open (text-to-audio). Both variants must be loaded

mkdocs.yml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -161,6 +161,11 @@ nav:
161161
- Design:
162162
- Overview: design/overview.md
163163
- Training Architecture: design/training_architecture.md
164+
- Server Contracts:
165+
- Overview: design/server_contracts/index.md
166+
- OpenAI HTTP: design/server_contracts/openai.md
167+
- Streaming WebSocket: design/server_contracts/streaming.md
168+
- Dynamo Integration: design/server_contracts/dynamo.md
164169
- Developer Guide:
165170
- Overview: contributing/overview.md
166171
- Developer Environment:
@@ -172,6 +177,7 @@ nav:
172177
- Testing: contributing/testing.md
173178
- Profiling: contributing/profiling.md
174179
- Coding Agents: contributing/coding_agents.md
180+
- Dreamverse Development: contributing/dreamverse-development.md
175181
- Attention Backend Development: contributing/attention_backend.md
176182
- API Reference:
177183
- FastVideo: api/fastvideo.md

0 commit comments

Comments
 (0)