[feat] Dreamverse 13/14: Activate LTX2 integration#1336
Merged
Conversation
mergify
Bot
added
type: feat
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces documentation for Dreamverse development and server contracts, registers the LTX2LatentUpsampler in the model registry, and reorders LTX-2 configuration registration to ensure correct model detection. Feedback was provided to format the new registry entry on a single line with a trailing comma for PEP 8 compliance and better maintainability.
Comment on lines
+118
to
+119
| "LTX2LatentUpsampler": | ||
| ("upsamplers", "ltx2_upsampler", "LTX2LatentUpsampler"), |
Contributor
There was a problem hiding this comment.
For consistency with other entries in this file and for better readability, this dictionary entry could be on a single line since it does not exceed the common line length limit. Additionally, adding a trailing comma is recommended by the PEP 8 style guide for better maintainability, as it simplifies future additions and leads to cleaner diffs.
"LTX2LatentUpsampler": ("upsamplers", "ltx2_upsampler", "LTX2LatentUpsampler"),References
- PEP 8 recommends adding a trailing comma to the last item of a multi-line list, tuple, or dictionary. This improves maintainability by making it easier to add new items and producing cleaner version control diffs. (link)
Davids048
force-pushed
the
feat/dreamverse-stack-13-activation
branch
from
d325f2d to
e857139
Compare
Davids048
force-pushed
the
feat/dreamverse-stack-12-ltx2-refine-upsampler
branch
from
3747994 to
ad96bbb
Compare
Contributor
Davids048
force-pushed
the
feat/dreamverse-stack-13-activation
branch
from
e857139 to
fb0e5d0
Compare
Davids048
force-pushed
the
feat/dreamverse-stack-12-ltx2-refine-upsampler
branch
from
ad96bbb to
5514037
Compare
Contributor
Davids048
force-pushed
the
feat/dreamverse-stack-13-activation
branch
from
fb0e5d0 to
cbb9e85
Compare
Contributor
Davids048
force-pushed
the
feat/dreamverse-stack-12-ltx2-refine-upsampler
branch
5 times, most recently
from
ccc0a50 to
6461b72
Compare
Davids048
force-pushed
the
feat/dreamverse-stack-13-activation
branch
from
cbb9e85 to
a74b018
Compare
Collaborator
Author
|
/test ssim |
Collaborator
Author
|
/merge |
Base automatically changed from
feat/dreamverse-stack-12-ltx2-refine-upsampler
to
main
May 15, 2026 19:21
mergify
Bot
added
scope: inference
Contributor
Merge ProtectionsYour pull request matches the following merge protections and will not be merged until they are valid. 🔴 PR merge requirementsWaiting for
This rule is failing.
|
Collaborator
Author
|
/merge |
github-actions
Bot
added
ready
Apply Dreamverse monorepo changes for stack slice 13/13 from the source branch. Source-Branch: will/dreamverse-monorepo Source-SHA: 03d3e61 Dreamverse-Stack: 13/13 Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com> Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com>
Davids048
force-pushed
the
feat/dreamverse-stack-13-activation
branch
from
a74b018 to
6c75924
Compare
SolitaryThinker
approved these changes
May 15, 2026
SolitaryThinker
added a commit
to SolitaryThinker/FastVideo
that referenced
this pull request
May 24, 2026
Slice 6 of 12 in the PR hao-ai-lab#1225 decomposition. Tier 2 — backward- compat additions, gated paths only. No activation in this slice. What this adds -------------- * fastvideo/layers/linear.py (+54): adds opt-in shape-tracking instrumentation to ``ReplicatedLinear`` so upcoming QAT-aware backends can discover which GEMM shapes need quantized kernels. Gated by the class attr ``enable_shape_tracking = False``; the default forward path is bit-identical to pre-slice behavior. Adds ``get_shape_mapping``, ``reset_shape_tracking``, ``_track_shape``, ``print_shape_summary``. No new constructor params. * fastvideo/layers/mlp.py (+22): adds an optional ``quant_config: QuantizationConfig | None = None`` kwarg to ``MLP.__init__`` and threads it (plus an explicit ``prefix``) into the two underlying ``ReplicatedLinear`` instances. When ``quant_config is not None``, runs ``process_weights_after_loading`` on each sub-layer's resolved quant method. When ``quant_config is None`` (default), behavior is unchanged: ``ReplicatedLinear`` falls back to ``UnquantizedLinearMethod`` exactly as before. * fastvideo/models/dits/wanvideo.py (+67): wires ``quant_config`` through ``WanSelfAttention``, ``WanI2VCrossAttention``, ``WanTransformerBlock``, ``WanTransformerBlock_VSA``, and ``WanTransformer3DModel`` constructors so a future ``NVFP4QAT``- configured Wan2.1 build can quantize its attention QKV/out projections and FFN. Reads ``config.quant_config`` from ``WanVideoConfig`` (the field is already present on the shared ``DiTBaseConfig``). All new kwargs default to ``None``; default Wan2.1 path stays bit-identical. Files in PR hao-ai-lab#1225 considered but NOT applied -------------------------------------------- The source-SHA ``fastvideo/layers/linear.py`` also contains several edits that pre-date current ``main`` and would silently regress it: * Removal of the ``NVFP4Config``-only-quantizes-a-curated-subset explanatory comments in ``LinearBase.__init__`` and ``ReplicatedLinear.__init__`` (added on main as part of slice 3 / PR hao-ai-lab#1336). * Removal of the ``if self.quant_method is None: self.quant_method = UnquantizedLinearMethod()`` fallback inside ``LinearBase.__init__`` (also part of the slice 3 hardening). * A constructor / ``create_weights`` reformat from multi-line to compact one-line style — pure style noise. * ``assert self.quant_method is not None`` → ``if self.quant_method is None: self.quant_method = UnquantizedLinearMethod()`` in ``ColumnParallelLinear.__init__/forward`` and ``RowParallelLinear.__init__/forward``. ``LinearBase.__init__`` on current ``main`` already guarantees ``quant_method`` is non-None, so the source PR's defensive checks would be no-ops; they pre-date the slice 3 base-class hardening. * The same ``if quant_method is None`` defensive insert in ``ReplicatedLinear.forward`` — also a no-op against current ``main`` for the same reason. None of the skipped edits affect the FP4 path; current ``main``'s behavior on those lines is strictly stronger than the source SHA's. This mirrors slice 5's intentional skip of the ``sage_attn3.py`` head_size removal (see PR hao-ai-lab#1383). Also dropped: an unused ``from contextlib import nullcontext`` import that the source PR staged in ``wanvideo.py`` for a deeper-stack slice (ruff would reject it as unused). Stacking -------- Base: ``main`` (slice 5 / PR hao-ai-lab#1383 merged at ``ba75ad82dbe4a7069412494c051c1c69155fdc9d``). No stack dependency — this is a clean linear PR off ``main``. Provenance ---------- Files extracted from PR hao-ai-lab#1225 (hao-ai-lab#1225) at source SHA ``3f818d0fc532ec6494b465967d5f485150917d0c`` and audited against current ``main``. ``mlp.py`` and ``wanvideo.py`` (modulo the dropped unused import) were applied directly — ``main`` had not diverged from the source's merge-base for those files. ``linear.py`` was hand-merged to preserve current ``main``'s slice-3 hardening (see the ``NOT applied`` list above); only the additive shape-tracking surface was carried over. Pre-commit gate (yapf + ruff + codespell + mypy) passes on all three changed files. Test plan --------- No new tests this slice. The shape-tracking surface is opt-in instrumentation (default disabled) and the ``quant_config`` plumbing is dormant until a future slice sets ``config.quant_config`` to a non-None value. The activation slice (12/12) will carry the contract test for the full FP4 Wan-2.1 path. Sequence -------- Attn-QAT-Stack: 6/12. Earlier merged slices: 4/12 (PR hao-ai-lab#1358), 5/12 (PR hao-ai-lab#1383). Out of scope for this slice: the actual FP4 activation switch, weight-loading conversion, and any cross-cutting config registration (later slices). Co-Authored-By: Peiyuan Zhang <a1286225768@gmail.com> Co-Authored-By: Matthew Noto <notomatthew31@gmail.com>
SolitaryThinker
added a commit
to SolitaryThinker/FastVideo
that referenced
this pull request
Jun 8, 2026
Slice 6 of 12 in the PR hao-ai-lab#1225 decomposition. Tier 2 — backward- compat additions, gated paths only. No activation in this slice. What this adds -------------- * fastvideo/layers/linear.py (+54): adds opt-in shape-tracking instrumentation to ``ReplicatedLinear`` so upcoming QAT-aware backends can discover which GEMM shapes need quantized kernels. Gated by the class attr ``enable_shape_tracking = False``; the default forward path is bit-identical to pre-slice behavior. Adds ``get_shape_mapping``, ``reset_shape_tracking``, ``_track_shape``, ``print_shape_summary``. No new constructor params. * fastvideo/layers/mlp.py (+22): adds an optional ``quant_config: QuantizationConfig | None = None`` kwarg to ``MLP.__init__`` and threads it (plus an explicit ``prefix``) into the two underlying ``ReplicatedLinear`` instances. When ``quant_config is not None``, runs ``process_weights_after_loading`` on each sub-layer's resolved quant method. When ``quant_config is None`` (default), behavior is unchanged: ``ReplicatedLinear`` falls back to ``UnquantizedLinearMethod`` exactly as before. * fastvideo/models/dits/wanvideo.py (+67): wires ``quant_config`` through ``WanSelfAttention``, ``WanI2VCrossAttention``, ``WanTransformerBlock``, ``WanTransformerBlock_VSA``, and ``WanTransformer3DModel`` constructors so a future ``NVFP4QAT``- configured Wan2.1 build can quantize its attention QKV/out projections and FFN. Reads ``config.quant_config`` from ``WanVideoConfig`` (the field is already present on the shared ``DiTBaseConfig``). All new kwargs default to ``None``; default Wan2.1 path stays bit-identical. Files in PR hao-ai-lab#1225 considered but NOT applied -------------------------------------------- The source-SHA ``fastvideo/layers/linear.py`` also contains several edits that pre-date current ``main`` and would silently regress it: * Removal of the ``NVFP4Config``-only-quantizes-a-curated-subset explanatory comments in ``LinearBase.__init__`` and ``ReplicatedLinear.__init__`` (added on main as part of slice 3 / PR hao-ai-lab#1336). * Removal of the ``if self.quant_method is None: self.quant_method = UnquantizedLinearMethod()`` fallback inside ``LinearBase.__init__`` (also part of the slice 3 hardening). * A constructor / ``create_weights`` reformat from multi-line to compact one-line style — pure style noise. * ``assert self.quant_method is not None`` → ``if self.quant_method is None: self.quant_method = UnquantizedLinearMethod()`` in ``ColumnParallelLinear.__init__/forward`` and ``RowParallelLinear.__init__/forward``. ``LinearBase.__init__`` on current ``main`` already guarantees ``quant_method`` is non-None, so the source PR's defensive checks would be no-ops; they pre-date the slice 3 base-class hardening. * The same ``if quant_method is None`` defensive insert in ``ReplicatedLinear.forward`` — also a no-op against current ``main`` for the same reason. None of the skipped edits affect the FP4 path; current ``main``'s behavior on those lines is strictly stronger than the source SHA's. This mirrors slice 5's intentional skip of the ``sage_attn3.py`` head_size removal (see PR hao-ai-lab#1383). Also dropped: an unused ``from contextlib import nullcontext`` import that the source PR staged in ``wanvideo.py`` for a deeper-stack slice (ruff would reject it as unused). Stacking -------- Base: ``main`` (slice 5 / PR hao-ai-lab#1383 merged at ``ba75ad82dbe4a7069412494c051c1c69155fdc9d``). No stack dependency — this is a clean linear PR off ``main``. Provenance ---------- Files extracted from PR hao-ai-lab#1225 (hao-ai-lab#1225) at source SHA ``3f818d0fc532ec6494b465967d5f485150917d0c`` and audited against current ``main``. ``mlp.py`` and ``wanvideo.py`` (modulo the dropped unused import) were applied directly — ``main`` had not diverged from the source's merge-base for those files. ``linear.py`` was hand-merged to preserve current ``main``'s slice-3 hardening (see the ``NOT applied`` list above); only the additive shape-tracking surface was carried over. Pre-commit gate (yapf + ruff + codespell + mypy) passes on all three changed files. Test plan --------- No new tests this slice. The shape-tracking surface is opt-in instrumentation (default disabled) and the ``quant_config`` plumbing is dormant until a future slice sets ``config.quant_config`` to a non-None value. The activation slice (12/12) will carry the contract test for the full FP4 Wan-2.1 path. Sequence -------- Attn-QAT-Stack: 6/12. Earlier merged slices: 4/12 (PR hao-ai-lab#1358), 5/12 (PR hao-ai-lab#1383). Out of scope for this slice: the actual FP4 activation switch, weight-loading conversion, and any cross-cutting config registration (later slices). Co-Authored-By: Peiyuan Zhang <a1286225768@gmail.com> Co-Authored-By: Matthew Noto <notomatthew31@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Activates the Dreamverse LTX2 integration after the supporting code has landed.
This PR updates the FastVideo registry/model registry and docs navigation so the previously-added LTX2/Dreamverse paths become discoverable. Activation is intentionally near the end so earlier stack PRs are isolated or dead-code additions.
Review focus
Please review this PR as stack slice 13/14. The base branch is the previous stack slice, so the Files Changed view should show only this slice rather than the full Dreamverse integration.
Stack / merge notes
This is part of the Dreamverse stacked PR series. Review this PR against its current base branch to see only this slice. After the previous stack PR lands into
main, retarget this PR's base tomainbefore merging it.Verification
feat/dreamverse-monorepo-main-syncequalsfeat/dreamverse-stack-13-activation.feat/dreamverse-monorepo-main-sync-plus-profileequalsfeat/dreamverse-stack-14-ltx2-profile-speedups.Attribution
This PR is a path-scoped reconstruction from
will/dreamverse-monorepoat03d3e61df69fe99a81eb62d86a3926a5f769f857. The synthetic commit keeps coauthor trailers for the source collaborators, omitting Junda Su because he is already the commit author.