Skip to content

Commit 4ad6bbc

Browse files
committed
[docs] Track Gemma device placement follow-up
1 parent adda651 commit 4ad6bbc

1 file changed

Lines changed: 35 additions & 1 deletion

File tree

.agents/memory/dreamverse-integration/open-threads.md

Lines changed: 35 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,8 @@ recommended next action.
66
For why each item is open see [decisions-log.md](decisions-log.md). For
77
PR-level context see [pr-roadmap.md](pr-roadmap.md).
88

9-
**Last updated:** 2026-05-13 (added DR-5 follow-up for PR #1333's LTX2
9+
**Last updated:** 2026-05-14 (added DR-6 follow-up for PR #1335's Gemma
10+
lazy-load/device-placement behavior outside compiled `forward`. Earlier: 2026-05-13 added DR-5 follow-up for PR #1333's LTX2
1011
distilled SSIM reference refresh. Earlier: 2026-05-12 added DR-4 follow-up for PR #1330's skipped
1112
`App websocket integration` suite. Earlier: 2026-05-05 D-20 broken-pipe root cause + fix landed on
1213
`will/dreamverse-monorepo` @ `5eaf0a13`; added new thread D-20-CP for
@@ -36,6 +37,7 @@ different vehicle.).
3637
| **DR-3** | Low | Replace Dreamverse `PromptEnhancer._run_blocking_request` manual thread/queue polling with `asyncio.to_thread` after the PR #1327 prompt-enhancer compatibility surface is retired or isolated | S | Review comment #1327 (`prompt_enhancer.py`) is valid, but deferred to avoid patching the local fork in this PR. |
3738
| **DR-4** | Low | Investigate unskipping PR #1330's skipped public `App websocket integration` suite | S-M | Public PR #1330 has `describe.skip(...)` around 27 websocket tests while the internal equivalent suite is active with 25 tests. The 2 public-only tests cover backend unreachable / GPU workers not ready. Not blocking while skipped, but stale assertions may need safe refresh before unskip. |
3839
| **DR-5** | Low | Regenerate LTX2-Distilled latent SSIM references under the intended neutral/distilled defaults, then remove the PR #1333 historical full-guidance pins | S-M | PR #1333 changed public LTX2 distilled defaults to neutral/distilled values, but existing LTX2 latent SSIM references appear to have been generated with historical full-guidance defaults. The current PR pins the SSIM test to old values to keep CI compatible until references are refreshed. |
40+
| **DR-6** | Low | Decide whether `LTX2GemmaTextEncoderModel` needs a non-forward device-placement hook after lazy Gemma load | S | PR #1335 should remove the `model.device` / `model.to(...)` guard from `forward` for Dynamo/fullgraph compatibility. Non-compiled runs probably do not need it because `gemma_model` moves Gemma at first load, but a later wrapper `.to(...)` after lazy load could leave Gemma on the old device unless lifecycle placement handles it. |
3941
| **3** | Med | Add `cerebras_ifm` to `PromptEnhancerConfig.provider` Literal + provider | S-M | Public-side resolution if DR-2 picks (a) |
4042
| **4** | Med | Expose `layer_profile` on typed `engine.quantization` | M | Removes Dreamverse's `experimental["pipeline_config"]` dodge for stage profiles |
4143
| **5** | Med | Design typed `dit_config.quant_config` carrier | L design + L impl | Removes broader `experimental["pipeline_config"]` escape hatch |
@@ -423,6 +425,38 @@ current PR #1330 review.
423425
stack end-to-end and compare the public assertions against the internal active
424426
suite.
425427

428+
### Item DR-6: Gemma lazy-load device placement outside `forward`
429+
430+
**Why:** PR #1335 review flagged this pattern in
431+
`fastvideo/models/encoders/gemma.py::LTX2GemmaTextEncoderModel.forward`:
432+
433+
```py
434+
if model.device != target_device:
435+
model.to(device=target_device)
436+
```
437+
438+
The immediate concern is the compiled/Dynamo path: `model.device` and
439+
`model.to(...)` inside `forward` can introduce non-tensor/device parsing work
440+
that fullgraph tracing should not see. Removing the guard from `forward` is the
441+
right PR #1335 review fix.
442+
443+
For non-compiled execution, the guard is mostly defensive rather than required:
444+
`gemma_model` already moves the lazily loaded HF Gemma model to the wrapper's
445+
current parameter device on first load. The remaining edge case is a lifecycle
446+
sequence where Gemma is loaded, then the parent wrapper is later moved to a
447+
different device; in that case Gemma could stay behind unless placement is
448+
handled outside `forward`.
449+
450+
**Action:** After PR #1335 review is unblocked, decide whether FastVideo needs a
451+
small lifecycle hook/helper for this class so lazy Gemma is moved whenever the
452+
wrapper/device placement changes. If yes, implement it outside `forward`; if no,
453+
document that Gemma must be loaded after final device placement.
454+
455+
**Effort:** Small.
456+
457+
**Dependencies:** Not blocking PR #1335 if the forward-path guard is removed and
458+
the normal load path keeps placing Gemma on the wrapper's current device.
459+
426460
### Item D-12-C: Avoid locking `PoolAssignment.gpu_id: int` as public
427461

428462
**Why:** Today `PoolAssignment` exposes `gpu_id: int`, assuming

0 commit comments

Comments
 (0)