Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .changelog/NEXT.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ TBD
- **Image Gen: hi-res presets opened to HiDream + Qwen-Image, plus a Qwen comic-page draft preset.** `client/src/lib/imageGenResolutions.js`'s `compatibilityKey` previously fell through to `flux1` for HIDREAM/QWEN, silently denying their natural `1024×1536` / `1536×1024` / `1536²` hi-res variants. Both runners now return their own keys, the three hi-res entries advertise compatibility for FLUX2 + QWEN + HIDREAM, and the Flux portrait/landscape entries (`832×1216`, `1216×832`) also open to them. Added a Qwen-only `1328×2048 (Qwen comic draft — experimental)` preset for US-comic-trim-aspect draft renders — past Qwen's documented ~1664 long-edge support window but coherent in practice on the 20B MMDiT; pair with a 1.5× upscale for final 1988×3056 output. Partial fix for `[hidream-qwen-client-runner-consumers]`; LoRA-picker side still pending.
- **Image Gen preview: step counter under the preview.** The live diffusion preview now shows a `step 32/50 · 64%` row below the image while a render is in flight. The progress payload already carried `step`/`totalSteps` from `imageGen/local.js`; just rendered them. Hidden for Codex / external paths that don't emit per-step events.
- **Qwen-Image stepwise preview decode.** `scripts/z_image_turbo.py` gained a Qwen-aware preview-decoder branch: Qwen-Image (and Img2Img / Edit siblings) packs latents to 3D `(B, num_patches, C*4)` like Flux, then unpacks to a 5-D video-VAE layout and unnormalizes with per-channel `latents_mean`/`latents_std` lists from `vae.config` before decoding. The generic `latents/scaling + shift` path matched neither step. New unpack helper (3D→5D) + Qwen-specific decoder; per-channel norm tensors are built CPU-side once and lazily migrated to (device, dtype) per call so the diffusion loop's ~30–50 fires don't rebuild from a list every step.
- **[ref-watch-phosphene-pin-mlx-lazy-eval-cadence-envs] Faster LTX-2 video renders on Apple Silicon.** Tuned the MLX lazy-eval cadence the LTX-2 helper uses so denoise steps run at the cadence phosphene's published M-series perf matrix shows is fastest — also avoids a known post-decode hang the prior setting could exhibit. No UI change; renders just finish sooner.
- **Suppress cosmetic CLIP-truncation warning on multi-encoder pipelines.** The "input was truncated because CLIP can only handle N tokens" warning from diffusers was bleeding red into PortOS server logs on long prompts with HiDream, Qwen-Image, FLUX.2, Z-Image, and ERNIE — but on every one of these pipelines CLIP only contributes pooled style vectors derived from the first ~77 tokens while a high-context encoder (T5 / Llama-3.1-8B / Qwen2-VL) reads the full prompt. New `suppress_cosmetic_clip_truncation()` helper in `scripts/_runner_common.py` attaches an idempotent logging filter to the `diffusers` and `transformers` handlers (filter is on handlers, not loggers, because logger-level filters don't apply to propagated child records). Called from `scripts/z_image_turbo.py` after `import diffusers` and from `scripts/flux2_macos.py` after the pipeline-load dispatcher.

## Fixed
Expand Down
1 change: 0 additions & 1 deletion PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@ _Batch-cleared 2026-05-25: 23 Next Up items shipped together via parallel sub-ag
- [ ] [chrome-canary-followups] **Hardening for the custom-Chrome-binary feature (xhigh code-review 2026-05-25).** _DONE in the v2.10.0 release review: (b) both `spawn()` calls now have `.on('error', …)` listeners; (d) `browser/server.js#loadConfig` now try/catches the `JSON.parse`; (e) `setup-browser.js#loadConfig` now warns + returns `null` and `applyCanaryToConfig` skips the save when the existing config is unreadable; (f) top-level `runCanarySetup()` is now `.catch()`-wrapped. (c) is tracked separately in `[setup-browser-canary-headless]`. Remaining: (a), (g)–(o)._ **High-severity:** (a) On macOS headed mode (the default), `browser/server.js:208` uses `macAppBundle` only and silently ignores `chromePath` — a UI user who fills in `chromePath` for Canary/Chromium/Brave but leaves `macAppBundle` empty gets stock Chrome and the log misreports the binary; couple the two fields in the UI (or auto-derive `macAppBundle` from `chromePath` when the latter is inside a `.app`). (b) Neither `spawn(chromePath, …)` (line 216) nor `spawn('/usr/bin/open', …)` (line 208) has an `.on('error', …)` listener — a typo in `chromePath` emits 'error' with no listener → `uncaughtException` → portos-browser PM2 child crashes and restart-loops. (c) `scripts/setup-browser.js#applyCanaryToConfig` writes `chromePath` + `macAppBundle` but doesn't flip `headless: false`; combined with the seed default of `headless: true` (`data.reference/browser-config.json`), a fresh install accepting Canary runs Canary invisibly. (d) `browser/server.js#loadConfig` does bare `JSON.parse(raw)` with no try/catch — combined with setup-browser's non-atomic `writeFileSync` and the `cachedConfig` race, a partial-file write crashes the supervisor on next start (PM2 restart-loop forever). **Medium-severity:** (e) `setup-browser.js#loadConfig` silently catches all JSON parse errors and returns `{}`, then `applyCanaryToConfig` saves `{chromePath, macAppBundle}` only — wiping every other user-customized key. (f) Top-level `await runCanarySetup()` has no try/catch — any EACCES on data/browser-config.json aborts `npm run setup` / `update.sh` at what was previously a no-op step. (g) Idempotency guard only checks `chromePath`: users who decline get re-prompted on every update, and users who set only `macAppBundle` keep getting re-prompted. (h) `PORTOS_USE_CANARY` only matches literal `'0'`/`'false'` (opt-out) or `'1'`/`'true'` (opt-in) — `'no'`/`'off'`/`'yes'`/`'on'`/`'True'` fall through both branches. (i) `spawnSync(install.cmd, …, { stdio: 'inherit' })` for brew/winget can hang on a sudo password prompt under non-TTY + `PORTOS_USE_CANARY=1` (update.sh stalls). (j) `cachedConfig` in `browserService.js` is stale relative to setup-browser's direct write — GET /api/browser/config returns pre-update values until process restart. (k) `saveConfig` uses bare `writeFileSync` — switch to the canonical `atomicWrite` pattern (`server/lib/fileUtils.js`). **Low-severity:** (l) `spawnSync` failure-status check `result.status !== 0` treats `status: null` (spawn-failure / signal kill) identically to a non-zero exit and never logs `result.error` — masks ENOENT/EPERM/SIGKILL. (m) `optionalPath` Zod schema accepts any string up to 1024 chars; no `.app`/`.exe` sanity check — user pastes the bundle into `chromePath` → spawn() EISDIR. (n) `launchBrowser`'s reuse-existing-Chrome early-return (line 167) fires BEFORE `headlessMode = config.headless === true` (line 170), leaving the module-level default `false` after a PM2 restart that reuses Chrome — /health reports wrong mode (pre-existing, but in a function touched by this change). (o) `detectCanary` on macOS only checks `/Applications/...`; misses per-user `~/Applications/Google Chrome Canary.app` installs (corporate Macs, `HOMEBREW_CASK_OPTS=--appdir=$HOME/Applications`).
- [ ] [ref-watch-phosphene-teacache-extend-a2v-denoise] **TeaCache through Extend + A2V Stage-1 denoise loops.** From `reference-watch` review of phosphene (commits `ea98aad8` + `17be2a79`, 2026-05-28). PortOS's `scripts/generate_ltx2.py` invokes `ExtendPipeline.extend_from_video(...)` (~line 378) and `AudioToVideoPipeline.generate_and_save(...)` (~line 417) directly without passing a TeaCache controller, so the slowest two modes in the panel run full denoise. Phosphene reuses the existing Stage-1 calibration (`ti2vid_two_stages._build_teacache_controller(n_steps, thresh=0.5)`) by monkey-patching `guided_denoise_loop` on the `ltx_pipelines_mlx.retake` and `ltx_pipelines_mlx.a2vid_two_stage` modules, gating activation on a per-call module-level config dict, set before the pipeline call and cleared in `finally`. Predicted ~1.2× on extend at default threshold (up to ~3× at 1.5); similar for A2V Stage 1. Fix: install both monkey-patches at the top of `generate_ltx2.py`, wire `_EXTEND_TC_CONFIG` / `_A2V_TC_CONFIG` from the helper's `run_extend`/`run_a2v` paths, and add a `--no-teacache` CLI flag (default-on). **Decision needed:** ride along with the `[ref-watch-phosphene-bump-ltx2-pin-v0148]` pin bump (the calibration constants are part of the same upstream surface that gets renamed) or land independently against today's pin? Scope: small/medium.
- [ ] [ref-watch-phosphene-strip-malloc-subprocess-env] **Strip `Malloc*` env vars before spawning Python subprocesses.** From `reference-watch` review of phosphene (commit `0d7cb137`, 2026-05-28). PortOS's `server/services/videoGen/local.js:609` passes `process.env` through to the Python child unchanged (`const childEnv = { ...process.env, ...(await hfTokenEnv()) };`). On macOS, when the user launched PortOS from Pinokio or any tool that exports an empty/zero `MallocStackLogging` (or related `Malloc*`) var, every Python subprocess prints `MallocStackLogging: can't turn off malloc stack logging because it was not enabled` once per child exit — flooding stderr (~50+ lines per gen for image-gen, which fans out into download/helper subprocesses). Fix: add a `stripDebugMallocEnv(env)` helper in `server/lib/processEnv.js` (new) — `Object.fromEntries(Object.entries(env).filter(([k]) => !k.startsWith('Malloc')))` — barrel it from `server/lib/index.js` per the module-org rule, document in `server/lib/README.md`. Then route every Node→Python spawn through it: `server/services/videoGen/local.js:611`, plus the image-gen spawn sites (grep `spawn.*python` under `server/`). Scope: small.
- [ ] [ref-watch-phosphene-pin-mlx-lazy-eval-cadence-envs] **Pin `LTX2_DIT_EVAL_EVERY=1` + `LTX2_GEMMA_EVAL_EVERY=1` at helper boot.** From `reference-watch` review of phosphene (commit `ec1437fe`, 2026-05-28). PortOS's `scripts/generate_ltx2.py` doesn't set either env, so the upstream `ltx-2-mlx` default (`=8`) governs lazy MLX graph eval cadence. Phosphene's M4 Max 64 GB matrix on I2V Balanced 5s / 121 f shows `=8` → ~3 min/step, `=1` → ~7 s/step (~25× faster denoise); `=0` is also fast but extends the documented post-decode hang. Fix: add at the very top of `generate_ltx2.py`, BEFORE any `from ltx_pipelines_mlx import …`, `os.environ.setdefault("LTX2_DIT_EVAL_EVERY", "1"); os.environ.setdefault("LTX2_GEMMA_EVAL_EVERY", "1")`. Smoke-test all five modes (t2v/i2v/fflf/extend/a2v) on a real Mac before merging since `=1` interacts with the same post-decode hang the existing `[ref-watch-phosphene-sigkill-watchdog-videogen-local]` slug already plans to mitigate. Scope: small.

## Deferred Architecture (human-led planning)

Expand Down
13 changes: 7 additions & 6 deletions scripts/generate_ltx2.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,12 +46,13 @@
from typing import NoReturn

# Must be set BEFORE any ltx_core_mlx import: ltx_core_mlx.model.transformer.model
# reads LTX2_DIT_EVAL_EVERY at import time. The upstream default (=8) causes
# ~15-25x perf loss on M-series Macs via per-block Metal command-buffer churn;
# =0 disables lazy evaluation and avoids that overhead. setdefault lets a
# caller-supplied env var override these if needed.
os.environ.setdefault("LTX2_DIT_EVAL_EVERY", "0")
os.environ.setdefault("LTX2_GEMMA_EVAL_EVERY", "0")
# reads LTX2_DIT_EVAL_EVERY at import time. Phosphene's M4 Max 64 GB I2V Balanced
# 5s / 121 f matrix: upstream default =8 runs ~3 min/step (per-block Metal
# command-buffer churn); =1 runs ~7 s/step (~25× faster denoise); =0 is also
# fast but extends the post-decode deallocator hang. =1 wins on both axes.
# setdefault lets a caller-supplied env var override.
os.environ.setdefault("LTX2_DIT_EVAL_EVERY", "1")
os.environ.setdefault("LTX2_GEMMA_EVAL_EVERY", "1")


def emit_status(msg: str) -> None:
Expand Down