Conversation
The Series Arc + Editorial Roadmap split and the inner text + 260px Themes panel both used viewport-based breakpoints (xl:, lg:), so a wide viewport always triggered side-by-side even when the open Bible drawer left the content area too narrow to host them — squeezing the logline text into a thin column. Switched both grids to Tailwind v4 container queries so they respond to the actual content-area / Arc-card width.
Add docs/plans/ as a repo record of feature design plans (one file per plan, dated). Seed it with the Unified Story Builder design and a docs/plans/README.md. Add a CLAUDE.md Git Workflow convention so future approved plans get copied from ~/.claude/plans/ into docs/plans/ before implementation.
Add /story-builder — a guided, linear conductor over the existing Universe/Series/Issue records that walks a story from idea -> universe aesthetic -> plot arc -> reader map -> characters -> issues -> production. Each step is LLM-assisted with an AI-refinement affordance and ends with an explicit lock before the next unlocks; revising an earlier step soft-flags downstream locked steps as stale (integrity gate) via a sha256 hash of each step's upstream inputs, without destroying content. Heavy per-issue production hands off to the existing Pipeline issue page. New series.arc.readerMap field: the audience-experience roadmap (hooks, payoffs, emotional beats, cliffhangers) built on the Vonnegut shape, distinct from the protagonist arc. generateReaderMap/refineReaderMap in arcPlanner.js; arc-overview regeneration now preserves it. Lockable via the existing arcFields mechanism. pipelineSeries wire-schema bumped 1->2 (per-category gate) so an older peer can't round-trip and strip it. Server: services/storyBuilder.js (local-only session store + state machine + integrity-on-read + generate/refine delegation), routes, lib/storyBuilderSteps.js + lib/storyBuilderIntegrity.js, Zod schemas, 3 prompt stages + migration 043. Client: pages/StoryBuilder.jsx stepper, services/apiStoryBuilder.js, App/Layout/navManifest wiring. Seed-mode v1; import-mode intake, in-builder issue seeding, SSE streaming, arc refine-with-feedback, and inline ArcCanvas embedding tracked in PLAN.md.
Data-loss fixes (the reader map / arc could be silently destroyed):
- arcSchema (routes/pipeline.js) now lists readerMap; Zod was stripping it
on every arc PATCH, then updateSeries's wholesale arc replace wiped it.
Regression test added (PATCH /series/:id preserves arc.readerMap).
- generateReaderMap/refineReaderMap throw on an empty LLM payload instead of
returning null; refineReaderMap also falls back to the existing map — a
refine (or generate) that yielded nothing no longer nulls out the saved
reader map. Regression tests added.
- generateStep('plotArc') refuses a null arc rather than writing
updateSeries({ arc: null }) and wiping the whole arc.
- characters step no longer blanket-toggles locked on every universe
character (clobbered per-entry locks + mutated a shared universe); the
session step-lock alone gates the wizard, UI hides per-entry refine.
UX fixes:
- StoryBuilder reload() passes { silent: true } to the universe/series/issues
GETs (added an options param to those wrappers) so a failed fetch no longer
double-toasts over its own .catch fallback.
- <StepPanel> keyed by stepId so RefineBox feedback resets between steps.
- Step manifest now loads before the loading gate clears, so the detail view
never renders an empty step rail.
Deferred (non-blocking, in PLAN.md): createStorySession orphan-universe
rollback, goToStep gate-divergence revert.
The intro screen now has two tabs: 'Start from an idea' (seed) and 'Import a finished work'. Import reuses the existing Importer pipeline — analyzeImport (creates the universe+series, extracts canon/arc/seasons/issues) → preview summary → commitImport (all canon + arc + seasons + issues) → createStorySession in intake='import' mode → drops into the wizard with the stages pre-filled to review and lock. Handles the comic-script mechanical split, an empty-split 'Retry issue split' affordance, and existing-universe merge. Seeds the session seedIdea from the extracted arc summary so the aesthetic step's Expand has a real starter. Fixes a React anti-pattern found while testing: the tab buttons were a component defined inline in the index render (new type each render), so the mode toggle didn't take — inlined the buttons. Tests: import analyze→preview→commit→create flow + the no-issues retry gate. Import aesthetic/reader-map pre-fill gaps tracked in PLAN.md.
Add a provider/model picker (reuses getProviders + filterSelectableModels) at the top of the Story Builder — in the detail header and the import tab. The choice persists on session.llm, and the conductor's generate/refine now default provider/model from session.llm when no explicit per-call override is given, so one selection applies to idea expand, aesthetic, plot arc, reader map, character refine, AND the importer's analyze/retry. The import flow threads the picked provider into analyzeImport/retryImporterIssues and stores it on the created session. - server: generateStep/refineStep resolve reqProviderId/reqModel = options.* || session.llm.* ; storySessionCreateSchema accepts llm. - client: ProviderModelPicker component; import panel local state + detail header persists via updateStorySession; inline tab buttons. Tests: server session.llm-default + explicit-override-wins; client provider threading on import + detail-picker persistence. server 8052 / client 592.
On a successful LOCK, the lock-toggle's onSuccess now advances the current step to the next one (persists the pointer + navigates), so the user no longer has to click Lock and then Next. Unlocking stays put. The standalone Next button remains for moving between already-locked steps.
…exists Each step's generate button flips to 'Re-generate' (with a refresh icon) once that step has generated content — idea (universe logline set), universe aesthetic (premise/styleNotes set), plot arc (arc logline/summary), reader map (readerMap present) — so it's clear the action was already run.
The characters step now renders a preview image per character so the world style + character style can be eyeballed together. Reuses the Universe Builder's exact render path — composeStyledPrompt + universeStylePreset → generateImage → EntryThumbSlot (diffusion spinner via MediaJobThumb, then the image, walk-back on 404) — and persists the resulting filename onto the universe canon entry's imageRefs (section-local renders don't get the server-side append hook). Honors the per-builder provider/model picker and the user's image-gen settings (readPipelineImageSettings). - EntryThumbSlot: optional onComplete pass-through to MediaJobThumb.onFilename. - apiSystem.generateImage: accept an options arg (so the call can be silent alongside its own catch toast). Tests: characters step renders the per-character slot + the generated prompt fuses the descriptor with the universe style. client 595.
…Builder) Add a 'base style image' that renders from the universe's raw style guide alone — styleNotes (style guide) + influences embrace (positive) / avoid (negative), with NO subject — so the user can see the world's base visual emphasis. New StyleProbeImage component (reuses generateImage + EntryThumbSlot + universeStylePreset); buildStyleProbePrompt/hasStyleForProbe helpers. - server: universe gains a styleImageRefs[] field (sanitizeTemplate + createUniverse field list + updateUniverse PATCHABLE_SCALARS, sanitized via sanitizeEntryImageRefs); create + patch Zod schemas accept it. Additive, not wire-version-gated (low-stakes, one-click regenerate). - client: rendered under the style/influences editor in UniverseBuilder (merges only styleImageRefs into the draft) and in the Story Builder Universe Aesthetic step; generateImage gained an options arg already. Tests: prompt-composition helpers (client) + styleImageRefs round-trip, dedupe-on-create, default-[] (server). server 8054 / client 600.
- StepCharacters preview persist: guard against MediaJobThumb's onFilename firing more than once (processedRef per char+filename) and refetch the freshest universe before the imageRefs append, so a sibling character's just-persisted ref isn't clobbered by a stale full-array PATCH. Also dedupe the appended filename. - StyleProbeImage: same multi-fire guard, dedupe, and only reflect the new styleImageRefs once the save actually succeeds (was optimistically showing an image the server may not have persisted). - Lock & continue auto-advance: navigate only AFTER setStoryCurrentStep resolves (don't strand the URL ahead of the persisted pointer on a gate rejection) + defensive !isStale guard. - saveLlm: merge the picker choice into local session state instead of a full reload() (avoids refetching universe+series+issues and flicker for an llm-only change). Deferred to PLAN.md: import partial-commit recovery, styleImageRefs wire/ conflict-journal trade-off, and extracting the duplicated image-render hook / provider picker. client 600.
Local review surfaced six data-loss / silent-failure bugs the prior xhigh
passes missed plus three tightening items:
- generateStep('plotArc') was doing updateSeries({ arc, seasons }) which
bypassed mergeArcWithLocks / mergeSeasonsWithLocks / buildSeasonRemap —
silently dropping per-field arc locks, locked seasons, and orphaning
every child issue attached to a renamed/removed season. Route through
commitSeasonsWithRemap (the Arc Canvas's own helper).
- resolveVerifyIssues (Arc Canvas auto-resolve) wasn't forwarding
series.arc.readerMap into its sanitizeArc call. Because the field has
no per-field lock by default, mergeArcWithLocks didn't restore it on
commit, so a user who'd generated a reader map without locking it lost
it the first time they auto-resolved a finding. Mirrors the fix already
present in generateArcOverview. Regression test added.
- Reader-map prompt JSON example used "kind": "hook|reveal|payoff|..."
which LLMs reproduce literally; sanitizeReaderBeat then drops every
beat because the joined string isn't in READER_MAP_BEAT_KINDS, leaving
beats: [] and tripping the empty-payload check. Use a single valid
example ("hook"); the existing {{beatKindsCsv}} clause enumerates the
vocabulary.
- currentStep PATCH was z.string().max(40); sanitizer silently coerced
unknown values to STEP_IDS[0]. Now z.enum(STEP_IDS) — sanitizer's
coerce-on-LOAD stays for resilience against corrupted persisted state.
- styleImageRefs route caps (.max(50) on POST + PATCH) were 4× the
sanitizer cap (IMAGE_REFS_PER_ENTRY_MAX = 12), so a 40-entry POST
silently 200'd with 28 dropped. Align to entryImageRefsField.
- storyBuilderStore() registered in the boot-time verifyCollectionVersions
array.
- reachable() in StoryBuilder.jsx returns 'unlocked' | 'stale' | true so
the "earlier steps" toast can distinguish staleness from unlocked when
the disabled-button gate fires. Booleanized at the sidebar disabled
prop so the truthy string doesn't re-enable a blocked button.
Deferred to PLAN.md (non-blocking): step-content staleness (lock vs
hash semantics), refineReaderMap providerId/model return, refineReaderMap
fallback changes/rationale, hash-shape vs derivation test, migration-043
line-ending precedent, StyleProbeImage draft-vs-saved divergence, second
swallow-rejection path in lock.toggle onSuccess.
server 8055/8055 + 7 skipped, client 600/600.
…ale hash
Two P2 findings from codex's review of the PR:
- generateStep('idea') at storyBuilder.js:441-443 was writing both
starterPrompt AND logline via a plain updateUniverse(). After the user
locks the universeAesthetic step (which sets universe.locked.{logline,
premise,styleNotes,influencesEmbrace,influencesAvoid}=true), re-running
the idea step would silently overwrite the locked logline because
updateUniverse doesn't enforce per-field locks on scalar writes. Skip
any patched scalar that's locked on the current record.
- buildUpstreamInputs() at storyBuilder.js:270-272 was tracking only
session.seedIdea for the universeAesthetic / plotArc steps. But the
idea expand is non-deterministic — same seed can produce different
universe.starterPrompt + series.logline/premise, all of which feed
the aesthetic and plotArc generators. A locked aesthetic / plotArc
step would NOT flag stale after re-running idea with the same seed,
leaving the wizard advancing on locks derived from older expanded-idea
outputs. Fix: include the idea-step OUTPUTS (universe.starterPrompt,
series.logline, series.premise) in the upstream hash inputs for both
downstream steps.
Two regression tests added:
- storyBuilder — generate delegation: generateStep(idea) skips writing a
locked universe.logline (asserts the lock-honoring fix).
- storyBuilder — integrity / staleness: flags a locked universeAesthetic
stale when the idea step re-runs with a new starterPrompt (asserts the
stale-hash fix).
server 8057/8057 + 7 skipped, client 600/600.
feat: Unified Story Builder + universe style-probe + arc reader map
…rrors When `ollama pull` fails with `412: requires a newer version of Ollama`, the Local LLMs tab now surfaces an inline upgrade affordance instead of the raw error: `brew upgrade ollama` on macOS, the official install script on Linux, restart the service, and auto-retry the pull. LM Studio gets the same path (Homebrew cask) with a manual restart hint.
The upgrade path assumed Homebrew; running `brew upgrade ollama` against the official `.app` install exits 1 with stderr `Error: ollama not installed`, which the streaming runner then dropped, surfacing only `exited with code 1` in the UI and nothing in server logs. Detect the actual install source on macOS — Homebrew formula (`ollama`), Homebrew cask (`ollama-app` / `lm-studio`), or the official self-updating `.app` — and route to the right action. For the `.app` case, return manual-update instructions (the app's own updater handles it) instead of running brew against a package it doesn't own. `runStreaming` now retains a 1KB tail of recent stderr/stdout and includes it in failure messages, and brew failures are logged server-side.
On macOS the .app's binary (linked through /usr/local/bin/ollama) is what `ollama serve` actually runs, so brew upgrade put a fresh binary in the Cellar but the old .app server kept serving — model pulls kept failing with the same 412 until Ollama was manually stopped. Upgrade path on macOS now bypasses brew when /Applications/Ollama.app exists: pulls the latest Ollama-darwin.zip from GitHub releases, force-kills the running Ollama, replaces the .app bundle in place, strips quarantine, relaunches it, and polls /api/version until the new binary is serving. On Linux still runs the official install script; for headless macOS installs (brew formula only, no .app) still runs brew upgrade ollama. UI now auto-triggers the upgrade on OLLAMA_OUTDATED (no confirm click) and renders a prominent yellow warning banner with AlertTriangle, bold heading, and live step-by-step progress instead of a quiet inline row.
Vite's host check rejected *.ts.net hostnames with 'Blocked request', breaking npm run dev when launched via Tailscale. Add '.ts.net' (and 'localhost') to server.allowedHosts.
…e, fix lightbox prompt The base style image misrepresented downstream renders by mixing styleNotes into its prompt — that field is prose for writers/CDs and never reaches the image model on canon refs, variations, sheets, or comic pages. The probe now uses only the same embrace/avoid influences preset every other prompt layers on, so it's a truthful preview. Also add a Regenerate button (visible once a probe exists) and fix the lightbox preview showing "Base style" instead of the real prompt — the gallery-metadata map driving the lightbox wasn't being refreshed after a style-probe render, so it fell through to the row label.
… filter - Brain Links quick-add gains collapsible optional title & tags fields - Edit form exposes URL alongside title with labels; URL change re-derives GitHub metadata, guards duplicates, resets stale clone state - Fix Other Links filter showing GitHub repos (z.coerce.boolean coerced the string 'false' to true); parse the query string explicitly
End-to-end fix for the "configure local Python → install packages →
generate a video" flow on Apple Silicon. The previous flow hit five
distinct walls in sequence; each one looked like the user's fault but
was a real bug:
1. Video Gen's "Local Python not configured" warning only linked to the
Settings drawer. Now LocalSetupPanel renders inline below the status
pill so detect / install / Create-venv all happen in place; saving
a new pythonPath re-polls status so the pill flips green.
2. detectPython() preferred /opt/anaconda3/bin/python3 over
/opt/homebrew/bin/python3. On Apple Silicon, anaconda is often x86_64,
and `mlx` ships arm64-only wheels, so the install fails with
"No matching distribution found for mlx". The detector now probes
platform.machine() per candidate on darwin/arm64 and prefers a matching
interpreter; /api/image-gen/setup/check returns interpreterArch +
suggestedArm64Python so LocalSetupPanel can surface the warning and
offer a one-click "Switch to detected arm64 Python" affordance.
3. isExternallyManaged() false-positived on PortOS-owned venvs created
from PEP 668 Homebrew Python — sysconfig.get_path("stdlib") inside a
venv resolves to the base interpreter's stdlib, so the venv inherited
Homebrew's EXTERNALLY-MANAGED marker. Now also checks
sys.prefix != sys.base_prefix and short-circuits to false for venvs.
4. /api/video-gen/status returned `connected: !!pythonPath` — the pill
went green as soon as any pythonPath was saved, even when none of the
required packages were installed. Now probes the imports via
checkPackages and returns connected:false + missingPackages list when
anything is absent, which keeps the inline panel showing the install
button. LocalSetupPanel also notifies the parent on local check
transition to "all installed" so the pill flips without waiting on a
manual refresh.
5. The PyPI package literally named `mlx_video` is unrelated to the
`mlx_video.generate_av` CLI the LTX renderer shells into — that ships
in `mlx-video-with-audio`. Both publish an `import mlx_video`
namespace, so the missing-check passed against the wrong one and the
spawn died with ModuleNotFoundError. pipNameFor('mlx_video') now
returns 'mlx-video-with-audio>=0.1.35' on macOS; the import probe
checks `import mlx_video.generate_av` to distinguish them;
installPackages gained a pre-uninstall step driven by
PIP_PRE_UNINSTALL so the conflicting plain `mlx_video` is removed
before pip refuses to "downgrade" across the namespace collision.
Mirrors what scripts/setup-image-video.sh already did out-of-band.
Separately:
* The media job worker now re-resolves pythonPath from live settings at
dispatch time for every local-Python job, instead of using the
snapshot captured at enqueue time. Symptom this fixes: user switches
their Python in the UI, clicks Generate, and the worker still shells
out to the previous interpreter — because the in-memory queue and
data/media-jobs.json carried the old snapshot. Codex image jobs are
unaffected (they don't run a local Python).
* probePythonHealth() consolidates the three subprocesses /setup/check
used to fan out (checkPackages + isExternallyManaged + probePythonArch)
into one python invocation that returns everything as JSON.
isExternallyManaged becomes a thin wrapper; the standalone export was
unused after the consolidation and is removed.
* installPackages and installFlux2Venv shared a near-identical
spawn-and-stream-stdout helper; extracted to a single streamSpawn().
Tests: 8061 server + 602 client passing (incl. new coverage for
mediaJobQueue's live-pythonPath re-resolution and videoGen /status's
missing-packages branch). Test count grew by 4 since main.
…warning to darwin/arm64 * LocalSetupPanel: the inline path input fired onPythonPathChange on every keystroke. In the VideoGen consumer that meant a settings PATCH + a ~1-2s status re-probe per character. Decouple via local draft state; commit on 800ms debounce or onBlur. Programmatic updates (Detect, Switch-to-arm64, Create-venv) still call onPythonPathChange directly so they take effect immediately. * /api/image-gen/setup/check: generic interpreterArch !== HOST_ARCH would false-positive on Windows (Python reports `AMD64` vs Node's `x86_64`) and on hypothetical arm64 Linux where mlx isn't even in REQUIRED_PACKAGES. Gate the warning to darwin/arm64 with an x86_64 interpreter — the only configuration where mlx actually breaks. The third codex finding (hfDownload.js falls back to broken FLUX.2 venv) is in a file not yet on origin and not part of this PR's surface; left for whichever branch ships hfDownload.
The gemini /do:review pass reviewed the whole repo instead of the branch diff (a known issue tracked under [triage-gemini-out-of-scope-findings]). All 6 findings were in files PR #515 doesn't touch. * Findings 5+6 (Express body limit, * CORS): explicitly intentional per CLAUDE.md Security Model — single-user private-network deployment. * Findings 1-4 (commands.js naive split, fileUtils non-atomic write, two cosAgents non-atomic writes): real but pre-existing, in files unrelated to this PR's surface. Appended to PLAN.md as a new triage entry so they're not silently dropped. Verify each finding's reproducibility before acting; gemini's /do:review has a track record of pattern-matching false-positives.
fix local Python setup for video gen on macOS Apple Silicon
Add arbitrary bucket groups beside the link list: favicon+title chips, inline add/rename/recolor, drag between buckets, per-row bucket dropdown, and an Ungrouped filter. Buckets are a federated brain collection; links gain bucketId/bucketOrder. Two-column desktop layout with the bucket grid in a container-query side panel.
…Qwen, Wan 2.2, HunyuanVideo, LTX-2.3 Unified) Registry-side scaffolding for models that fit comfortably on 64+ GB unified- memory hardware. Runner scripts wired in follow-up commits on this branch. Image: - flux2-klein-9b-bf16 (~36 GB) — native bf16 of the gated 9B Klein repo - hidream-i1-full / hidream-i1-fast — 17B MoE DiT, needs Llama-3.1-8B as text-encoder-4 (gated) - qwen-image / qwen-image-edit — 20B MMDiT, Apache 2.0 ungated - new runner families: 'hidream', 'qwen' (server + client mirrors) Video: - ltx23_unified un-deprecated — bf16 ~48 GB now practical - wan22_t2v_a14b, wan22_i2v_a14b — MLX port at osama-ata/Wan2.2-mlx - hunyuan_video — MLX port at gaurav-nelson/HunyuanVideo_MLX cfgDisabled backfill extended to flux2-klein-9b-bf16 and hidream-i1-fast.
…koro
Workflow: 'I want to render with a big diffusion model (FLUX.2 9B bf16,
HiDream, Qwen-Image) — what's holding unified memory right now?'
Backend additions:
- ollamaManager.getLoadedModels() wraps Ollama /api/ps
- ollamaManager.unloadModel() uses the documented keep_alive=0 trick on
/api/generate with an empty prompt — no tokens generated, immediate evict
- voice/tts-kokoro.unloadKokoro() drops the cached kokoro-js instance
- Routes:
- GET /api/local-llm/loaded
- POST /api/local-llm/unload { backend: 'ollama', modelId }
- GET /api/voice/tts/status
- POST /api/voice/tts/unload
- POST /api/voice/whisper { action: 'start' | 'stop' }
UI: MemoryManagement.jsx mounted at top of Local-LLM settings tab. Lists
each resident model with per-row Unload + a 'Free everything' fan-out.
Polls every 5s. Errors surface via default apiCore toast (single layer).
HiDream + Qwen reuse the existing diffusers runner (scripts/z_image_turbo.py) and the FLUX.2 venv. The runner script gains three new flags: --text-encoder-repo / --text-encoder-class / --tokenizer-class which trigger load_external_text_encoder() — used by HiDream to load meta-llama/Meta-Llama-3.1-8B-Instruct as text_encoder_4 / tokenizer_4 on the HiDreamImagePipeline constructor. Other diffusers-runner models leave the new flags unset and their load path is unchanged. FLUX.2 9B bf16 lands as quantization='none' in scripts/flux2_macos.py — loads Flux2KleinPipeline directly from the gated base repo with no SDNQ / Int8 detour. Practical on 64+ GB unified-memory hardware. server/services/imageGen/local.js dispatch collapsed from explicit 'isZImage || isErnie' branches to usesDiffusersRunner(model) — adding the HiDream + Qwen runner ids works without touching the branch.
…nst stale closure, drop tqdm message spam, add route tests
Four Copilot findings, all real:
1. /active leaked server-internal absolute paths (sourceImagePath,
audioFilePath, pythonPath, uploadedTempPath(s), extendFromVideoPath)
via the raw job.params spread. Whitelist the UI-safe fields the form
actually consumes via pickJobParams.
2. The resume useEffect([]) captured `generating` from the initial
render, so a render started during the in-flight getActiveVideoJob()
request would see the stale `false` and the resume path would
attach a second SSE. Switch the guard to runTokenRef.current — refs
are always live — keeping the eventSourceRef belt-and-suspenders
for the brief POST-pending window before attachJobEvents runs.
3. The tqdm-percent branch forwarded the raw progress-bar line as the
SSE `message`, so every percent update clobbered the last
meaningful STATUS/STAGE line ("Loading model", "Generating I2V")
with terminal noise like '60%|██████ | 6/10 [00:30<00:20, ...]'.
The UI renders the percentage separately; omit `message` on the
queue-dispatched emit so the prior status text persists.
4. Route tests for /active were missing. Added 4 cases covering the
null/no-jobs path, running-over-queued precedence, newest-queued
selection (matches /cancel), and the params whitelist enforcement.
Post-rebase self-review flagged that ACTIVE_JOB_PARAM_FIELDS in /api/video-gen/active intentionally omits `keyframes`, and the VideoGen resume effect has no setter for them either — so a user reloading mid-render on a multi-keyframe FFLF job loses the keyframe-picker state. Out of scope for this PR (UI restoration is independent of the SSE re-attach); captured as a follow-up.
…reference editing in flux2 runner scripts/flux2_macos.py now branches into Flux2KleinKVPipeline when one or more --reference-images are supplied, builds the parallel PIL list at the output resolution, and forwards it as the pipeline's image= kwarg so the K/V-cached reference-token attention path actually runs. The three load helpers (sdnq, int8, bf16) take an explicit pipeline_cls so either pipeline class can drive each quantization branch; component init signatures are identical so the same flux2-klein repo loads either way. flux2-klein-9b's tokenizerRepo moves to black-forest-labs/FLUX.2-klein-9B-kv in the seed registry (data.reference/media-models.json) and the inline DEFAULT_REGISTRY (server/lib/mediaModels.js). The runner's existing probe_hf_auth then surfaces a clean USER_ERROR:gated_repo message pointing the user at the kv license URL on first use — accepting it enables both single-image and multi-reference renders on this model. Migration 045 swaps the persisted value on existing installs only when it still matches the pre-change shipped string, preserving any user-customized fork pin. The Python runner warns when --reference-strengths carries non-default values (the KV pipeline doesn't yet expose per-reference attention weighting; route still emits 1.0 for unset slots so the warning only fires on explicit non-1.0 weights) and when --image-path is supplied alongside refs (init image gets ignored — i2i strength and KV-cache reference conditioning are incompatible in the current pipeline).
…view to render aspect Wan 2.2's import probe checked torch+transformers+huggingface_hub, which passed even when einops (imported transitively by wan/modules/vae2_1.py but absent from upstream's pyproject.toml) was missing — install banner stayed hidden and renders failed with an opaque "Exit code 1" over a deep ModuleNotFoundError. Probes now exercise the real import chain (`import wan`; `sys.path.insert` + `import hyvideo.inference`), and the stderr handler scans for ModuleNotFoundError, swapping the generic exit code for a runtime-specific install hint and invalidating the ready cache so the install banner re-appears immediately. Preview pane was hardcoded aspect-video (16:9), so a portrait 576×1024 request previewed in a letterbox. Now sizes to the configured W×H ratio within a 420×420 budget.
…load Resume video-gen progress on page reload
- Refuse multi-reference editing on int8 quantization with a clear error instead of crashing mid-inference; Flux2KleinKVPipeline's reference-attention hooks don't compose with the quanto-rehydrated transformer attached in the int8 load path. - Rename sidecar field referenceStrengths -> referenceStrengthsRequested so it doesn't pretend per-ref weights were applied (the KV pipeline doesn't expose per-reference attention weighting today; the runner only warns on non-1.0 entries).
- Lazy-import the chosen pipeline class so a venv whose diffusers pin predates Flux2KleinKVPipeline still serves single-image renders; the KV branch fails loudly with an INSTALL_FLUX2 hint instead of breaking every FLUX.2 render with an opaque ImportError. The existing FLUX.2 venv health probe only verifies Flux2KleinPipeline; before this change, importing the KV class at module load would mark such an install healthy then crash on every single-image render too. - Rename the JS sidecar field referenceImageStrengths -> referenceImageStrengthsRequested to mirror the Python rename and match the contract that per-ref weights are NOT yet honored. - Update the Image Gen reference-images warning to reflect the new reality (multi-ref is wired; per-ref strength sliders are advisory until diffusers exposes a per-ref attention knob).
…ck the largest model that fits The model dropdown previously offered every video model in every mode, leaving the user staring at a warning telling them to switch to a model that wasn't even in the list. a2v requires the ltx2 runtime (server enforces A2V_REQUIRES_LTX2), so filter the options to compatible models when mode is a2v. Auto-select picks the highest-memory ltx2 model that fits within systemMemoryGb - 16 GB headroom (OS + text encoder + working set). Falls back to the smallest if nothing fits so the user can still try. Server /status now returns systemMemoryGb via os.totalmem() to drive the choice client-side. When no ltx2 models are installed at all, the inline warning explains how to add a dgrauet entry + provision the runtime, replacing the old copy that pointed at entries that may not exist.
… name the swapped model in toast + cover new status field - typeof status?.systemMemoryGb === 'number' so 0 GB (sub-GB box) flows through the fits check and picks the smallest model instead of being treated as 'absent, assume unlimited' and picking the largest. - Toast on stale-id swap now names the destination model. In a2v mode the fallback is the largest-fits model, not status.defaultModel, so 'using default' was misleading. - Pin systemMemoryGb (type + > 0) in the /status route test so a future accidental removal of the field is caught.
- Extend the int8+kv refuse gate to bf16 (quantization='none') too: the bf16 model's repo points at the base 9B (not the kv variant) whose transformer wasn't tuned for the reference-editing task, so the KV pipeline would run but produce off-task output. Lifting the gate per quantization once the corresponding KV path is validated is tracked as [flux2-bf16-kv-multi-reference-support]. - Pass formats=['PNG','JPEG','WEBP'] to PIL Image.open for the init and reference inputs so a magic-byte-spoofed file can't route into PIL's EPS/PDF/Ghostscript handler. Defense in depth — the route layer already enforces the mime allow-list, but PIL's Image.open dispatches on magic bytes, not extensions. - Add PLAN follow-ups for the bf16+kv gate lift and the deferred sidecar-meta test coverage gap.
…thon-runner feat([flux2-multi-reference-python-runner]): wire diffusers KV multi-reference editing for FLUX.2
Filter Video Gen model dropdown to ltx2 runtime in a2v mode + auto-select largest model that fits
…-local-dir` / `--ignore` flags
`expandHome` was duplicated inline in `server/lib/mediaModels.js` and
`server/services/referenceRepos.js`. Moved to `server/lib/fileUtils.js`
and added a `typeof p !== 'string'` guard so a non-string input falls
through instead of throwing on `.startsWith` (the old mediaModels copy
would have thrown). Updated the lib README and barrel-tracked exports.
`scripts/hf_download_repo.py` gains two flags needed by the upcoming
HiDream-O1 BYOV installer:
--local-dir materialize the repo as a flat copy at a given path
instead of relying on the standard HF cache symlinks
(HiDream-O1's installer needs a real on-disk repo)
--ignore PAT fnmatch glob skip filter, repeatable — lets the caller
drop non-weight subdirs (scripts/, docs/) on download
Also backfills `RUNNER_FAMILIES.HIDREAM` / `QWEN` cases in
`server/lib/runners.test.js` (the runtime module already exports them
and the client mirror already carries them — the test was stale).
…ect unit tests - voice/config.js `expandPath` is now a re-export of the shared `expandHome` (5 caller sites under server/services/voice/ keep working via the alias). - routes/scaffold.js now routes its `~` / `~/` / `~\` handling through `expandHome` instead of its own inline branch — required extending `expandHome` to also accept `~\` (Windows). Embedded `~` chars (e.g. `iCloud~md~obsidian`) still pass through unchanged. - lib/fileUtils.test.js gets a direct describe block for `expandHome` covering bare `~`, `~/foo`, `~\foo`, absolute / relative / empty passthrough, non-string passthrough, and the embedded-tilde case. - runners.test.js now exercises `isQwen` alongside `isHiDream` so the predicate pair stays in sync if either is renamed/dropped.
Gemini's /do:review pass against PR #526 surfaced a real-looking latent bug in `server/services/videoGen/local.js:571` — the `lastImageWillBeUsed` gate skips the ltx2 FFLF case where both start AND end frames are provided, leaving the end frame unresized. Out-of-scope from PR #526 (which only touches `expandHome` consolidation + HF download flags), so filed as a PLAN follow-up for verification + triage instead of applying.
…ymlinks) `hf_hub_download(..., local_dir=...)` defaulted to symlinking into the HF cache on huggingface_hub < 0.23 — which defeats the whole point of `--local-dir` for BYOV installers like HiDream-O1 that need real files to walk a flat on-disk repo. Pass `local_dir_use_symlinks=False` whenever `--local-dir` is set, but gate it on a one-time `inspect.signature` probe: newer huggingface_hub deprecated the kwarg (always copies) and eventually removed it, so the probe keeps us forward-compatible as the FLUX.2 venv rolls upstream.
extract `~/`-expanding `expandHome` to fileUtils + add HF download `--local-dir` / `--ignore` flags
scripts/flux2_macos.py now installs a runtime patch on diffusers' Flux2KVLayerCache.store (steps 1+) and _flux2_kv_causal_attention (step 0 extract path) that scales each reference's V slice by its strength while leaving K and softmax allocation untouched - IP-Adapter-style "ref scale" semantics. 1.0 reproduces upstream behavior, 0.0 ignores that reference, intermediate values attenuate proportionally. The store-path scaling mutates the V tensor in-place because upstream already passes a fresh clone (Flux2KVAttnProcessor line ~454); the attention path still clones because `value` there is the live shared sequence used by downstream tokens. UI: removed the misleading "Per-reference strengths not honored yet" warning. Sidecar fields renamed (server: referenceImageStrengthsRequested -> referenceImageStrengths; python: referenceStrengthsRequested -> referenceStrengths) since values now reflect what was actually applied. A curl caller mismatching strength count vs ref-token count gets a one-shot stderr warning instead of silent fallthrough. PLAN [flux2-runner-sidecar-tests] updated to the new field name.
…, privacy-respecting favicon, TOCTOU race in BYOV install SSE, skip empty story-builder updateSeries
…ream aux text encoder in download status + prefetch
…rify PORTOS_SCHEMA_VERSIONS docstring vs storage-layout version
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release v2.12.0
Released: 2026-05-29
Overview
A "big local models" + storytelling release. The image and video stacks open up to 64+ GB unified-memory Apple Silicon hardware: FLUX.2 Klein 9B bf16, HiDream-I1 Full/Fast, Qwen-Image / Qwen-Image-Edit, Wan 2.2 T2V/I2V, and HunyuanVideo land as first-class entries, sharing a single diffusers runner where they can and getting dedicated BYOV runtime installers where they can't. A new Memory Management panel (Settings → Local LLMs) surfaces what's resident and lets you evict Ollama models / Whisper / Kokoro before a render. FLUX.2 multi-reference editing on Apple Silicon is wired through diffusers' KV-cache pipeline, with per-reference strengths actually honored end-to-end via a runtime patch.
On the authoring side, a new Unified Story Builder (
/story-builder) walks a story from idea → universe aesthetic → plot arc → reader map → characters → issues → production as one guided flow — including an importer that reverse-engineers a finished work (comic script / screenplay / novel) back into the same wizard. Universes get a base style probe image shared between the Universe Builder and Story Builder. A series arc now has a Reader Map distinct from the protagonist arc.Brain → Links picks up buckets (drag-to-organize bookmark groups, federated across peers), keyword search, optional title/tags on quick-add, an editable URL on existing links, and a fixed Other-Links filter. Image / Video Gen surfaces inline model download status + SSE progress so users see a multi-GB pull coming before hitting Render; the live preview matches the configured aspect ratio and shows a step counter; hi-res presets open to HiDream + Qwen; Qwen gets a comic-page draft preset and a stepwise preview decoder. Ollama auto-upgrades in place when a
pullhits a 412-newer-version error, with a prominent progress banner.Rounding it out: scheduled reference-watch can actually write to PLAN.md again (the cron default had drifted to read-only), Vite dev allows
*.ts.nethosts for Tailscale MagicDNS, the Apple-Silicon local-Python detector skips x86_64 candidates, and a multi-agent review pass cleaned up eight cross-file regressions in the big-local-models branch.See
.changelog/v2.12.0.mdfor the full Added / Changed / Fixed breakdown.Test plan
cd server && npm test— 365 files / 8113 tests passcd client && npm test -- --run— 61 files / 608 tests passcd client && npm run build— cleanFull Diff: v2.11.0...v2.12.0