fix local Python setup for video gen on macOS Apple Silicon#515
Merged
Conversation
End-to-end fix for the "configure local Python → install packages →
generate a video" flow on Apple Silicon. The previous flow hit five
distinct walls in sequence; each one looked like the user's fault but
was a real bug:
1. Video Gen's "Local Python not configured" warning only linked to the
Settings drawer. Now LocalSetupPanel renders inline below the status
pill so detect / install / Create-venv all happen in place; saving
a new pythonPath re-polls status so the pill flips green.
2. detectPython() preferred /opt/anaconda3/bin/python3 over
/opt/homebrew/bin/python3. On Apple Silicon, anaconda is often x86_64,
and `mlx` ships arm64-only wheels, so the install fails with
"No matching distribution found for mlx". The detector now probes
platform.machine() per candidate on darwin/arm64 and prefers a matching
interpreter; /api/image-gen/setup/check returns interpreterArch +
suggestedArm64Python so LocalSetupPanel can surface the warning and
offer a one-click "Switch to detected arm64 Python" affordance.
3. isExternallyManaged() false-positived on PortOS-owned venvs created
from PEP 668 Homebrew Python — sysconfig.get_path("stdlib") inside a
venv resolves to the base interpreter's stdlib, so the venv inherited
Homebrew's EXTERNALLY-MANAGED marker. Now also checks
sys.prefix != sys.base_prefix and short-circuits to false for venvs.
4. /api/video-gen/status returned `connected: !!pythonPath` — the pill
went green as soon as any pythonPath was saved, even when none of the
required packages were installed. Now probes the imports via
checkPackages and returns connected:false + missingPackages list when
anything is absent, which keeps the inline panel showing the install
button. LocalSetupPanel also notifies the parent on local check
transition to "all installed" so the pill flips without waiting on a
manual refresh.
5. The PyPI package literally named `mlx_video` is unrelated to the
`mlx_video.generate_av` CLI the LTX renderer shells into — that ships
in `mlx-video-with-audio`. Both publish an `import mlx_video`
namespace, so the missing-check passed against the wrong one and the
spawn died with ModuleNotFoundError. pipNameFor('mlx_video') now
returns 'mlx-video-with-audio>=0.1.35' on macOS; the import probe
checks `import mlx_video.generate_av` to distinguish them;
installPackages gained a pre-uninstall step driven by
PIP_PRE_UNINSTALL so the conflicting plain `mlx_video` is removed
before pip refuses to "downgrade" across the namespace collision.
Mirrors what scripts/setup-image-video.sh already did out-of-band.
Separately:
* The media job worker now re-resolves pythonPath from live settings at
dispatch time for every local-Python job, instead of using the
snapshot captured at enqueue time. Symptom this fixes: user switches
their Python in the UI, clicks Generate, and the worker still shells
out to the previous interpreter — because the in-memory queue and
data/media-jobs.json carried the old snapshot. Codex image jobs are
unaffected (they don't run a local Python).
* probePythonHealth() consolidates the three subprocesses /setup/check
used to fan out (checkPackages + isExternallyManaged + probePythonArch)
into one python invocation that returns everything as JSON.
isExternallyManaged becomes a thin wrapper; the standalone export was
unused after the consolidation and is removed.
* installPackages and installFlux2Venv shared a near-identical
spawn-and-stream-stdout helper; extracted to a single streamSpawn().
Tests: 8061 server + 602 client passing (incl. new coverage for
mediaJobQueue's live-pythonPath re-resolution and videoGen /status's
missing-packages branch). Test count grew by 4 since main.
…warning to darwin/arm64 * LocalSetupPanel: the inline path input fired onPythonPathChange on every keystroke. In the VideoGen consumer that meant a settings PATCH + a ~1-2s status re-probe per character. Decouple via local draft state; commit on 800ms debounce or onBlur. Programmatic updates (Detect, Switch-to-arm64, Create-venv) still call onPythonPathChange directly so they take effect immediately. * /api/image-gen/setup/check: generic interpreterArch !== HOST_ARCH would false-positive on Windows (Python reports `AMD64` vs Node's `x86_64`) and on hypothetical arm64 Linux where mlx isn't even in REQUIRED_PACKAGES. Gate the warning to darwin/arm64 with an x86_64 interpreter — the only configuration where mlx actually breaks. The third codex finding (hfDownload.js falls back to broken FLUX.2 venv) is in a file not yet on origin and not part of this PR's surface; left for whichever branch ships hfDownload.
atomantic
force-pushed
the
fix-local-python-setup
branch
from
af3c2a2 to
e076590
Compare
The gemini /do:review pass reviewed the whole repo instead of the branch diff (a known issue tracked under [triage-gemini-out-of-scope-findings]). All 6 findings were in files PR #515 doesn't touch. * Findings 5+6 (Express body limit, * CORS): explicitly intentional per CLAUDE.md Security Model — single-user private-network deployment. * Findings 1-4 (commands.js naive split, fileUtils non-atomic write, two cosAgents non-atomic writes): real but pre-existing, in files unrelated to this PR's surface. Appended to PLAN.md as a new triage entry so they're not silently dropped. Verify each finding's reproducibility before acting; gemini's /do:review has a track record of pattern-matching false-positives.
Contributor
There was a problem hiding this comment.
Pull request overview
Fixes the end-to-end Apple Silicon local-Python flow for video generation by surfacing setup inline, correcting arch-aware Python detection, fixing PEP 668 venv mis-detection, making /video-gen/status truthful about missing packages, and resolving the mlx_video vs mlx-video-with-audio package-name collision. Also re-resolves pythonPath from live settings at media-job dispatch time so a stale snapshot can't poison the spawn.
Changes:
- Server: arch-aware
detectPython+ newprobePythonHealth(consolidates 3 subprocesses, fixes venv externally-managed detection),mlx_videoimport probe + pre-uninstall,/video-gen/statusprobes imports, media-job worker re-resolves live pythonPath. - Client: inline
LocalSetupPanelon VideoGen with debounced path commit, arch-mismatch warning + one-click switch,onPackagesChangednotify on transition to all-installed. - Tests: regression coverage for
/video-gen/statusmissing-packages branch and live-pythonPath re-resolution at dispatch.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| server/lib/pythonSetup.js | Adds HOST_ARCH, arch probes, arm64-preferring detectPython, consolidated probePythonHealth (handles venv stdlib quirk), mlx_video deeper-import probe, PIP_PRE_UNINSTALL, shared streamSpawn. |
| server/routes/imageGen.js | /setup/check now uses probePythonHealth and returns interpreterArch, hostArch, archMismatch, suggestedArm64Python. |
| server/routes/videoGen.js | /status now probes packages via checkPackages and returns connected, reason, missingPackages. |
| server/routes/videoGen.test.js | Adds regression test for missing-packages branch of /status. |
| server/services/mediaJobQueue/index.js | Re-resolves pythonPath from live settings for video and non-codex image jobs at dispatch. |
| server/services/mediaJobQueue/index.test.js | Adds tests for live-pythonPath re-resolution and codex params.pythonPath left untouched. |
| client/src/pages/VideoGen.jsx | Inline LocalSetupPanel below status pill; handleSavePythonPath does the imageGen-slice round-trip and re-polls status. |
| client/src/components/settings/LocalSetupPanel.jsx | Adds debounced draft input, onPackagesChanged transition notify, arch-mismatch banner + Switch-to-arm64 button. |
| .changelog/NEXT.md | Changelog entries for the 6 user-visible fixes. |
| PLAN.md | New deferred items (helper extractions, cache, hook extraction, arch tests) and gemini PR #515 triage entry. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This was referenced May 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
End-to-end fix for the "configure local Python → install packages → generate a video" flow on Apple Silicon. The previous flow hit five distinct walls in sequence; each one looked like the user's fault but was a real bug.
What was broken
Detect/install was buried in the Settings drawer. The VideoGen "Local Python not configured" warning only linked to Settings.
LocalSetupPanelnow renders inline below the disconnected status pill so detect / install / Create-venv all happen in place; saving a newpythonPathre-polls status so the pill flips green automatically.detectPython()preferred Intel Anaconda over Homebrew on Apple Silicon./opt/anaconda3/bin/python3(often x86_64) won over/opt/homebrew/bin/python3(arm64), and the install then failed withNo matching distribution found for mlxbecausemlxships arm64-only wheels. The detector now probesplatform.machine()per candidate ondarwin/arm64and prefers a matching interpreter;/api/image-gen/setup/checkreturnsinterpreterArch+suggestedArm64PythonsoLocalSetupPanelcan warn and offer a one-click "Switch to detected arm64 Python" button when the saved path is the wrong arch.PortOS-owned venvs from PEP 668 base Python were misreported as externally managed. Inside a venv,
sysconfig.get_path("stdlib")resolves to the base interpreter's stdlib, so a venv from Homebrew Python inherited Homebrew'sEXTERNALLY-MANAGEDmarker. The check now also verifiessys.prefix != sys.base_prefixand short-circuits tofalsefor venvs./api/video-gen/statuslied about readiness. It wasconnected: !!pythonPath, so the green pill appeared the moment any path was saved — even with zero required packages installed. The user only saw the failure when they hit Generate./statusnow probes imports viacheckPackagesand returnsconnected: false+ amissingPackageslist when anything's absent.LocalSetupPanelalso notifies the parent on local check transition to "all installed" viaonPackagesChanged, so the pill flips green without a manual refresh.The PyPI package literally named
mlx_videois unrelated to themlx_video.generate_avCLI the LTX renderer shells into — that CLI ships inmlx-video-with-audio. Both publish animport mlx_videonamespace, so the missing-check passed against the wrong package and the spawn died withModuleNotFoundError.pipNameFor('mlx_video')now returnsmlx-video-with-audio>=0.1.35on macOS; the import probe checksimport mlx_video.generate_avto distinguish them;installPackagesgained a pre-uninstall step driven byPIP_PRE_UNINSTALLso the conflicting plainmlx_videois removed before pip refuses to "downgrade" across the namespace collision. Mirrors the same conflict resolutionscripts/setup-image-video.shalready did out-of-band.Separately
pythonPathfrom live settings at dispatch time for every local-Python job (video + non-codex image), instead of using the snapshot captured at enqueue time. Symptom this fixes: user switches their Python in the UI, clicks Generate, and the worker still shells out to the previous interpreter — because the in-memory queue anddata/media-jobs.jsoncarried the old snapshot. Codex image jobs are unaffected.probePythonHealth()consolidates the three subprocesses/setup/checkused to fan out (checkPackages+isExternallyManaged+probePythonArch) into one Python invocation that returns everything as JSON. The standaloneisExternallyManagedexport became unused after the consolidation and was removed.installPackagesandinstallFlux2Venvshared a near-identical spawn-and-stream-stdout helper; extracted to a singlestreamSpawn().Test plan
server/routes/videoGen.test.jsadds a regression test for the "pythonPath set but packages missing → connected:false + missingPackages list" branch.server/services/mediaJobQueue/index.test.jsadds two tests — video job uses livepythonPathnot the snapshot, codex image job's params stay untouched./video-genand confirm the inline panel auto-selects the arm64 Homebrew Python.pip uninstall -y mlx_video (resolving package-name conflict)) and the pill flips green when install completes.pythonPathwhile a video job is queued; confirm the worker log showspythonPath re-resolved from settings: {old} → {new}and uses the new interpreter.