Commit 336ed90
feat: Multiprocessing support in packaged desktop apps (#6662)
* feat(build): macOS runner intercepts multiprocessing child re-execs
Python's multiprocessing (spawn/forkserver start methods and the resource
tracker) launches child processes by re-executing sys.executable with a
CPython command line — and in a flet-built app sys.executable is the app
binary itself. Each worker therefore booted a full Flutter GUI instance
that treated '-B' as a dev-mode page URL, never ran the spawn payload,
and never exited: one stray window per worker, hung pools, and
"resource_tracker: process died unexpectedly, relaunching" loops.
Add macos/Runner/main.swift as the explicit entry point (AppDelegate
drops @main; the xib still instantiates and wires the delegate): before
NSApplicationMain, it checks argv against dart_bridge >= 1.5.0's
serious_python_is_mp_invocation and, on a match, exits with
serious_python_main's return code — the embedded interpreter services
the child headlessly (Py_BytesMain) with no AppKit/Flutter
initialization. The child resolves the bundled stdlib/site-packages via
the PYTHONHOME/PYTHONPATH the parent already setenv'd process-wide.
Both entry points are resolved via dlsym from the current process image
(dart_bridge is a static archive force-loaded into the host binary), so
apps built against an older dart_bridge still link and launch — they
just don't get the interception.
Verified end-to-end on macOS with an instrumented probe app:
Process+Queue round-trip with a main.py-defined worker, and a
ProcessPoolExecutor run (4 workers, reused across 8 tasks) — headless
children, correct results, ~4x speedup over sequential, no leftover
processes.
* feat(build): Windows runner intercepts multiprocessing child re-execs
Same mechanism as the macOS runner: multiprocessing spawn children (and
the resource tracker) re-execute sys.executable — the app .exe — with a
CPython command line, which previously booted one GUI window per worker
in dev-mode and never ran the spawn payload.
At the top of wWinMain, before any console/COM/window/Flutter work,
MaybeRunPythonChild() parses the wide command line
(CommandLineToArgvW), loads dart_bridge.dll (falling back to
dart_bridge_d.dll for debug-CRT builds), and resolves the wide-char
entry points serious_python_is_mp_invocation_w / serious_python_main_w
(dart_bridge >= 1.5.0; wide variants because decoding wWinMain argv
through the ANSI code page would be lossy — serious_python_main_w wraps
Py_Main). On a match, the process runs as a headless interpreter and
returns its exit code.
Resolution is dynamic (GetProcAddress with graceful fallthrough), so
apps built against an older dart_bridge launch unchanged. Eagerly
loading the DLL costs nothing on the normal startup path — the plugin
loads it moments later anyway.
Note: not yet runtime-verified on Windows (the macOS equivalent is
verified end-to-end); needs a probe run on a Windows machine.
* feat(build): Linux runner intercepts multiprocessing child re-execs
Same mechanism as the macOS/Windows runners, with an extra reason to
care on Linux: Python 3.14 changed the default start method from fork
to forkserver (gh-84559), and the forkserver's server process is
launched through the same sys.executable re-exec as spawn — so Linux
apps that previously happened to work via raw fork break by default on
the bundled 3.14.
At the top of main(), before any GTK/Flutter initialization,
maybe_run_python_child() dlopen()s libdart_bridge.so — resolved through
the runner's $ORIGIN/lib RUNPATH, exactly the way the Dart FFI's
DynamicLibrary.open() finds it later — and resolves
serious_python_is_mp_invocation / serious_python_main (dart_bridge >=
1.5.0). On a match, the process runs as a headless interpreter and
returns its exit code.
Resolution is dynamic with graceful fallthrough, so apps built against
an older dart_bridge launch unchanged. ${CMAKE_DL_LIBS} is added to the
runner link for dlopen/dlsym (a no-op with glibc >= 2.34, where libdl
is merged into libc).
Note: not yet runtime-verified on Linux (the macOS equivalent is
verified end-to-end); needs a probe run on a Linux machine.
* fix(build): run the app module as the real __main__; point multiprocessing at the host binary
Two independent multiprocessing breakages lived in the boot script, and
either alone was fatal even with the runners' child interception in
place:
1. __main__ identity. runpy.run_module(run_name="__main__") executes the
app module in a scratch namespace that is never installed in
sys.modules, so pickling any function defined in the app module
failed IN THE PARENT with "Can't pickle <fn>: it's not found as
__main__.<name>" — before a child was ever spawned. Replace it with
_sp_run_module_as_main(): resolve the module spec (with `python -m`
package semantics — pkg runs pkg.__main__ — and clear ImportErrors
for loaderless/codeless specs), build a fresh module with
__spec__/__file__/__cached__/__loader__/__package__ set, install it
as sys.modules["__main__"], and exec the module code in its dict.
Spawn children then re-import the app module as __mp_main__ via the
standard init_main_from_name path — which is why the documented
`if __name__ == "__main__":` guard around ft.run() is now mandatory
for multiprocessing users, exactly as in plain CPython.
The module is also aliased as sys.modules["__mp_main__"] in the
parent, matching what multiprocessing does in children: objects
defined in the app module and pickled by a child carry
__module__ == "__mp_main__", and without the alias the parent fails
to unpickle them (ModuleNotFoundError: __mp_main__).
2. Re-exec target. sys.executable / sys._base_executable are set to the
host binary (new {host_executable} placeholder, filled from
Platform.resolvedExecutable in native_runtime.dart; JSON string
literals are valid Python string literals). On macOS/Windows CPython
already computes the host binary itself, but on Linux bare
Py_Initialize() PATH-guesses an unrelated "python3" (or none) — this
makes the target the runner binary, whose argv interception services
the children, on all three desktops deterministically. Set before
any user import because multiprocessing snapshots sys.executable at
import time.
PYTHONINSPECT is also dropped from the inherited environment: it did
nothing in the embedded parent, but a real interpreter child would
stay open in interactive mode after its -c command completed.
(Defense in depth — serious_python >= 4.3.0 stops setting it and
dart_bridge's serious_python_main unsets it too.)
Verified end-to-end on macOS together with the runner interception; the
rendered boot script is byte-checked to compile after all placeholder
substitutions.
* feat(create): guard ft.run() with __main__ in the app scaffold
Start every new project with the standard entry-point guard:
if __name__ == "__main__":
ft.run(main)
With multiprocessing now working in packaged desktop apps, spawn
children re-import the app's main module (as __mp_main__) exactly like
plain CPython — an unguarded ft.run() would start a whole new app
session in every worker process. The guard has always been Python best
practice; the scaffold now models it so multiprocessing users don't
learn it the hard way.
Also type the scaffold's event handler parameter
(e: ft.Event[ft.FloatingActionButton]) to model typed event handlers.
* refactor(build): splice all dynamic boot-script values via jsonEncode
JSON string/array literals are valid Python literals, so jsonEncode gives
correct escaping for free. The previous hand-rolled escaping was uneven:
{outLogFilename} doubled backslashes but not quotes, and {argv} escaped
quotes but not backslashes — a Windows-style path in either would corrupt
the generated Python source. {host_executable} already used jsonEncode;
now all three share the one convention. Empty argv still renders as [""]
(CPython always has a sys.argv[0]).
Verified: boot script rendered through the real Dart substitution with
hostile inputs (backslashes, quotes, non-ASCII) compiles as valid Python
and every value round-trips exactly; full flet build macos + the
multiprocessing probe suite passes unchanged.
* docs: Multiprocessing cookbook recipe
New cookbook page documenting multiprocessing support in Flet apps, now
that packaged desktop apps service the spawn re-exec protocol:
- when to reach for processes vs async/threads, and the platform/version
support matrix (desktop-only, Flet >= 0.86.0; iOS/Android/browser
unsupported);
- the rules that are standard Python multiprocessing discipline but
MANDATORY in packaged apps: the `if __name__ == "__main__":` guard
(spawn/forkserver children re-import the main module), importable and
picklable worker functions (top-level, plain data, no controls/page/
lambdas), and no GUI access from workers;
- how it works in a `flet build` app: the embedded interpreter, the app
binary recognizing CPython helper command lines and running them as a
headless interpreter, and the practical consequences — sys.executable
points at the app binary by design (don't set_executable() over it),
freeze_support() is unnecessary-but-harmless, worker stdout isn't the
app console log, and forcing fork on Linux is unsafe under a running
Flutter engine;
- a runnable ProcessPoolExecutor example (parallel chunk sorting with
live progress) driven off the UI thread via page.run_thread.
Listed in the cookbook sidebar after Subprocess.
* add changelog entry
* fix(build): open console.log as UTF-8
The boot script opened the console-log tee file without an explicit
encoding, so Windows used the locale codepage (e.g. cp1252) and the
first non-ASCII character an app printed raised UnicodeEncodeError
inside the stdout tee — crashing the printing thread. macOS/Linux never
hit it because their default filesystem encoding is UTF-8 already.
Found while verifying the multiprocessing fix on a Windows VM: the demo
app's status line contains "→" and its print() died mid-benchmark.
errors="backslashreplace" additionally guarantees that even malformed
data (e.g. lone surrogates) degrades to an escaped representation
instead of ever breaking the log.
* fix: hide the console window of the git version-fallback on Windows
flet/version.py resolves the package version at import time; in a
source checkout (empty baked flet_version) it falls back to
`git describe`. On Windows, git.exe is a console-subsystem program, so
spawning it from a windowless GUI process pops a visible conhost window.
In a dev-built app this meant one console flash at app start — and,
with multiprocessing now working, one more flash per spawned
worker, since every spawn child re-imports flet and re-runs the
fallback. Traced on a Windows VM via Win32_ProcessStartTrace: each
worker pid parented a git.exe (+ conhost.exe) at click time.
Pass CREATE_NO_WINDOW on Windows so the fallback stays invisible.
Released packages are unaffected either way (CI bakes flet_version, so
the git path never runs).
* add cookbook recipe and examples
* docs: add detailed usage guides to CLI command docstrings
Enhance docstrings for `debug`, `test`, `create`, `build`, `run`, and `publish` CLI commands with links to detailed usage guides and examples from the Flet documentation.
* docs: update README of build-template with installation, platform support, and usage guidance
* breaking changes in version folders
* update changelog
* fix(build): splice {module_name} into the boot script via jsonEncode too
The jsonEncode refactor's comment promised "every dynamic value is
spliced into the boot script through jsonEncode", but {module_name} was
still inserted raw inside hand-written quotes. No live bug — the module
name must already survive the cookiecutter render and find_spec(), so
hostile values can't reach it — but the exception made the comment
wrong and left a footgun for future entry-point changes (e.g. dotted or
path-like module names).
Encode moduleName like the other values; the Python template now calls
_sp_run_module_as_main({module_name}) without surrounding quotes, since
the JSON string literal brings its own.
Addresses the review note on #6662 (discussion_r3544535152). The
rendered boot script is byte-checked to compile after all substitutions.
* docs: drop leftover debug prints from the persistent-worker example
calc_worker still carried two print() calls from local testing. They
would also be misleading in a packaged app, where worker stdout isn't
connected to the app's console log — which the cookbook page itself
points out.
* docs: fix relative links broken by the breaking-changes version folders
Moving the v0.85/v0.86 breaking-change pages into v0-85-0/ and v0-86-0/
subfolders (876a006) was a pure rename, so every relative link inside
them started resolving one directory too shallow and the site build
failed its broken-links check ("Docusaurus found broken links!"):
- "](.)" pointed at the (non-existent) version-folder index instead of
the breaking-changes index -> now ../index.md
- ../release-notes.md -> ../../release-notes.md
- ../../{cli,reference,cookbook}/... -> ../../../{cli,reference,cookbook}/...
All nine moved pages updated; every relative .md link target verified to
exist, and a full local `docusaurus build` passes the broken-links check
again.
* Bump python build release and serious_python
Update the pinned python-build release date to 20260708 and raise the build template's `serious_python` dependency to 4.3.0 to keep the Python packaging stack in sync.
---------
Co-authored-by: Feodor Fitsner <feodor@appveyor.com>1 parent 889b5e0 commit 336ed90
37 files changed
Lines changed: 772 additions & 80 deletions
File tree
- sdk/python
- examples/cookbook/multiprocessing
- packages
- flet-cli/src/flet_cli
- commands
- utils
- flet/src/flet
- templates
- app
- app/{{cookiecutter.out_dir}}/src
- extension/{{cookiecutter.out_dir}}
- build/{{cookiecutter.out_dir}}
- lib
- linux
- macos
- Runner.xcodeproj
- Runner
- windows/runner
- website
- docs
- cookbook
- extend
- updates/breaking-changes
- v0-85-0
- v0-86-0
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
| 5 | + | |
5 | 6 | | |
6 | 7 | | |
7 | 8 | | |
| |||
24 | 25 | | |
25 | 26 | | |
26 | 27 | | |
27 | | - | |
| 28 | + | |
28 | 29 | | |
29 | 30 | | |
30 | | - | |
31 | | - | |
| 31 | + | |
| 32 | + | |
32 | 33 | | |
33 | 34 | | |
34 | 35 | | |
| |||
Lines changed: 69 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
Lines changed: 46 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
Lines changed: 65 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
Lines changed: 58 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
20 | | - | |
| 20 | + | |
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
| |||
0 commit comments