Skip to content

Commit 2a7d905

Browse files
ayhammoudaclaude
andcommitted
fix(ingestion): serial Sphinx build on Python 3.14+ (forkserver pickling)
Python 3.14 changed the default multiprocessing start method on POSIX from fork to forkserver. forkserver pickles the work sent to child processes, but Sphinx's parallel build (-j auto) passes a ParallelTasks object containing an unpicklable local closure (Builder._read_parallel.<locals>.merge), so build-index failed on 3.14 with _pickle.PicklingError. Caught by the Slow E2E 3.14 job; 3.13 (still fork) was unaffected. Add sphinx_parallel_jobs(version_info) -> '1' on >=3.14 else 'auto', and use it in build_sphinx_json_command. The Sphinx venv is created from the current interpreter, so sys.version_info is the correct signal. Server runtime is unaffected; only the offline index build changes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 8be635a commit 2a7d905

3 files changed

Lines changed: 46 additions & 2 deletions

File tree

CHANGELOG.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,18 @@ All notable changes to `python-docs-mcp-server` are documented here.
44
Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/);
55
this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
66

7+
## [Unreleased]
8+
9+
### Fixed
10+
11+
- **`build-index` on Python 3.14** — Sphinx's parallel JSON build (`-j auto`)
12+
raised `_pickle.PicklingError` on Python 3.14, which flipped the default
13+
`multiprocessing` start method on POSIX from `fork` to `forkserver`
14+
(`forkserver` must pickle worker tasks, but Sphinx's `ParallelTasks` holds an
15+
unpicklable local closure). The builder now falls back to a serial build
16+
(`-j 1`) on 3.14+ while keeping the parallel path on `fork`-default
17+
interpreters (3.13 and earlier). Server runtime behavior is unchanged.
18+
719
## [0.2.0] — 2026-05-29
820

921
### Added

src/mcp_server_python_docs/ingestion/sphinx_json.py

Lines changed: 21 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@
1313
import os
1414
import re
1515
import sqlite3
16+
import sys
1617
from collections.abc import Mapping
1718
from pathlib import Path
1819

@@ -166,6 +167,25 @@ def make_sphinx_json_env(
166167
return env
167168

168169

170+
def sphinx_parallel_jobs(
171+
version_info: tuple[int, ...] = (sys.version_info.major, sys.version_info.minor),
172+
) -> str:
173+
"""Return the Sphinx ``-j`` value for the build interpreter.
174+
175+
The Sphinx venv is created from the current interpreter (``venv.create``),
176+
so ``sys.version_info`` reflects the Python that will run ``sphinx-build``.
177+
178+
Python 3.14 changed the default ``multiprocessing`` start method on
179+
POSIX from ``fork`` to ``forkserver``. ``forkserver`` pickles the work
180+
handed to child processes, but Sphinx's parallel build passes a
181+
``ParallelTasks`` object holding a local closure
182+
(``Builder._read_parallel.<locals>.merge``) that cannot be pickled, so
183+
``-j auto`` raises ``_pickle.PicklingError`` on 3.14+. Fall back to a
184+
serial build there; ``fork``-default interpreters keep the parallel path.
185+
"""
186+
return "1" if version_info >= (3, 14) else "auto"
187+
188+
169189
def build_sphinx_json_command(
170190
sphinx_build: Path | str,
171191
doc_dir: Path | str,
@@ -179,7 +199,7 @@ def build_sphinx_json_command(
179199
"-D",
180200
"html_theme=classic",
181201
"-j",
182-
"auto",
202+
sphinx_parallel_jobs(),
183203
str(doc_dir),
184204
str(json_out),
185205
]

tests/test_ingestion.py

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@
3434
parse_fjson,
3535
populate_synonyms,
3636
rebuild_fts_indexes,
37+
sphinx_parallel_jobs,
3738
write_json_build_requirements,
3839
write_sphinx_json_sitecustomize,
3940
)
@@ -227,11 +228,22 @@ def test_build_command_uses_json_builder_and_classic_theme(self, tmp_path):
227228
"-D",
228229
"html_theme=classic",
229230
"-j",
230-
"auto",
231+
sphinx_parallel_jobs(),
231232
str(doc_dir),
232233
str(json_out),
233234
]
234235

236+
def test_sphinx_parallel_jobs_auto_before_314(self):
237+
"""Pre-3.14 interpreters keep the parallel 'auto' build (fork default)."""
238+
assert sphinx_parallel_jobs((3, 10, 0)) == "auto"
239+
assert sphinx_parallel_jobs((3, 13, 5)) == "auto"
240+
241+
def test_sphinx_parallel_jobs_serial_on_314_plus(self):
242+
"""3.14+ flipped multiprocessing to forkserver, which can't pickle
243+
Sphinx's parallel ParallelTasks closures -> force a serial build."""
244+
assert sphinx_parallel_jobs((3, 14, 0)) == "1"
245+
assert sphinx_parallel_jobs((3, 15, 1)) == "1"
246+
235247

236248
# ── fjson parsing tests (INGR-C-04) ──
237249

0 commit comments

Comments
 (0)