Skip to content

fix: server startup on Windows + missing runtime deps in server.txt#391

Merged
pancacake merged 1 commit intoHKUDS:devfrom
jonathanzhan1975:fix/server-deps-and-utf8-encoding
Apr 27, 2026
Merged

fix: server startup on Windows + missing runtime deps in server.txt#391
pancacake merged 1 commit intoHKUDS:devfrom
jonathanzhan1975:fix/server-deps-and-utf8-encoding

Conversation

@jonathanzhan1975
Copy link
Copy Markdown
Contributor

Two independent issues that block python scripts/start_web.py after a clean install on Windows. Both block users from reaching the API server, so I'm bundling them into a single PR.


Issue 1 — UnicodeDecodeError streaming subprocess output

_spawn() opens the backend / frontend stdout pipes with text=True but no explicit encoding. On Windows, text=True falls back to the system ANSI codepage (e.g. gbk on Chinese Windows). The reader thread crashes the moment a child process emits a UTF-8 byte:

File ""E:\works\study\DeepTutor\scripts\start_web.py"", line 40, in _stream_output
    for line in process.stdout:
UnicodeDecodeError: 'gbk' codec can't decode byte 0xb2 in position 2: illegal multibyte sequence

Fix

Pin encoding=""utf-8"" and errors=""replace"" on the Popen kwargs. Both Node 24 (Next.js dev) and the FastAPI/uvicorn backend write UTF-8 to stdout, so this matches the actual byte stream and is locale-independent.

kwargs: dict[str, object] = {
    ""cwd"": str(cwd),
    ""env"": env,
    ""stdout"": subprocess.PIPE,
    ""stderr"": subprocess.STDOUT,
    ""text"": True,
    ""bufsize"": 1,
+   ""encoding"": ""utf-8"",
+   ""errors"": ""replace"",
}

Issue 2 — ModuleNotFoundError: loguru / json_repair after pip install -r requirements/server.txt

deeptutor/services/llm/provider_core/__init__.py unconditionally imports:

  • AnthropicProviderimport json_repair
  • OpenAICompatProviderimport json_repair
  • OpenAICodexProviderfrom loguru import logger
  • provider_core/base.pyfrom loguru import logger
  • openai_responses/parsing.py → both

Both packages are declared only in requirements/tutorbot.txt. So the documented install path for the web server:

pip install -r requirements/server.txt && pip install -e .
python scripts/start_web.py

crashes the uvicorn worker before it can serve a single request:

File ""...\deeptutor\services\llm\provider_core\anthropic_provider.py"", line 16, in <module>
    import json_repair
ModuleNotFoundError: No module named 'json_repair'

Fix

Add both packages to requirements/server.txt with the same version bounds already used in tutorbot.txt:

loguru>=0.7.3,<1.0.0
json-repair>=0.57.0,<1.0.0

These are core LLM-provider dependencies, not TutorBot-specific, so the right home is server.txt.


Reproduction

git clone https://github.com/HKUDS/DeepTutor
cd DeepTutor
python -m venv .venv
.venv\Scripts\activate
pip install -r requirements/server.txt
pip install -e .
python scripts/start_web.py

Verification

Tested on Windows 11 + Python 3.12.1 + a fresh .venv. After applying both fixes, python scripts/start_web.py boots cleanly:

  • backend: Uvicorn running on http://0.0.0.0:8001
  • frontend: Next.js Ready in <1s

Frontend page loads and successfully reaches the backend /api/... routes.

Two independent issues that block `python scripts/start_web.py` after a
clean install on Windows:

1. UnicodeDecodeError when streaming subprocess output

   `_spawn()` opened the backend / frontend stdout pipe with `text=True`
   but no explicit encoding. On Windows this defaults to the system
   ANSI codepage (e.g. `gbk` on Chinese Windows), which raises
   `UnicodeDecodeError` as soon as the child process emits a UTF-8 byte.

   Fix: pin `encoding="utf-8"` and `errors="replace"` so any locale
   produces clean, lossless output.

2. Missing runtime dependencies in requirements/server.txt

   `deeptutor/services/llm/provider_core/__init__.py` unconditionally
   imports `AnthropicProvider`, `OpenAICompatProvider`, and
   `OpenAIResponses`, which in turn import:

     - `loguru`        (provider_core/base.py, openai_codex_provider.py,
                        openai_responses/parsing.py)
     - `json_repair`   (anthropic_provider.py, openai_compat_provider.py,
                        openai_responses/parsing.py)

   These are declared only in `requirements/tutorbot.txt`, so
   `pip install -r requirements/server.txt && pip install -e .` followed
   by `python scripts/start_web.py` crashes with `ModuleNotFoundError`
   before the API server can serve a single request.

   Fix: declare both packages in `server.txt` with the same version
   bounds used by `tutorbot.txt`.

Reproduced and verified on Windows 11 + Python 3.12 + a fresh venv.
@pancacake pancacake changed the base branch from main to dev April 27, 2026 02:32
@pancacake pancacake merged commit 8594b46 into HKUDS:dev Apr 27, 2026
4 checks passed
@pancacake
Copy link
Copy Markdown
Collaborator

Thanks!! This is a good one

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Question]: I am looking to work on TODO tasks but.

2 participants