Skip to content

silence_msvc: UnicodeEncodeError when writing msvc_capture.log on non-UTF-8 consoles (e.g. cp932) #2010

Description

@shiena

Godot version

Godot v4.8.dev (549d1846b) - Windows 11 (build 26200) - Multi-window, 2 monitors - OpenGL 3 (Compatibility) - NVIDIA GeForce RTX 5070 (NVIDIA; 32.0.16.1047) - AMD Ryzen 9 9900X 12-Core Processor (24 threads) - 61.66 GiB memory - WASAPI (48000 Hz, Stereo/mono)

godot-cpp version

ba0edfe

System information

Windows 11, Japanese locale (console encoding cp932)

Issue description

silence_msvc creates msvc_capture.log with UTF-8 encoding, but appends
captured lines to it using sys.stdout.encoding:

https://github.com/godotengine/godot-cpp/blob/master/tools/windows.py#L17
https://github.com/godotengine/godot-cpp/blob/master/tools/windows.py#L60

This causes two problems on consoles whose encoding is not UTF-8 (e.g. cp932 on Japanese Windows):

  1. The log file ends up with mixed encodings (created as UTF-8, appended as cp932).
  2. The compiler output is read with errors="replace", which can introduce U+FFFD replacement characters. cp932 cannot encode U+FFFD, so the append raises UnicodeEncodeError. Only OSError is caught, so the exception propagates out of the SPAWN hook and aborts the build instead of falling back to the intended warning message.

Steps to reproduce

  1. On Windows with a non-UTF-8 console encoding (e.g. Japanese Windows, cp932), build with scons silence_msvc=yes.
  2. Have cl emit output that is not valid in the console encoding (e.g. a UTF-8 file name that cp932 cannot decode), so the errors="replace" read inserts U+FFFD into a captured line.
  3. Writing that line to msvc_capture.log raises UnicodeEncodeError and the build fails.

Minimal reproduction project

N/A

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions