fix(qwen3-tts-cpp): hold qwentts.cpp at 35ebe537, upstream master hangs in synthesis - #11286
Conversation
…gs in synthesis
tests-qwen3-tts-cpp has been failing on master since 2026-07-31. The suite
loads every component fine and then stops: TTS() never returns from the
native call, so a job that takes ~5 minutes runs into the 20 minute Go test
timeout instead.
goroutine 74 [syscall, 19 minutes]:
github.com/ebitengine/purego.RegisterFunc.func4
qwen3-tts-cpp.(*Qwen3TtsCpp).TTS goqwen3ttscpp.go:154
qwen3-tts-cpp.init.func2.4 e2e_test.go:90
Not a flake: reproduced on master and again on an explicit re-run.
Bisected across this cycle's eight qwentts.cpp bumps by their own check:
10832, 10850, 10902, 10964, 11006, 11039 and 11127 all pass in ~5 minutes;
11241 (abab6b3) fails at 1h58m. That PR was merged with this check already
red, which is how the hang reached master.
35ebe537..abab6b3 is three upstream commits, and the only functional one is
26dd8adb, "predictor: unroll the frame into one cgraph and sample in standard
ops", which is consistent with a generation loop that never reaches its stop
condition.
Hold the pin at the last known-good commit. The bump entry is commented out
rather than left in place, because it tracks upstream master and would put
the hang straight back on the next nightly run. Both spots carry a pointer to
the other so the hold is discoverable, and restoring it is uncommenting four
lines once upstream is fixed.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash]
|
Reported upstream: ServeurpersoCom/qwentts.cpp#19 CI has now confirmed the hold works: Restoring the bump is: revert the pin, and uncomment the four |
|
Correcting the mechanism described in the commit message, now that the upstream fix is understood. This commit (and issue #19 as originally filed) says Generation runs to the 2048-frame cap instead of stopping at EOS around frame 17-28, so each synthesis does roughly 100x the work and the suite blows its 20 minute budget. The call does terminate, just far too late to be useful. Root cause upstream, from ServeurpersoCom/qwentts.cpp#20: None of this changes what this PR does or why. Holding the pin at |
2 participants
Problem
tests-qwen3-tts-cpphas been red on master since 2026-07-31. The suite loads every component successfully and then stops:TTS()never returns from the native call:A job that normally takes about 5 minutes runs into the 20 minute Go test timeout instead.
Not a flake. Reproduced on master (run 30689716806) and again on an explicit re-run of the same job.
Which bump did it
Bisected across this cycle's eight qwentts.cpp bumps using each PR's own
tests-qwen3-tts-cppcheck:tests-qwen3-tts-cpp7bb918869e11ce4195b4840ae93292beba4c7f7882cd05b935ebe537abab6b3#11241 was merged with this check already red, which is how the hang reached master.
Upstream cause
35ebe537..abab6b3is three upstream commits:26dd8adbpredictor: unroll the frame into one cgraph and sample in standard ops9a9a4252tests: exercise the fused decode in every cossim harnessabab6b3btests: refresh the clone grid logsOnly
26dd8adbis functional; the other two are test-only. Rewriting the predictor's sampling into standard ops is consistent with a generation loop that never reaches its stop condition, which is exactly what an unbounded native call looks like from the Go side.Fix
Hold the pin at
35ebe537, the last known-good commit.The
bump_depsentry is commented out rather than left in place: it tracks upstreammaster, so the nightly job would put the pin straight back onto the hang. Both the Makefile and the workflow carry a comment pointing at the other, so the hold is discoverable rather than mysterious, and restoring it is uncommenting four lines once upstream is fixed.Why this blocks the release
This is not only a CI problem.
TTS()hanging means a realqwen3-tts-cppsynthesis request never returns, so the backend wedges for users on the current master. Found during the pre-release CI sweep for v4.8.0.Follow-up (not in this PR)
Report
26dd8adbupstream to ServeurpersoCom/qwentts.cpp with this bisection and stack trace, then restore the bump once it is fixed.