Skip to content

Commit dec8471

Browse files
ehsan6shaclaude
andcommitted
runtime: hold ctypes c_char_p byte buffers in Python locals to prevent GC
Lab-observed bug 2026-05-27: model emitted Chinese tokens + random text fragments after the role-based v1.2.3 refactor. Root cause was a ctypes lifetime issue — ctypes c_char_p does NOT own the bytes it points at, so temporaries like role.encode('utf-8') get garbage- collected before the C side (rkllm_run) finishes reading them. The pre-refactor code only had ONE c_char_p assignment per call (input_data.prompt_input), which Python happened to keep alive in the inp struct via reference. The refactor added a second (inp.role = role.encode('utf-8')) — both temporaries became collectable and the C side saw uninitialized memory. Fix: hold both encoded bytes objects in NAMED Python locals so they stay reachable for the full duration of the rkllm_run call. Mirrors the pattern in Rockchip's reference Python example which wraps with ctypes.c_char_p(...) for the same reason. 55/55 tests pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
1 parent 00bf346 commit dec8471

1 file changed

Lines changed: 11 additions & 2 deletions

File tree

src/runtime/rkllm_runtime.py

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -470,12 +470,21 @@ def generate(
470470
except queue.Empty:
471471
break
472472

473+
# CRITICAL: ctypes c_char_p does NOT own the bytes — if the
474+
# Python bytes object is garbage-collected before the C call
475+
# finishes reading it, the C side sees uninitialized memory.
476+
# This caused the lab-observed gibberish-output bug 2026-05-27
477+
# (model emitted "用户" + random text fragments). The
478+
# encoded bytes objects MUST be held in named Python locals
479+
# for the duration of the rkllm_run call.
480+
role_bytes = role.encode("utf-8")
481+
prompt_bytes = prompt.encode("utf-8")
473482
inp = RKLLMInput()
474483
ctypes.memset(ctypes.byref(inp), 0, ctypes.sizeof(inp))
475-
inp.role = role.encode("utf-8")
484+
inp.role = role_bytes
476485
inp.enable_thinking = enable_thinking
477486
inp.input_type = RKLLM_INPUT_PROMPT
478-
inp.input_data.prompt_input = prompt.encode("utf-8")
487+
inp.input_data.prompt_input = prompt_bytes
479488
infer = RKLLMInferParam()
480489
ctypes.memset(ctypes.byref(infer), 0, ctypes.sizeof(infer))
481490
infer.mode = RKLLM_INFER_GENERATE

0 commit comments

Comments
 (0)