Skip to content

Commit daef6c4

Browse files
author
Brendan Gray
committed
v1.8.4: Fix stopReason passthrough, creative writing preamble, JSON template literal parsing
Bug 1: LLMEngine.generateStream() now passes through metadata.stopReason='maxTokens' from node-llama-cpp instead of hardcoding 'natural'. Enables seamless continuation to trigger. Bug 2: Added general instruction to preambles telling model to respond with creative text directly instead of creating code files. Bug 3: Improved fixBackticks() in toolParser.js to properly escape multi-line content and inner double-quotes when converting template literals to JSON strings.
1 parent acd9aa2 commit daef6c4

3 files changed

Lines changed: 24 additions & 3 deletions

File tree

main/constants.js

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,7 @@ You can also answer general questions, help with writing, and have normal conver
5151
- **You have no knowledge of what files exist in the project until you call list_directory.** Never list, name, or assume project files from memory — always call list_directory first.
5252
- Use tools when action is required: reading files, running commands, browsing, writing or editing code
5353
- For general knowledge, concept questions, conversations, stories, creative writing, or any non-file task: respond directly — no tools needed
54+
- When the user asks for a story, poem, essay, or any creative/written text, respond with the text directly in your message. Do not create files unless the user explicitly asks for a file to be saved.
5455
- When the user describes a bug, error, or unexpected behavior: call read_file on the relevant file first, then diagnose — name the specific file in your answer
5556
- If a bug is described with no file name, error, or stack trace, ask ONE clarifying question — do not call any tools yet
5657
- Use web_search when the answer may have changed since your training (current doc versions, real-time info, recent events, anything that varies over time). Do not use for static programming knowledge you can answer directly.
@@ -73,6 +74,7 @@ read_file, write_file, edit_file, list_directory, find_files, grep_search, run_c
7374
- You do not know file contents until you call read_file. Never guess.
7475
- You do not know what files exist until you call list_directory.
7576
- For general knowledge, concept questions, conversations, stories, creative writing, or any non-file task: answer directly — no tools needed.
77+
- When the user asks for a story, poem, essay, or any creative/written text, respond with the text directly in your message. Do not create files unless the user explicitly asks for a file to be saved.
7678
- For bugs: read_file the relevant file first, then diagnose.
7779
- For live/current/time-sensitive info: call web_search. Never guess dates or current state.
7880
- To visit a URL: call browser_navigate. To read a page: browser_snapshot first.

main/llmEngine.js

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -620,13 +620,19 @@ class LLMEngine extends EventEmitter {
620620
if (this._kvReuseCooldown > 0) this._kvReuseCooldown--;
621621

622622
const sanitized = this._sanitizeResponse(fullResponse);
623+
// Pass through node-llama-cpp's stopReason when it indicates maxTokens
624+
let finalStopReason = detectedToolBlock ? 'tool_call' : 'natural';
625+
if (result?.metadata?.stopReason === 'maxTokens') {
626+
finalStopReason = 'maxTokens';
627+
console.log(`[LLM] Generation stopped at maxTokens (${fullResponse.length} chars)`);
628+
}
623629
return {
624630
text: sanitized,
625631
rawText: fullResponse,
626632
model: this.modelInfo?.name || 'unknown',
627633
tokensUsed: this.sequence?.nTokens || 0,
628634
contextUsed: this.context?.contextSize || 0,
629-
stopReason: detectedToolBlock ? 'tool_call' : 'natural',
635+
stopReason: finalStopReason,
630636
};
631637
} catch (err) {
632638
return this._handleGenerationError(err, fullResponse, detectedToolBlock);
@@ -917,11 +923,14 @@ class LLMEngine extends EventEmitter {
917923
if (!dup) collectedCalls.push({ functionName: rc.functionName, params: rc.params });
918924
}
919925

926+
// Pass through node-llama-cpp's stopReason when it indicates maxTokens
927+
let finalStopReason = collectedCalls.length > 0 ? 'function_call' : 'natural';
928+
if (result?.metadata?.stopReason === 'maxTokens') finalStopReason = 'maxTokens';
920929
return {
921930
text: this._sanitizeResponse(fullResponse),
922931
response: fullResponse,
923932
functionCalls: collectedCalls,
924-
stopReason: collectedCalls.length > 0 ? 'function_call' : 'natural',
933+
stopReason: finalStopReason,
925934
};
926935
} catch (err) {
927936
if (err.name === 'AbortError' || err.message?.includes('aborted')) {

main/tools/toolParser.js

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -124,7 +124,17 @@ function fixQuoting(raw) {
124124

125125
function fixBackticks(raw) {
126126
if (!raw) return raw;
127-
return raw.replace(/`([^`]*)`/g, '"$1"');
127+
// Replace backtick-delimited strings with properly escaped JSON double-quoted strings
128+
return raw.replace(/`([\s\S]*?)`/g, (match, inner) => {
129+
// Escape characters that are invalid in JSON strings
130+
const escaped = inner
131+
.replace(/\\/g, '\\\\')
132+
.replace(/"/g, '\\"')
133+
.replace(/\n/g, '\\n')
134+
.replace(/\r/g, '\\r')
135+
.replace(/\t/g, '\\t');
136+
return '"' + escaped + '"';
137+
});
128138
}
129139

130140
function tryParseJson(raw) {

0 commit comments

Comments
 (0)