Improve Qwen3.6 coding agent and inference defaults (#300)

* Improve Qwen3.6 coding agent and inference defaults

* Clarify TUI agent cancellation behavior

* Trim coding agent tool paths

Author: leehack

CHANGELOG.md

@@ -1,5 +1,35 @@
 ## Unreleased
 
+* Changed unset native decoder/generative context batching from full-context
+  batches to llama.cpp-aligned caps of `n_batch = min(n_ctx, 2048)` and
+  `n_ubatch = min(n_batch, 512)`. Explicit positive values remain supported,
+  native encoder-only models retain full-context batching, and WebGPU preserves
+  its architecture-agnostic full-context fallback to avoid encoder regressions.
+* Reworked the TUI coding agent into a small Pi-style demo with one sequential
+  JSON loop, exactly `read`/`write`/`edit`/`bash`, a single-screen session, and
+  startup-only model selection, plus an optional read-only mode. It retains the
+  Qwen3.6 35B-A3B preset, shared model cache, cancellation and resource limits,
+  workspace-confined file tools, and an explicit unsandboxed-shell trust
+  boundary. The quality default now uses Unsloth `UD-Q4_K_M` with the
+  publisher-aligned non-thinking sampler, while `--thinking` opts into a larger
+  coding-focused reasoning profile. Model loading now delegates directly to
+  `LlamaEngine.loadModelSource`, replacing the example-specific downloader and
+  fuzzy repository resolver with the core local, HTTP, and exact `hf://` source
+  formats. Reasoning and normal answers now stream as separate transcript
+  channels, and final answers render with compact Markdown, syntax-highlighted
+  fenced code, solid code-block backgrounds, and no automatic block/message
+  spacer rows. Frame-coalesced presentation keeps long streams responsive, and
+  a lightweight single-window TurboVision theme restores the example's visual
+  identity without its former window manager. The example now targets Nocterm
+  `0.8.0`.
+* Added `GenerationParams.presencePenalty` for native llama.cpp sampling.
+  WebGPU and LiteRT-LM reject non-zero values until their runtimes expose an
+  equivalent control.
+* Made `ChatSession` context trimming reserve the requested output budget up to
+  half the active context, account for tool schemas, retain protocol turn
+  anchors while compacting completed exchanges, and reduce streaming
+  accumulation overhead.
+
 * Switched the OpenAI-compatible server example default to Unsloth's Qwen3.6
   27B `UD-Q4_K_XL` GGUF, added model-specific non-thinking and thinking sampler profiles,
   and documented its text-only / desktop-memory requirements. The server keeps

example/README.md

@@ -83,12 +83,26 @@ dart run bin/llamadart_cli.dart --help
 ```
 
 ### 5. TUI coding agent (`tui_coding_agent/`)
-A terminal UI coding agent built with `nocterm` and `llamadart` showing:
-- Streaming coding-chat UX in a TUI layout
-- Built-in tool loop (`list_files`, `read_file`, `search_files`, `write_file`, `run_command`)
-- Workspace-scoped path safety guard
-- Runtime model switching with `/model <source>`
-- Default GLM-4.7-Flash model source with custom model override support
+A small, Pi-style terminal coding agent built with `nocterm` and `llamadart`
+showing:
+
+- One sequential JSON tool loop with exactly `read`, `write`, `edit`, and
+  `bash`
+- A single-screen, single-session chat UI
+- A compact TurboVision-inspired frame with streamed reasoning and dense
+  Markdown answers, including syntax-highlighted fenced code blocks
+- Workspace-confined file tools plus bounded, cancellable source downloads,
+  generation, and active shell work; a model load cancelled during native
+  allocation unloads after that allocation call returns
+- An Unsloth Qwen3.6 35B-A3B `UD-Q4_K_M` default with publisher-aligned
+  non-thinking sampling and shared `llamadart` model caching
+- Optional `--thinking` mode with a larger context and coding-focused sampler
+- Startup-only workspace and model selection
+- Optional `--read-only` mode that exposes only the `read` tool
+
+The `bash` tool uses Bash on Unix or `cmd.exe` on Windows and runs with the
+current user's permissions. It is not sandboxed, and file-tool workspace
+confinement does not apply to shell commands.
 
 **Best for:** Building local terminal coding-agent workflows in Dart
 

example/basic_app/bin/llamadart_embedding_example.dart

@@ -48,12 +48,12 @@ Future<void> main(List<String> arguments) async {
     )
     ..addOption(
       'batch-size',
-      help: 'Logical batch size (0 uses context size).',
+      help: 'Logical batch size (0 uses the model-aware default).',
       defaultsTo: '0',
     )
     ..addOption(
       'ubatch-size',
-      help: 'Micro-batch size (0 follows batch size).',
+      help: 'Micro-batch size (0 uses the model-aware default).',
       defaultsTo: '0',
     )
     ..addOption(

example/basic_app/lib/services/sqlite_vector_cli_options.dart

@@ -181,12 +181,12 @@ ArgParser createSqliteVectorArgParser({required String defaultModelUrl}) {
     )
     ..addOption(
       'batch-size',
-      help: 'Logical batch size (0 uses context size).',
+      help: 'Logical batch size (0 uses the model-aware default).',
       defaultsTo: '0',
     )
     ..addOption(
       'ubatch-size',
-      help: 'Micro-batch size (0 follows batch size).',
+      help: 'Micro-batch size (0 uses the model-aware default).',
       defaultsTo: '0',
     )
     ..addOption(

example/tui_coding_agent/README.md

@@ -2,128 +2,176 @@
 
 Path: `example/tui_coding_agent`
 
-This example shows a terminal UI coding agent built with:
-
-- `llamadart` for local model inference and tool calling
-- `nocterm` for the interactive TUI layer
-
-By default it starts with GLM 4.7 Flash (`unsloth/GLM-4.7-Flash-GGUF:UD-Q4_K_XL`).
-You can switch to any local path, URL, or Hugging Face shorthand at startup or
-inside the app.
-
-Tool mode defaults to a **stable text-protocol tool loop** (recommended for
-GLM). Native template grammar tool-calling is optional and can be enabled with
-`--native-tool-calling`.
-
-## Features
-
-- Streaming assistant output in a TUI chat layout
-- TurboVision-inspired desktop with overlapping windows
-- Turbo C-style menu bar with popup menus (`Alt` + mnemonic)
-- Compact centered exit confirmation dialog
-- Built-in coding tools: `list_files`, `read_file`, `search_files`,
-  `write_file`, `run_command`
-- Workspace path guard (blocks tool access outside the selected workspace)
-- Model switching command at runtime (`/model <source>`)
-- Multi-session chat workflow (create/switch/close local sessions)
-- One desktop window per session (MDI-style overlapping chat windows)
-- Mouse window controls (focus, drag title bar, resize from lower-right handle)
-- Slash command autocomplete (`Tab`, `Shift+Tab`, `Up`, `Down`)
-- Tool-policy guardrails:
-  - direct conceptual prompts default to no-tool answers
-  - workspace/repo prompts enforce inspection before answering
-  - duplicate and over-limit tool calls are skipped safely
-- Default coding-oriented sampling values
+This is a deliberately small, Pi-style local coding agent built with
+`llamadart` and `nocterm`. It keeps one model, one conversation, one screen,
+four general tools, and a sequential model-to-tool loop that is easy to read
+and adapt.
 
-## Run
+## Agent loop
 
-```bash
-cd example/tui_coding_agent
-dart pub get
-dart run bin/tui_coding_agent.dart
-```
+For each user request, the example:
 
-### Run with a specific workspace
+1. asks the model for the next response;
+2. accepts either a normal final answer or exactly one standalone JSON tool
+   call;
+3. executes that tool and returns its JSON result to the model; and
+4. repeats until the model answers normally or the round limit is reached.
 
-```bash
-dart run bin/tui_coding_agent.dart --workspace /path/to/project
+The only executable text-protocol form is:
+
+```text
+<tool_call>{"name":"tool_name","arguments":{...}}</tool_call>
 ```
 
-### Run with a specific model
+The block must be the complete assistant response. Calls run one at a time;
+prose around a call, sibling calls, shorthand, XML variants, unknown tools,
+and malformed or incomplete JSON are not executed.
 
-```bash
-dart run bin/tui_coding_agent.dart --model /path/to/model.gguf
+## Tools
+
+The model normally receives exactly four tools:
+
+| Tool | Purpose |
+| --- | --- |
+| `read` | Read bounded UTF-8 text from a workspace file. |
+| `write` | Create or overwrite a UTF-8 workspace file. |
+| `edit` | Replace exactly one literal occurrence in a workspace file. |
+| `bash` | Run a command with Bash on Unix or `cmd.exe` on Windows. |
+
+The file tools resolve paths canonically and reject paths, including symlink
+targets, outside the selected workspace. Applicable `AGENTS.md` files from the
+workspace and its ancestors are loaded once when the session starts and added
+to the system prompt. Start with `--read-only` to expose only `read` and omit
+all mutation and shell tools.
+
+## Default model and cache
+
+The default model is Qwen3.6 35B-A3B:
+
+```text
+hf://unsloth/Qwen3.6-35B-A3B-GGUF/Qwen3.6-35B-A3B-UD-Q4_K_M.gguf
 ```
 
-or
+The native-desktop preset targets systems with at least 32 GB of RAM and uses:
+
+- 16,384-token context
+- up to 4,096 output tokens
+- full GPU offload where supported
+- logical batch size `2,048`, physical micro-batch size `512`
+- temperature `0.7`, top-K `20`, top-P `0.8`, min-P `0`
+- repeat penalty `1.0`, presence penalty `1.5`
+- non-thinking generation
+- at most 24 tool rounds per request
+
+Pass `--thinking` to opt into the higher-quality reasoning profile:
+
+- 32,768-token context
+- up to 8,192 output tokens
+- logical batch size `2,048`, physical micro-batch size `512`
+- temperature `0.6`, top-K `20`, top-P `0.95`, min-P `0`
+- repeat penalty `1.0`, presence penalty `0`
+- thinking enabled
+
+Thinking is opt-in because it is slower, uses substantially more context and
+memory, and can spend much of the output budget reasoning before it reaches a
+tool call or final answer. The default non-thinking profile keeps the local
+demo responsive; use `--thinking` when solution quality matters more than
+latency and your machine can accommodate the larger context. The 32K profile
+keeps practical memory headroom for the quantized model on a 64 GB Apple Silicon
+system; larger 48K and 64K contexts can exhaust Metal allocations.
+
+Qwen3.6 exposes thinking as an on/off capability, not a trained
+low/medium/high reasoning-effort scale. The example therefore keeps the control
+honest: omit `--thinking` for off, or pass it for on. When enabled, prior
+reasoning is preserved across the agent's tool rounds through Qwen's
+`preserve_thinking` template option. Upstream llama.cpp also has a mechanical
+reasoning-token budget, but llamadart's embedded inference API does not expose
+that control yet; total output tokens are not used as a fake reasoning level.
+
+The session passes this source directly to `LlamaEngine.loadModelSource`, so
+downloads use `llamadart`'s shared per-user model cache and resume behavior.
+Use `--cache-dir` only to override that cache root. An earlier flat
+`UD-Q4_K_S` TUI file remains usable by passing its local path explicitly; it is
+not substituted for the new `UD-Q4_K_M` default.
+
+The model is selected only at startup. Pass a local GGUF path, HTTP(S) URL, or
+exact `hf://owner/repo/path/to/model.gguf` reference with `--model` before
+launching the TUI.
+
+## Run
 
 ```bash
-dart run bin/tui_coding_agent.dart --model owner/repo:Q4_K_M
+cd example/tui_coding_agent
+dart pub get
+dart run bin/tui_coding_agent.dart
 ```
 
-### Enable native template tool-calling (experimental)
+Choose a workspace and model at startup when needed:
 
 ```bash
-dart run bin/tui_coding_agent.dart --native-tool-calling
+dart run bin/tui_coding_agent.dart --workspace /path/to/project
+dart run bin/tui_coding_agent.dart --thinking
+dart run bin/tui_coding_agent.dart --read-only
+dart run bin/tui_coding_agent.dart --model /path/to/model.gguf
+dart run bin/tui_coding_agent.dart \
+  --model hf://owner/repo/path/to/model.gguf \
+  --cache-dir /path/to/model-cache
 ```
 
-## Interactive Commands
-
-- `/help` - Show available commands
-- `/clear` - Clear active session log
-- `/model` - Show current model source and loaded path
-- `/model <path|url|owner/repo[:hint]>` - Switch model and reset session history
-- `/workspace` - Show workspace root
-- `/new` - Create a new session
-- `/next` - Switch to next session
-- `/prev` - Switch to previous session
-- `/close` - Close current session
-- `/zoom-window` - Zoom/unzoom active desktop window
-- `/next-window` - Focus next desktop window
-- `/prev-window` - Focus previous desktop window
-- `/tile-windows` - Arrange all windows in a tiled grid
-- `/stack-windows` - Arrange all windows in stacked cascade
-- `/cancel` - Cancel active generation
-- `/exit` - Open exit confirmation dialog
-- `/quit` - Open exit confirmation dialog
-
-## Keyboard Shortcuts
-
-- `F1` - Show help
-- `F2` - Seed `/model ` command
-- `F3` - Clear conversation
-- `F4` - Previous session
-- `F5` - Zoom/unzoom active window
-- `F6` - Focus next desktop window
-- `F7` - Create new session
-- `F8` - Next session
-- `F9` - Focus previous desktop window
-- `F10` - Open/close top menu bar
-- `F12` - Close active session
-- `Alt+W` - Close active session
-- `Alt+X` - Open exit confirmation
-- `Alt` + Arrow keys - Move active window
-- `Alt` + `Shift` + Arrow keys - Resize active window
-- `Esc` - Cancel generation when busy, otherwise open exit confirmation
-- `Alt+<menu letter>` - Open top menu (`File/Edit/Search/...`)
-
-In the exit confirmation dialog:
-
-- `Left`/`Right`/`Tab` (`h`/`l` also supported) toggles YES/NO
-- `Enter` confirms the currently selected option
-- `Esc` or `N` selects NO and dismisses the dialog
-
-## Notes
-
-- The first run may take time due to model download and initialization.
-- Downloaded models use the per-user shared `llamadart` model cache by default;
-  pass `--cache-dir` to use a workspace-local or app-specific cache instead.
-- `run_command` uses a safety filter and executes from the workspace root
-  (or a workspace-relative subdirectory).
-- `run_command` accepts `command` and also tolerates alias keys (`cmd`,
-  `input`, `shell_command`) for model-compatibility.
-- If you hit grammar-related crashes in native mode, run without
-  `--native-tool-calling` (default stable mode).
+Use `--help` for the complete startup options.
+
+## TUI
+
+The single-screen UI restores a compact TurboVision-inspired blue window,
+gray status chrome, and framed transcript without restoring the old multi-window
+desktop. It maintains one in-memory conversation for the loaded model.
+
+Normal answers stream incrementally through a compact GitHub-flavored Markdown
+renderer with headings, lists, links, inline code, and syntax-highlighted fenced
+code. Code fences use their language label when available, cheaply recognize a
+few common unlabelled formats, and fall back safely to plain code. Their
+background fills every row, including empty lines. Markdown blocks and
+transcript messages no longer add automatic blank spacer rows. With
+`--thinking`, reasoning streams separately under a subdued `[think]` label and
+the final answer remains under `[agent]`. The stream gate withholds text that
+could be a split `<tool_call>` envelope, so raw executable JSON never flashes
+in the transcript before validation. High-frequency deltas are coalesced to
+terminal frame cadence and accumulated in buffers. Off-screen transcript rows
+are built lazily, while the active Markdown row is reparsed only at presentation
+cadence rather than for every burst of backend deltas. Syntax highlighting is
+bounded for unusually large generated blocks so streaming stays responsive.
+
+- `/help` shows the small command list.
+- `/clear` resets the conversation without reloading the model.
+- `/model` reports the startup-selected source and loaded model name.
+- `/workspace` reports the workspace root.
+- `/cancel` cancels current work.
+- `/quit` exits.
+- `Esc` cancels while busy; otherwise press it twice to quit.
+- `Ctrl+C` cancels while busy and exits cleanly while idle.
+
+## Limits and trust boundary
+
+Model resolution and downloads, generation, and the active `bash` process tree
+can be cancelled. File reads and edits, tool results, shell output, command
+duration, context use, and tool rounds are bounded so the demo fails clearly
+instead of growing without limit.
+
+Native model allocation itself cannot be interrupted. If cancellation is
+requested during that phase, the session unloads the model as soon as the
+allocation call returns.
+
+Cancellation does not roll back effects that already happened. The transcript
+shows the command or file path, a bounded result summary, and a warning when an
+interrupted tool may have changed state; the next turn retains that warning.
+
+`bash` is intentionally a general shell tool and is **not sandboxed**. Its
+commands run with the current user's normal permissions and can read or modify
+files outside the selected workspace, access the environment and network, and
+start arbitrary child processes. Workspace confinement applies only to
+`read`, `write`, and `edit`. Use the demo only with trusted prompts and
+repositories, or run the whole process inside an external sandbox or
+container.
 
 ## Test