Improve Qwen3.6 coding agent and inference defaults

[leehack]

Summary

• Rework the Nocterm TUI into a deliberately small Pi-style coding agent for Unsloth Qwen3.6 35B-A3B, with one sequential JSON tool loop and exactly read, write, edit, and bash.
• Load local, HTTP(S), and exact Hugging Face sources through LlamaEngine.loadModelSource, delegating the default cache and resumable downloads to llamadart.
• Add separate streamed thinking/final-answer channels, opt-in --thinking, compact Markdown rendering, fenced-code syntax highlighting, solid code-block backgrounds, and a single-window TurboVision theme.
• Align native decoder batching with llama.cpp (n_batch <= 2048, n_ubatch <= 512), add native presencePenalty, and make ChatSession context compaction reserve output/tool-schema space while retaining coherent agent turns.
• Refresh affected docs, examples, changelog entries, and compatible dependency locks.

Production-readiness scope

• User-facing scope: Run a local Qwen3.6 coding-agent demo with a bounded/cancellable sequential tool loop, optional read-only mode, shared model caching, streamed reasoning/final output, and terminal Markdown/code rendering. Native decoder defaults and context management are safer for agent workloads.
• Supported platforms/paths: Dart VM/native llama.cpp for the TUI; core local/HTTP/exact-hf:// model-source paths; existing WebGPU and LiteRT-LM paths retain their documented behavior.
• Unsupported or intentionally unavailable paths: bash is deliberately unsandboxed and documented as such. WebGPU and LiteRT-LM reject non-zero presence penalty until their runtimes expose an equivalent control. Qwen reasoning is honestly exposed as on/off; no synthetic low/medium/high effort control is claimed.
• Out of scope / follow-ups: None.

Completeness checklist

• Declared scope is fully implemented, or explicitly reduced above.
• Unsupported platform/option combinations fail loudly with actionable diagnostics or are clearly documented as unavailable.
• Public API docs, README/website docs, examples, support matrices, and changelog entries are updated where relevant.
• Regression coverage covers the original issue plus key negative/version-skew paths where relevant.
• Security/privacy review completed: no secrets, bearer tokens, signed URLs, or raw secret-bearing paths leak through logs, cache keys, metadata, errors, or snapshots.
• Follow-up work that is useful but not required for this PR is tracked in GitHub Issues, or explicitly marked "None" above.

PR type guidance

• Feature PR: includes API/docs/example updates, platform behavior, explicit unsupported paths, and positive/negative regression coverage.

Test Plan

• dart format --output=none --set-exit-if-changed . — 507 files, 0 changed. The unfetched Flutter chat example emitted only its existing local flutter_lints resolution warning.
• dart analyze — root lib test and the TUI package report no issues.
• dart test -p vm -j 1 --exclude-tags local-only — 1,289 passed; 66 expected fixture/model skips. The repository's small native llama.cpp smoke model loaded and generated successfully.
• dart test -p chrome --exclude-tags local-only — 724 passed; 1 expected skip.
• Other targeted/local-only validation:
  • TUI package: 79 tests passed.
  • ChatSession targeted suite: 13 passed; 1 existing native-template skip.
  • Coverage: 74.87% (10,542 / 14,081 lines), above the 70% gate.
  • Website build and link validation passed under Node 20/npm 10.
  • Release-doc version consistency passed.
  • Root and TUI direct/dev dependencies are current within their declared constraints; website compatible lock refresh completed. npm audit reports 19 moderate development-server transitive advisories with no available fix, and no high/critical advisories.
  • CLI --help smoke passed.
  • The cached ~21 GB Qwen artifact was not loaded during validation to avoid repeating the reported machine-memory crash; model-backed Qwen generation is therefore recorded as N/A below.

Matrix Evidence

Matrix row	Scope covered	Platform / model / backend	Result	Evidence / notes
static-format-analyze	format, analyzer, import boundaries	macOS arm64 / Dart VM	PASS	507 files unchanged; root and TUI analyzers clean
root-vm	native-safe unit/integration coverage	macOS arm64 / Dart VM	PASS	1,289 passed; 66 expected fixture/model skips; small llama.cpp smoke generated
root-chrome	browser-compatible unit/integration coverage	Chrome	PASS	724 passed; 1 expected skip
coverage-lib	maintainable lib/ coverage	Dart VM	PASS	74.87%, above 70% threshold
docs-site	website build and docs links	Node 20 / npm 10	PASS	site build and link validator passed
release-doc-version-consistency	install/version snippets	Dart VM	PASS	verifier passed
examples-tests	TUI agent loop, tools, streaming, theme, Markdown	macOS arm64 / mocked backend	PASS	79 TUI tests passed
native-hook-bundles	native ABI/symbol regression coverage	macOS arm64 / llama.cpp	PASS	covered by full VM suite and small native smoke
gguf-chat-features-smoke	Qwen3.6 thinking/tool loop on target GGUF	macOS arm64 / 35B-A3B Q4_K_M	N/A	21 GB target model intentionally not loaded after prior host memory crash
native-inference-benchmark	target-model throughput and memory	macOS arm64 / 35B-A3B Q4_K_M	N/A	no safe target-model load in this run
web-bridge-smoke	WebGPU runtime behavior	WebGPU	N/A	no bridge/runtime assets changed; Chrome unit coverage passed
litert-lm-engine-smoke	LiteRT-LM runtime behavior	LiteRT-LM	N/A	only explicit unsupported presence-penalty validation changed; covered by unit tests
linux-x64-ci-runtime	cross-platform native behavior	Linux x64	N/A	delegated to CI
windows-x64-ci-runtime	cross-platform native behavior	Windows x64	N/A	delegated to CI

Review Notes

• Independent review status: local deep-clean/readiness pass completed after the initial commit; no remaining findings. Both root and TUI dart fix --dry-run report nothing to fix.
• CI status / head SHA: all required checks passed on e4d8c3a4f.

[github-actions]

Chat app preview removed for leehack/llamadart-chat-pr-300.

[Copilot]

Pull request overview

This PR updates llamadart’s defaults and examples to better support a Qwen3.6-focused, sequential tool-loop coding-agent workflow, while aligning native batching and sampling behavior with upstream llama.cpp and tightening ChatSession context budgeting for agent turns.

Changes:

• Aligns native decoder/generative batching defaults to n_batch <= 2048 and n_ubatch <= 512, while preserving full-context defaults for encoder-only models and WebGPU’s architecture-agnostic fallback.
• Adds native llama.cpp presence-penalty support via GenerationParams.presencePenalty, explicitly rejecting unsupported backends (WebGPU, LiteRT-LM).
• Reworks the TUI coding-agent example into a small single-screen, four-tool, strict JSON-envelope loop with streaming thinking/final channels and compact Markdown + syntax highlighting.

Reviewed changes

Copilot reviewed 66 out of 69 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
website/docs/guides/performance-tuning.md	Documents new native batching defaults and presence penalty guidance.
website/docs/guides/embeddings.md	Clarifies encoder throughput batching vs decoder defaults and WebGPU behavior.
website/docs/examples/tui-coding-agent.md	Updates the example guide to the new small Pi-style agent loop and flags.
website/docs/configuration/runtime-parameters.md	Updates runtime parameter docs to reflect model-aware batching and presence penalty semantics.
website/docs/changelog/recent-releases.md	Adds an Unreleased entry summarizing the behavioral and example updates.
test/unit/core/models/inference/model_params_test.dart	Adds/updates tests for new default batching constants and resolution policy.
test/unit/core/models/inference/generation_params_test.dart	Adds tests ensuring presencePenalty is stored and copied.
test/unit/core/models/chat/chat_message_test.dart	Tests continuesPreviousTurn metadata behavior and JSON serialization exclusion.
test/unit/core/engine/engine_test.dart	Adds regression test ensuring cancelled downloads don’t proceed to model load.
test/unit/core/engine/chat_session_test.dart	Adds coverage for new context budgeting/trimming and continuation-turn handling.
test/unit/backends/webgpu/webgpu_backend_test.dart	Updates tests for WebGPU batching defaults and presence-penalty rejection.
test/unit/backends/llama_cpp/llama_cpp_service_test.dart	Adds tests for presence-penalty mapping and updated batch-size resolution.
test/unit/backends/litert_lm/litert_lm_service_test.dart	Updates unsupported-parameter reporting to include presence penalty.
test/unit/backends/litert_lm/litert_lm_backend_web_test.dart	Adds web backend test rejecting non-zero presence penalty.
test/integration/backends/llama_cpp/native_symbol_integration_test.dart	Verifies batch constants match pinned llama.cpp defaults.
lib/src/core/models/inference/model_params.dart	Introduces default batch constants and a model-aware batch resolution policy hook.
lib/src/core/models/inference/generation_params.dart	Adds presencePenalty to the public generation parameter surface.
lib/src/core/models/chat/chat_message.dart	Adds continuesPreviousTurn as internal metadata for trimming coherence.
lib/src/core/engine/engine.dart	Adds cancellation checks during loadModelSource resolution/download flow.
lib/src/core/engine/chat_session.dart	Reworks context budgeting to reserve output/tool-schema space and compact completed exchanges.
lib/src/backends/webgpu/webgpu_backend.dart	Preserves full-context WebGPU auto-batching and rejects presence penalty explicitly.
lib/src/backends/llama_cpp/llama_cpp_service.dart	Detects encoder-only models for full-context defaults and maps presence penalty to sampler penalties.
lib/src/backends/litert_lm/litert_lm_service.dart	Adds presence-penalty to unsupported-parameter validation.
lib/src/backends/litert_lm/litert_lm_backend_web.dart	Adds presence-penalty to unsupported-parameter validation (web).
example/tui_coding_agent/test/workspace_tools_test.dart	Expands tests for the new four-tool surface, bounds, cancellation, and confinement rules.
example/tui_coding_agent/test/workspace_guard_test.dart	Adds tests for canonicalization and symlink escape prevention.
example/tui_coding_agent/test/tool_usage_policy_test.dart	Removes tests for deleted tool-usage heuristics (simplified agent loop).
example/tui_coding_agent/test/tool_call_gate_test.dart	Removes tests for deleted per-round tool-call gating.
example/tui_coding_agent/test/text_tool_call_parser_test.dart	Replaces permissive parsing tests with strict single-envelope JSON parsing tests.
example/tui_coding_agent/test/session_event_coalescer_test.dart	Adds tests for frame-coalesced streaming event batching.
example/tui_coding_agent/test/runner_tool_regression_test.dart	Removes regression tests tied to the old inline-tag parsing and tool set.
example/tui_coding_agent/test/model_source_resolver_test.dart	Removes tests for deleted fuzzy HF resolver/downloader (now uses loadModelSource).
example/tui_coding_agent/test/coding_agent_tui_test.dart	Adds UI regressions for the TurboVision shell, badges, and lazy transcript behavior.
example/tui_coding_agent/test/coding_agent_markdown_test.dart	Adds coverage for compact Markdown + fenced-code highlighting and styling.
example/tui_coding_agent/test/coding_agent_config_test.dart	Adds tests for Qwen presets, cache delegation, and tool-round defaults.
example/tui_coding_agent/test/assistant_text_stream_test.dart	Adds tests for streaming text gating around tool-call marker splits.
example/tui_coding_agent/test/assistant_draft_rows_test.dart	Adds tests for resetting assistant draft rows when a tool-call attempt is detected.
example/tui_coding_agent/README.md	Rewrites README to match the new small agent design, presets, limits, and trust boundary.
example/tui_coding_agent/pubspec.yaml	Updates dependencies for Markdown rendering and syntax highlighting; bumps nocterm.
example/tui_coding_agent/lib/tui_coding_agent.dart	Narrows the library export surface to the intended public API.
example/tui_coding_agent/lib/src/workspace_guard.dart	Hardens workspace confinement and symlink resolution semantics.
example/tui_coding_agent/lib/src/tool_usage_policy.dart	Removes old heuristic “tool usage policy” implementation.
example/tui_coding_agent/lib/src/tool_call_gate.dart	Removes old tool-call gating implementation.
example/tui_coding_agent/lib/src/text_tool_call_parser.dart	Replaces permissive parser with strict single-envelope JSON tool-call parser.
example/tui_coding_agent/lib/src/session_event.dart	Expands event types to support thinking/final channels, warnings, and draft resets.
example/tui_coding_agent/lib/src/session_event_coalescer.dart	Adds frame-based coalescing for high-frequency stream deltas.
example/tui_coding_agent/lib/src/model_source_resolver.dart	Removes old custom resolver/downloader in favor of LlamaEngine.loadModelSource.
example/tui_coding_agent/lib/src/coding_agent_theme.dart	Adds a consolidated TurboVision-inspired theme and Markdown style sheets.
example/tui_coding_agent/lib/src/coding_agent_syntax_highlighter.dart	Adds fenced-code highlighting via Highlight.js grammars with size bounds and aliasing.
example/tui_coding_agent/lib/src/coding_agent_markdown.dart	Adds a compact Markdown renderer with solid code backgrounds and table/list support.
example/tui_coding_agent/lib/src/coding_agent_config.dart	Adds Qwen3.6 presets, cache override semantics, and config validation.
example/tui_coding_agent/lib/src/assistant_text_stream.dart	Adds streaming gate to prevent tool-call JSON from flashing as assistant prose.
example/tui_coding_agent/lib/src/assistant_draft_rows.dart	Tracks assistant/thinking draft rows for safe reset when tool markers appear.
example/tui_coding_agent/bin/tui_coding_agent.dart	Updates CLI flags to startup-only configuration, --thinking, and --read-only.
example/README.md	Updates repository example overview to match the new TUI coding-agent behavior.
example/basic_app/lib/services/sqlite_vector_cli_options.dart	Updates CLI help strings to reflect model-aware batching defaults.
example/basic_app/bin/llamadart_embedding_example.dart	Updates CLI help strings to reflect model-aware batching defaults.
CHANGELOG.md	Adds Unreleased entries for batching defaults, TUI agent rework, presence penalty, and ChatSession trimming.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 66 out of 69 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 66 out of 69 changed files in this pull request and generated no new comments.