Improve OpenAI server tool calling

* Improve OpenAI server tool calling

* Reject incomplete tool transcripts

Author: leehack

CHANGELOG.md

@@ -1,3 +1,19 @@
+## Unreleased
+
+* Switched the OpenAI-compatible server example default to Unsloth's Qwen3.6
+  27B `UD-Q4_K_XL` GGUF, added model-specific non-thinking and thinking sampler profiles,
+  and documented its text-only / desktop-memory requirements. The server keeps
+  thinking opt-in through the `enable_thinking` request field and now uses
+  Relic 1.2.0.
+
+* Removed the example server's nonstandard automatic tool-execution loop. Its
+  Chat Completions endpoint now consistently returns model-emitted `tool_calls`
+  for client-side execution, accepts standard assistant/tool follow-up
+  transcripts, honors named function `tool_choice`, and documents the full
+  request sequence with runnable Swagger examples. Server model inputs now load
+  through `LlamaEngine.loadModelSource(...)`, sharing the core runtime's source
+  resolution and cache behavior.
+
 ## 0.8.15
 
 * Added clipboard media attachments to the runnable chat app. Desktop and web

example/llamadart_server/README.md

@@ -27,7 +27,9 @@ This example now follows a feature-first structure:
 - `lib/src/features/openai_api/` - OpenAI-compatible HTTP server and docs
 - `lib/src/features/chat_completion/` - chat request model, parser, mapper, and
   completion use cases
-- `lib/src/features/model_management/` - model path resolution and download
+- Model paths, URLs, and Hugging Face sources load through
+  `LlamaEngine.loadModelSource(...)`, using the core runtime's cache and
+  download policy
 - `lib/src/features/server_engine/` - engine contract + llama engine adapter
 - `lib/src/features/shared/` - shared API error types
 - `lib/src/bootstrap/` - CLI argument parsing and runtime wiring
@@ -38,30 +40,34 @@ Public APIs are exported directly from `lib/llamadart_server.dart`.
 
 - Supports a single generation at a time (returns 429 while busy)
 - Supports `n = 1` only
-- By default, tools are passed through to the model prompt only (no server-side
-  execution). Use `--enable-tool-execution` to enable an example built-in tool
-  loop.
+- Function tools follow the standard client-managed Chat Completions flow: the
+  server returns `tool_calls`, your client executes them, then submits the
+  assistant tool-call message and matching `role: "tool"` result in a new
+  request. The server never executes application tools.
 - No legacy Completions endpoint
+- The default Qwen3.6 27B GGUF is loaded text-only; this example does not
+  download or load its separate vision projector.
 
 ## Run
 
 ```bash
 dart pub get
-dart run llamadart_server --model /path/to/model.gguf
+dart run llamadart_server
 ```
 
-Use a remote GGUF URL instead of a path if you want automatic download.
+With no `--model` override, the first launch downloads the default 17.6 GB
+Unsloth model. Pass `--model /path/to/model.gguf` to use a local GGUF instead.
+You can also pass a remote GGUF URL or an `hf://owner/repo/path` source for
+automatic resolution and download.
 
 Optional flags:
 
 - `--model-id` (default: `llamadart-local`)
 - `--host` (default: `127.0.0.1`)
 - `--port` (default: `8080`)
 - `--api-key` (optional)
-- `--context-size` (default: `4096`)
+- `--context-size` (default: `16384`)
 - `--gpu-layers` (default: `999`)
-- `--enable-tool-execution` (default: disabled; enables built-in demo handlers)
-- `--max-tool-rounds` (default: `5`; used only when tool execution is enabled)
 - `--log` (enable verbose Dart + HTTP request logs; native logs stay error-only)
 
 ### Exit codes
@@ -72,24 +78,50 @@ Optional flags:
 
 ### Sampling defaults
 
-When omitted in a request body, this example server applies a stable default
-`GenerationParams` baseline:
+When omitted in a request body, this example server applies the supported
+subset of Qwen3.6 27B's non-thinking text profile:
 
-- `penalty = 1.0`
-- `top_p = 0.95`
-- `min_p = 0.05`
+- `temperature = 0.7`
+- `top_k = 20`
+- `top_p = 0.8`
+- `min_p = 0.0`
+- `repetition penalty = 1.0`
 
 Request-provided sampling fields (for example `temperature`, `top_p`, `seed`,
 `max_tokens`) override these defaults per call.
 
+Qwen also recommends a presence penalty, but this example's current backend
+does not expose presence-penalty control. Do not substitute repetition penalty
+for it; they have different effects.
+
+### Default model and thinking mode
+
+The default is Unsloth's `Qwen3.6-27B-UD-Q4_K_XL.gguf` (about 17.6 GB). Plan
+for at least 32 GB of free unified memory or VRAM plus runtime headroom; use
+`--gpu-layers 0` to force CPU inference when GPU memory is insufficient.
+The 16,384-token context is a practical server default that leaves room for
+normal chat;
+increase it cautiously because larger contexts need substantially more memory.
+
+Qwen3.6 thinks by default, but this API server intentionally disables thinking
+unless a request sets `"enable_thinking": true`, so ordinary calls return
+assistant `content` promptly. Thinking responses use a separate
+`reasoning_content` channel and the Qwen thinking profile (`temperature: 1.0`,
+`top_p: 0.95`). `max_tokens` covers both reasoning and the final answer; the
+server's inherited default is 4,096 tokens. Use streaming and raise the limit
+for thought-intensive requests while leaving room for the prompt in the
+16,384-token context. A request that ends during reasoning can correctly have
+`reasoning_content` but no final `content`.
+
 ## API Examples
 
 ### 0. OpenAPI and Swagger UI
 
 - OpenAPI JSON: `http://127.0.0.1:8080/openapi.json`
 - Swagger UI: `http://127.0.0.1:8080/docs`
-- Swagger includes ready-made chat request examples (basic, streaming, tools)
-  and embeddings examples under `POST /v1/embeddings`.
+- Swagger includes ready-made chat request examples for basic and streaming
+  completions, initial tool calls, streamed tool calls, thinking plus streamed
+  tool calls, and the client-executed tool-result follow-up.
 
 ```bash
 curl http://127.0.0.1:8080/openapi.json
@@ -115,7 +147,8 @@ curl http://127.0.0.1:8080/v1/chat/completions \
       {"role": "system", "content": "You are concise."},
       {"role": "user", "content": "Give me one sentence about Seoul."}
     ],
-    "max_tokens": 128
+    "max_tokens": 128,
+    "enable_thinking": false
   }'
 ```
 
@@ -133,14 +166,40 @@ curl -N http://127.0.0.1:8080/v1/chat/completions \
   }'
 ```
 
-### 4. With API key
+### 4. Tool calling (client-executed)
+
+This server uses the standard OpenAI Chat Completions sequence:
+
+1. Send the user messages, `tools`, and a `tool_choice`.
+2. Receive an assistant response with `finish_reason: "tool_calls"` and
+   `message.tool_calls`.
+3. Execute each function in your application.
+4. Append the entire assistant tool-call message and one `role: "tool"` message
+   per result. Each result must use the exact returned `tool_call_id`.
+5. Send the expanded transcript in a new `POST /v1/chat/completions` request.
+
+Set `parallel_tool_calls: true` to permit multiple calls in one response when
+the active model template supports it; append a result for every returned call.
+
+Use Swagger's **Tool call: request a function** example for step 1 and
+**Tool call: submit the function result** for step 5. Swagger cannot transfer
+a runtime call ID between requests, so replace `call_weather_example` and its
+illustrative assistant message with the actual response from step 1. The
+minimal standard tool-result message does not need a `name` field.
+
+For a streaming tool call, accumulate `delta.tool_calls` fragments by index
+until the terminal `finish_reason` is `tool_calls`; then follow the same steps.
+The **Tool call: reasoning + function request (SSE)** Swagger example also
+demonstrates Qwen's optional `reasoning_content` stream.
+
+### 5. With API key
 
 ```bash
 curl http://127.0.0.1:8080/v1/models \
   -H "Authorization: Bearer YOUR_KEY"
 ```
 
-### 5. Embeddings
+### 6. Embeddings
 
 ```bash
 curl http://127.0.0.1:8080/v1/embeddings \

example/llamadart_server/lib/llamadart_server.dart

@@ -1,7 +1,6 @@
 /// Public entry point for the llamadart server example package.
 library;
 
-export 'src/features/model_management/model_management.dart';
 export 'src/features/shared/shared.dart';
 export 'src/features/chat_completion/chat_completion.dart';
 export 'src/features/embeddings/embeddings.dart';

example/llamadart_server/lib/src/bootstrap/cli/server_cli_arg_parser.dart

@@ -2,8 +2,8 @@ import 'package:args/args.dart';
 import 'package:llamadart/llamadart.dart';
 
 const String _defaultModelUrl =
-    'https://huggingface.co/unsloth/Qwen3.5-0.8B-GGUF/resolve/main/'
-    'Qwen3.5-0.8B-Q4_K_M.gguf?download=true';
+    'https://huggingface.co/unsloth/Qwen3.6-27B-GGUF/resolve/main/'
+    'Qwen3.6-27B-UD-Q4_K_XL.gguf?download=true';
 
 /// Builds the CLI argument parser.
 ArgParser buildServerCliArgParser() {
@@ -12,7 +12,7 @@ ArgParser buildServerCliArgParser() {
       'model',
       abbr: 'm',
       defaultsTo: _defaultModelUrl,
-      help: 'Path or URL to a GGUF model.',
+      help: 'Path, URL, or hf:// model source.',
     )
     ..addOption(
       'model-id',
@@ -35,26 +35,14 @@ ArgParser buildServerCliArgParser() {
     )
     ..addOption(
       'context-size',
-      defaultsTo: '4096',
+      defaultsTo: '16384',
       help: 'Model context size in tokens.',
     )
     ..addOption(
       'gpu-layers',
       defaultsTo: '${ModelParams.maxGpuLayers}',
       help: 'Number of layers to offload to GPU.',
     )
-    ..addFlag(
-      'enable-tool-execution',
-      defaultsTo: false,
-      help:
-          'Enable server-side execution loop for model-emitted tool calls. '
-          'This example includes built-in mock handlers for common demo tools.',
-    )
-    ..addOption(
-      'max-tool-rounds',
-      defaultsTo: '5',
-      help: 'Maximum tool-call rounds per request when execution is enabled.',
-    )
     ..addFlag(
       'log',
       abbr: 'g',

example/llamadart_server/lib/src/bootstrap/cli/server_cli_config.dart

@@ -26,12 +26,6 @@ class ServerCliConfig {
   /// Whether verbose Dart/request logs are enabled.
   final bool enableDartLogs;
 
-  /// Whether server-side tool execution is enabled.
-  final bool enableToolExecution;
-
-  /// Maximum tool-call rounds per request.
-  final int maxToolRounds;
-
   /// Creates a parsed runtime config.
   const ServerCliConfig({
     required this.modelInput,
@@ -42,8 +36,6 @@ class ServerCliConfig {
     required this.contextSize,
     required this.gpuLayers,
     required this.enableDartLogs,
-    required this.enableToolExecution,
-    required this.maxToolRounds,
   });
 
   /// Whether auth is enabled.

example/llamadart_server/lib/src/bootstrap/cli/server_cli_config_parser.dart

@@ -12,14 +12,6 @@ ServerCliConfig parseServerCliConfig(ArgResults results) {
     throw ArgumentError('Invalid --host value: $host');
   }
 
-  final maxToolRounds = _parseIntOption(
-    results['max-tool-rounds'] as String,
-    'max-tool-rounds',
-  );
-  if (maxToolRounds < 1) {
-    throw ArgumentError('`max-tool-rounds` must be >= 1.');
-  }
-
   return ServerCliConfig(
     modelInput: results['model'] as String,
     modelId: results['model-id'] as String,
@@ -32,8 +24,6 @@ ServerCliConfig parseServerCliConfig(ArgResults results) {
     ),
     gpuLayers: _parseIntOption(results['gpu-layers'] as String, 'gpu-layers'),
     enableDartLogs: results['log'] as bool,
-    enableToolExecution: results['enable-tool-execution'] as bool,
-    maxToolRounds: maxToolRounds,
   );
 }