Native Tool Calling and Model Registry Cleanup and Better Runtime Info

.dockerignore

@@ -2,4 +2,7 @@
 node_modules
 bots/*/
 !bots/*
-keys.json
+keys.json
+settings_llm_providers.json
+results
+tests/results

.gitignore

@@ -3,10 +3,12 @@
 node_modules/
 package-lock.json
 code_records/
+docs/
 scratch.js
 bots/**/action-code/**
 bots/**/
 keys.json
+settings_llm_providers.json
 services/viaproxy/jars/**
 services/viaproxy/logs/**
 services/viaproxy/plugins/**
@@ -19,8 +21,10 @@ experiments/
 andy_*.json
 jill_*.json
 src/models/logs/*
+results/
+tests/results/
+src/mindcraft/public/trace-demo.json
 server_data/*
-results/*
 tasks/construction_tasks/test_multiagent_construction_tasks.json
 tasks/construction_tasks/train_multiagent_construction_tasks.json
 tasks/construction_tasks/test/**

FAQ.md

@@ -26,7 +26,7 @@
   - The bot will still get stuck occasionally, but not constantly.
 
 - `Why I added the api key but still prompted that the key can't be found?`
-  - Possible reason 1: Did not modify keys.example.json to keys.json.
+  - Possible reason 1: Did not modify settings_llm_providers.example.json to settings_llm_providers.json.
   - Possible reason 2: If you use vscode to edit, you need to `ctrl+s` to save the file for the changes to take effect.
   - Possible reason 3: Not setting the code path correctly in setting.js, use andy.js by default. 
 

README.md

@@ -35,7 +35,7 @@ Do not connect this bot to public servers with coding enabled. This project allo
 
 2. Download the [latest release](https://github.com/mindcraft-bots/mindcraft/releases/latest) and unzip it, or clone the repository.
 
-3. Rename `keys.example.json` to `keys.json` and fill in your API keys (you only need one). The desired model is set in `andy.json` or other profiles. For other models refer to the table below.
+3. Rename `settings_llm_providers.example.json` to `settings_llm_providers.json` and fill in your API keys (you only need one). The desired model is set in `andy.json` or other profiles. For other models refer to the table below.
 
 4. In terminal/command prompt, run `npm install` from the installed directory
 
@@ -71,7 +71,6 @@ You can configure the agent's name, model, and prompts in their profile like `an
 | `huggingface` | `HUGGINGFACE_API_KEY` | [docs](https://huggingface.co/models) |
 | `novita` | `NOVITA_API_KEY` | [docs](https://novita.ai/model-api/product/llm-api?utm_source=github_mindcraft&utm_medium=github_readme&utm_campaign=link) |
 | `openrouter` | `OPENROUTER_API_KEY` | [docs](https://openrouter.ai/models) |
-| `glhf` | `GHLF_API_KEY` | [docs](https://glhf.chat/user-settings/api) |
 | `hyperbolic` | `HYPERBOLIC_API_KEY` | [docs](https://docs.hyperbolic.xyz/docs/getting-started) |
 | `vllm` | n/a | n/a |
 | `cerebras` | `CEREBRAS_API_KEY` | [docs](https://inference-docs.cerebras.ai/introduction) |
@@ -81,11 +80,8 @@ You can configure the agent's name, model, and prompts in their profile like `an
 
 For more comprehensive model configuration and syntax, see [Model Specifications](#model-specifications).
 
-For local models we support [ollama](https://ollama.com/) and we provide our own finetuned models for you to use. 
-To install our models, install ollama and run the following terminal command:
-```bash
-ollama pull sweaterdog/andy-4:micro-q8_0 && ollama pull embeddinggemma
-```
+For local models, we recommend you use **LM Studio** for the Andy series of models. Ollama breaks current models, and should be avoided.
+Please see our [huggingface page for more info.](https://huggingface.co/collections/Mindcraft-CE)
 
 ## Online Servers
 To connect to online servers your bot will need an official Microsoft/Minecraft account. You can use your own personal one, but will need another account if you want to connect too and play with it. To connect, change these lines in `settings.js`:
@@ -144,7 +140,7 @@ If you want more optimization and automatic launching of the minecraft world, yo
 If you intend to `allow_insecure_coding`, it is a good idea to run the app in a docker container to reduce risks of running unknown code. This is strongly recommended before connecting to remote servers, although still does not guarantee complete safety.
 
 ```bash
-docker build -t mindcraft . && docker run --rm --add-host=host.docker.internal:host-gateway -p 8080:8080 -p 3000-3003:3000-3003 -e SETTINGS_JSON='{"auto_open_ui":false,"profiles":["./profiles/gemini.json"],"host":"host.docker.internal"}' --volume ./keys.json:/app/keys.json --name mindcraft mindcraft
+docker build -t mindcraft . && docker run --rm --add-host=host.docker.internal:host-gateway -p 8080:8080 -p 3000-3003:3000-3003 -e SETTINGS_JSON='{"auto_open_ui":false,"profiles":["./profiles/gemini.json"],"host":"host.docker.internal"}' --volume ./settings_llm_providers.json:/app/settings_llm_providers.json --name mindcraft mindcraft
 ```
 or simply
 ```bash

andy.json

@@ -1,6 +1,19 @@
 {
     "name": "andy",
-
-    "model": "gpt-5.4-mini"
-
-}
+    "model": {
+        "provider": "openai",
+        "model": "gpt-5.4-mini"
+    },
+    "embedding": {
+        "provider": "",
+        "model": ""
+    },
+    "code_model": {
+        "provider": "",
+        "model": ""
+    },
+    "vision_model": {
+        "provider": "",
+        "model": ""
+    }
+}

develop-diff-summary.md

@@ -0,0 +1,261 @@
+# Diff Summary vs develop
+
+- Generated: 2026-05-03 20:30:51
+- Branch: `native-tool` (`4ffd9f9`)
+- Compared against: `develop` (`8acbd90`)
+- Diff command: `git diff develop` (includes current working tree / uncommitted changes)
+
+## Overall
+
+| Metric | Value |
+|---|---:|
+| Files changed | 133 |
+| Added files | 58 |
+| Modified files | 52 |
+| Deleted files | 23 |
+| Renamed files | 0 |
+| Shortstat | `133 files changed, 15090 insertions(+), 2787 deletions(-)` |
+
+> Note: committed-only branch diff (`git diff develop...HEAD`) is:
+> `133 files changed, 14822 insertions(+), 2766 deletions(-)`
+> The difference comes from local uncommitted working-tree changes listed below.
+
+## Current uncommitted changes
+
+- ` M profiles/defaults/prompts/bot_responder.md`
+- ` M settings.js`
+- ` M src/agent/coder.js`
+- ` M src/mindcraft/public/index.html`
+- ` M src/models/codex_chatgpt/client.js`
+- ` M src/models/prompter.js`
+- ` M tests/agent_native_text_policy.test.js`
+- ` M tests/codex_chatgpt.test.js`
+- ` M tests/native_prompt_hygiene.test.js`
+
+## Directory / area churn
+
+| Area | Files | Additions | Deletions | Churn |
+|---|---:|---:|---:|---:|
+| `src/` | 63 | 9201 | 2266 | 11467 |
+| `tests/` | 16 | 4247 | 0 | 4247 |
+| `profiles/` | 39 | 749 | 453 | 1202 |
+| `scripts/` | 3 | 496 | 0 | 496 |
+| `settings_llm_providers.example.json` | 1 | 285 | 0 | 285 |
+| `settings.js` | 1 | 62 | 27 | 89 |
+| `andy.json` | 1 | 17 | 4 | 21 |
+| `keys.example.json` | 1 | 0 | 19 | 19 |
+| `package.json` | 1 | 7 | 4 | 11 |
+| `eslint.config.js` | 1 | 6 | 3 | 9 |
+| `tasks/` | 1 | 4 | 4 | 8 |
+| `.gitignore` | 1 | 5 | 1 | 6 |
+| `.dockerignore` | 1 | 4 | 1 | 5 |
+| `README.md` | 1 | 2 | 3 | 5 |
+| `main.js` | 1 | 4 | 1 | 5 |
+| `FAQ.md` | 1 | 1 | 1 | 2 |
+
+## Top churn files
+
+| File | Additions | Deletions | Churn | Status |
+|---|---:|---:|---:|---|
+| `src/mindcraft/public/index.html` | 2649 | 41 | 2690 | M |
+| `tests/agent_native_text_policy.test.js` | 1127 | 0 | 1127 | A |
+| `tests/codex_chatgpt.test.js` | 1051 | 0 | 1051 | A |
+| `src/models/codex_chatgpt/client.js` | 609 | 0 | 609 | A |
+| `src/agent/history.js` | 574 | 29 | 603 | M |
+| `src/models/native_tools.js` | 584 | 0 | 584 | A |
+| `src/models/codex_chatgpt/auth.js` | 500 | 0 | 500 | A |
+| `src/agent/agent.js` | 436 | 46 | 482 | M |
+| `tests/chat_history_trace.test.js` | 439 | 0 | 439 | A |
+| `src/models/prompter.js` | 310 | 79 | 389 | M |
+| `tests/openai_compatible.test.js` | 389 | 0 | 389 | A |
+| `tests/native_tools.test.js` | 370 | 0 | 370 | A |
+| `src/mindcraft/public/chat_trace_projector.js` | 351 | 0 | 351 | A |
+| `src/models/openai_compatible.js` | 329 | 0 | 329 | A |
+| `src/models/codex_chatgpt/transport.js` | 288 | 0 | 288 | A |
+| `settings_llm_providers.example.json` | 285 | 0 | 285 | A |
+| `scripts/smoke/live_model_matrix.js` | 282 | 0 | 282 | A |
+| `src/mindcraft/mindserver.js` | 263 | 7 | 270 | M |
+| `src/models/google_generative_ai.js` | 266 | 0 | 266 | A |
+| `src/models/replicate.js` | 206 | 56 | 262 | M |
+| `profiles/defaults/_default.json` | 17 | 242 | 259 | M |
+| `src/models/codex_chatgpt/protocol.js` | 249 | 0 | 249 | A |
+| `tests/conversation_queue.test.js` | 188 | 0 | 188 | A |
+| `src/models/gemini.js` | 0 | 176 | 176 | D |
+| `src/models/_model_map.js` | 140 | 31 | 171 | M |
+| `src/agent/commands/tool_adapter.js` | 170 | 0 | 170 | A |
+| `scripts/smoke/live_function_call_smoke.js` | 169 | 0 | 169 | A |
+| `src/agent/state_snapshot.js` | 152 | 0 | 152 | A |
+| `src/models/gpt.js` | 0 | 147 | 147 | D |
+| `tests/llm_providers_config.test.js` | 143 | 0 | 143 | A |
+| `tests/mindserver_chat_history.test.js` | 138 | 0 | 138 | A |
+| `src/agent/conversation.js` | 94 | 43 | 137 | M |
+| `src/models/anthropic_messages.js` | 132 | 0 | 132 | A |
+| `src/agent/library/skills.js` | 110 | 17 | 127 | M |
+| `src/agent/react_message_manager.js` | 118 | 0 | 118 | A |
+| `src/models/openai_responses.js` | 117 | 0 | 117 | A |
+| `src/models/ollama.js` | 0 | 115 | 115 | D |
+| `src/models/hyperbolic.js` | 0 | 114 | 114 | D |
+| `profiles/defaults/prompts/coding.md` | 110 | 0 | 110 | A |
+| `tests/native_prompt_hygiene.test.js` | 98 | 0 | 98 | A |
+
+## Added files
+
+- `profiles/cerebras.json` (+19/-0)
+- `profiles/codex.json` (+25/-0)
+- `profiles/defaults/prompts/bot_responder.md` (+11/-0)
+- `profiles/defaults/prompts/coding.md` (+110/-0)
+- `profiles/defaults/prompts/conversing.md` (+34/-0)
+- `profiles/defaults/prompts/image_analysis.md` (+1/-0)
+- `profiles/defaults/prompts/saving_memory.md` (+48/-0)
+- `profiles/groq.json` (+19/-0)
+- `profiles/kimi.json` (+19/-0)
+- `profiles/minimax-cn.json` (+19/-0)
+- `profiles/minimax-intl.json` (+19/-0)
+- `profiles/novita.json` (+19/-0)
+- `profiles/ollama.json` (+19/-0)
+- `profiles/openrouter.json` (+19/-0)
+- `profiles/qwen-cn.json` (+20/-0)
+- `profiles/replicate.json` (+19/-0)
+- `profiles/siliconflow.json` (+19/-0)
+- `profiles/tasks/construction_prompt.md` (+14/-0)
+- `profiles/tasks/cooking_prompt.md` (+19/-0)
+- `profiles/tasks/crafting_prompt.md` (+14/-0)
+- `scripts/smoke/dev_native_tool_loop.js` (+45/-0)
+- `scripts/smoke/live_function_call_smoke.js` (+169/-0)
+- `scripts/smoke/live_model_matrix.js` (+282/-0)
+- `settings_llm_providers.example.json` (+285/-0)
+- `src/agent/commands/tool_adapter.js` (+170/-0)
+- `src/agent/react_message_manager.js` (+118/-0)
+- `src/agent/state_snapshot.js` (+152/-0)
+- `src/mindcraft/public/chat_trace_projector.js` (+351/-0)
+- `src/models/anthropic_messages.js` (+132/-0)
+- `src/models/azure_openai_responses.js` (+36/-0)
+- `src/models/codex_chatgpt.js` (+15/-0)
+- `src/models/codex_chatgpt/auth.js` (+500/-0)
+- `src/models/codex_chatgpt/client.js` (+609/-0)
+- `src/models/codex_chatgpt/constants.js` (+12/-0)
+- `src/models/codex_chatgpt/protocol.js` (+249/-0)
+- `src/models/codex_chatgpt/transport.js` (+288/-0)
+- `src/models/codex_chatgpt/utils.js` (+86/-0)
+- `src/models/google_generative_ai.js` (+266/-0)
+- `src/models/native_tools.js` (+584/-0)
+- `src/models/openai_compatible.js` (+329/-0)
+- `src/models/openai_responses.js` (+117/-0)
+- `src/models/token_usage.js` (+81/-0)
+- `tests/agent_native_text_policy.test.js` (+1127/-0)
+- `tests/chat_history_trace.test.js` (+439/-0)
+- `tests/codex_chatgpt.test.js` (+1051/-0)
+- `tests/conversation_queue.test.js` (+188/-0)
+- `tests/llm_providers_config.test.js` (+143/-0)
+- `tests/memory_summary_tool_history.test.js` (+25/-0)
+- `tests/mindserver_chat_history.test.js` (+138/-0)
+- `tests/mindserver_settings_spec.test.js` (+17/-0)
+- `tests/native_prompt_hygiene.test.js` (+98/-0)
+- `tests/native_tools.test.js` (+370/-0)
+- `tests/openai_compatible.test.js` (+389/-0)
+- `tests/profile_shape.test.js` (+97/-0)
+- `tests/prompt_markdown_refs.test.js` (+18/-0)
+- `tests/token_usage.test.js` (+50/-0)
+- `tests/tool_result_policy.test.js` (+64/-0)
+- `tests/vision_interpreter.test.js` (+33/-0)
+
+## Deleted files
+
+- `keys.example.json` (+0/-19)
+- `profiles/qwen.json` (+0/-17)
+- `src/models/azure.js` (+0/-32)
+- `src/models/cerebras.js` (+0/-61)
+- `src/models/claude.js` (+0/-87)
+- `src/models/deepseek.js` (+0/-59)
+- `src/models/gemini.js` (+0/-176)
+- `src/models/glhf.js` (+0/-71)
+- `src/models/gpt.js` (+0/-147)
+- `src/models/grok.js` (+0/-82)
+- `src/models/groq.js` (+0/-95)
+- `src/models/huggingface.js` (+0/-86)
+- `src/models/hyperbolic.js` (+0/-114)
+- `src/models/lmstudio.js` (+0/-74)
+- `src/models/mercury.js` (+0/-95)
+- `src/models/mistral.js` (+0/-94)
+- `src/models/novita.js` (+0/-71)
+- `src/models/ollama.js` (+0/-115)
+- `src/models/openrouter.js` (+0/-77)
+- `src/models/qwen.js` (+0/-80)
+- `src/models/vllm.js` (+0/-78)
+- `src/utils/examples.js` (+0/-83)
+- `src/utils/translator.js` (+0/-30)
+
+## Modified files
+
+### (root)
+
+- `.dockerignore` (+4/-1)
+- `.gitignore` (+5/-1)
+- `FAQ.md` (+1/-1)
+- `README.md` (+2/-3)
+- `andy.json` (+17/-4)
+- `eslint.config.js` (+6/-3)
+- `main.js` (+4/-1)
+- `package.json` (+7/-4)
+- `settings.js` (+62/-27)
+
+### profiles/
+
+- `profiles/andy-4-reasoning.json` (+16/-11)
+- `profiles/andy-4.json` (+16/-4)
+- `profiles/azure.json` (+13/-13)
+- `profiles/claude.json` (+17/-5)
+- `profiles/claude_thinker.json` (+14/-4)
+- `profiles/deepseek.json` (+17/-5)
+- `profiles/defaults/_default.json` (+17/-242)
+- `profiles/freeguy.json` (+18/-5)
+- `profiles/gemini.json` (+17/-6)
+- `profiles/gpt.json` (+16/-4)
+- `profiles/grok.json` (+17/-5)
+- `profiles/llama.json` (+17/-7)
+- `profiles/mercury.json` (+17/-6)
+- `profiles/mistral.json` (+17/-3)
+- `profiles/tasks/construction_profile.json` (+5/-37)
+- `profiles/tasks/cooking_profile.json` (+7/-6)
+- `profiles/tasks/crafting_profile.json` (+7/-67)
+- `profiles/vllm.json` (+15/-6)
+
+### src/
+
+- `src/agent/action_manager.js` (+32/-16)
+- `src/agent/agent.js` (+436/-46)
+- `src/agent/coder.js` (+40/-16)
+- `src/agent/commands/actions.js` (+14/-11)
+- `src/agent/commands/index.js` (+3/-2)
+- `src/agent/conversation.js` (+94/-43)
+- `src/agent/history.js` (+574/-29)
+- `src/agent/library/skill_library.js` (+5/-0)
+- `src/agent/library/skills.js` (+110/-17)
+- `src/agent/mindserver_proxy.js` (+13/-0)
+- `src/agent/modes.js` (+1/-0)
+- `src/agent/self_prompter.js` (+9/-3)
+- `src/agent/speak.js` (+63/-32)
+- `src/agent/vision/vision_interpreter.js` (+16/-4)
+- `src/mindcraft/mindcraft.js` (+5/-2)
+- `src/mindcraft/mindserver.js` (+263/-7)
+- `src/mindcraft/public/index.html` (+2649/-41)
+- `src/mindcraft/public/settings_spec.json` (+31/-12)
+- `src/models/_model_map.js` (+140/-31)
+- `src/models/prompter.js` (+310/-79)
+- `src/models/replicate.js` (+206/-56)
+- `src/process/agent_process.js` (+3/-3)
+- `src/utils/keys.js` (+35/-5)
+- `src/utils/text.js` (+54/-4)
+
+### tasks/
+
+- `tasks/evaluation_script.py` (+4/-4)
+
+## Review-oriented summary
+
+- Biggest review surface is `src/`, especially runtime/chat UI, native tool plumbing, model adapters, and agent history/message flow.
+- Test coverage was expanded substantially under `tests/` with new native-tool, provider, trace, queue, token-usage, and UI projection tests.
+- Provider layer changed from many provider-specific files toward registry-driven adapters and shared protocol implementations.
+- Prompt text moved into markdown files under `profiles/defaults/prompts/`, making prompt edits easier to review separately from code.
+- Local uncommitted files are included in this report because the diff target is current worktree vs `develop`.
+

eslint.config.js

@@ -14,8 +14,11 @@ export default [
       "no-floating-promise": noFloatingPromise,
     },
     languageOptions: {
-      globals: globals.browser,
-      ecmaVersion: 2021,
+      globals: {
+        ...globals.browser,
+        ...globals.node,
+      },
+      ecmaVersion: "latest",
       sourceType: "module",
     },
     rules: {
@@ -24,7 +27,7 @@ export default [
       "curly": "off",                   // Do not enforce the use of curly braces around blocks of code.
       "no-unused-vars": "off",          // Disable warnings for unused variables.
       "no-unreachable": "off",          // Disable warnings for unreachable code.
-      "require-await": "error",         // Disallow async functions which have no await expression
+      "require-await": "off",           // Many provider interfaces are intentionally async-compatible.
       "no-floating-promise/no-floating-promise": "error", // Disallow Promises without error handling or awaiting
     },
   },