-
Notifications
You must be signed in to change notification settings - Fork 476
Add agent memory search for OpenClaw and Hermes Agent using llama.cpp #2438
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
76e11dd
b7b7378
a66972f
083b470
aceb15c
1c56c66
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -100,9 +100,34 @@ openclaw onboard --non-interactive \ | |
|
|
||
| You can also run `openclaw onboard` interactively, select `custom-compatibility` with `openai`, and pass the same configuration. | ||
|
|
||
| ## Hermes | ||
| ### Local Memory Search for OpenClaw | ||
|
|
||
| [Hermes](https://hermes-agent.nousresearch.com/) works locally with llama.cpp. Define a default config as: | ||
| You can run local embedding models with Llama.cpp for your agent's memory search. To do so, make sure to have node-llama-cpp. | ||
|
|
||
| ```bash | ||
| npm i node-llama-cpp | ||
| ``` | ||
|
|
||
| Here's an example snippet to run [quantized EmbeddingGemma-300M](https://huggingface.co/ggml-org/embeddinggemma-300M-GGUF?show_file_info=embeddinggemma-300M-Q8_0.gguf) locally for memory search. OpenClaw automatically downloads and serves the model with the command below. | ||
|
|
||
| ```bash | ||
| openclaw config set agents.defaults.memorySearch.provider local | ||
| openclaw config set agents.defaults.memorySearch.local.modelPath "hf:ggml-org/embeddinggemma-300m-qat-q8_0-GGUF/embeddinggemma-300m-qat-Q8_0.gguf" | ||
|
pcuenca marked this conversation as resolved.
|
||
| ``` | ||
|
|
||
| Restart the gateway and validate. | ||
|
|
||
| ```bash | ||
| openclaw gateway restart | ||
| openclaw memory status | ||
| # Memory Search (main) | ||
| # Provider: local (requested: local) | ||
| # Model: hf:ggml-org/embeddinggemma-300m-qat-q8_0-GGUF/embeddinggemma-300m-qat-Q8_0.gguf | ||
| ``` | ||
|
|
||
| ## Hermes Agent | ||
|
|
||
| [Hermes Agent](https://hermes-agent.nousresearch.com/) works locally with llama.cpp. Define a default config as: | ||
|
|
||
| ```yaml | ||
| model: | ||
|
|
@@ -118,6 +143,30 @@ custom_providers: | |
| model: ggml-org/gemma-4-26B-A4B-it-GGUF:Q4_K_M | ||
| ``` | ||
|
|
||
| ### Local Memory Search for Hermes Agent | ||
|
|
||
| Hermes Agent consumes semantic search models through endpoints. Once you get your preferred embedding model up on endpoint 8080 with llama.cpp or the inference engine of your choice, add the following to `~/.hermes/config.yaml`. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. How does memory search work if you don't do anything? Because it's able to look up things in memory by default, AFAIU. Here's my output from the command you mention below. I haven't done anything about semantic search, but the first lines match what you said:
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. it does exact text search afaik, where it just does look-up from sessions. imo it's counterproductive |
||
|
|
||
| ```bash | ||
| auxiliary: | ||
| session_search: | ||
| base_url: "http://127.0.0.1:8080/v1" | ||
| api_key: "no-key-required" | ||
| model: "local-llama" # your model alias | ||
| timeout: 90 | ||
| max_concurrency: 1 | ||
| ``` | ||
|
|
||
| Check if this works, `none - built-in only` shows that no other memory plug-ins are used. The output below shows that local serving is active. | ||
|
|
||
| ```bash | ||
| $ hermes memory status | ||
| # Memory status | ||
| #──────────────────────────────────────── | ||
| # Built-in: always active | ||
| # Provider: (none — built-in only) | ||
|
pcuenca marked this conversation as resolved.
|
||
| ``` | ||
|
|
||
| ## OpenCode | ||
|
|
||
| [OpenCode](https://opencode.ai) works locally with llama.cpp. Define a `~/.config/opencode/opencode.json`: | ||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.