✨ Feat: add active memory tools (StoreMemoryTool, SearchMemoryTool)#3197
Conversation
Release v2.2.0
- Implement StoreMemoryTool for explicit memory storage during agent reasoning - Implement SearchMemoryTool for on-demand memory retrieval during conversations - Integrate tools into agent creation flow (create_agent_info.py) - Register tools in nexent_agent.py and tools/__init__.py - Add MEMORY_OPERATION tool sign for proper categorization - Fix memory_core.py cache key to include event loop ID (prevents cross-loop conflicts) - Add comprehensive test coverage for both tools - Add procedural memory verification documentation Tools follow existing patterns: lazy imports, observer integration, error handling, and respect user memory preferences (agent_share_option, disabled_agent_ids).
There was a problem hiding this comment.
Pull request overview
This PR adds “active memory” capabilities to Nexent agents by introducing two new tools that can explicitly store and search long-term memory during an agent run, and wires them into agent/tool creation and tracing.
Changes:
- Added
StoreMemoryToolandSearchMemoryToolplus corresponding unit tests. - Integrated memory tools into runtime tool injection and tool classification/tracing.
- Adjusted memory-core caching behavior to avoid cross-event-loop binding errors.
Reviewed changes
Copilot reviewed 10 out of 11 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
sdk/nexent/core/tools/store_memory_tool.py |
New tool to actively store extracted facts into long-term memory with per-run rate limiting. |
sdk/nexent/core/tools/search_memory_tool.py |
New tool to actively query long-term memory across configured levels. |
sdk/nexent/core/tools/__init__.py |
Exports the new memory tools from the tools package. |
sdk/nexent/core/agents/nexent_agent.py |
Adds tool creation branch for memory tools and classifies SearchMemoryTool as a retriever for tracing. |
backend/agents/create_agent_info.py |
Injects active memory tools into the agent tool list when memory is enabled. |
sdk/nexent/core/utils/tools_common_message.py |
Adds a new ToolSign for memory operations and updates mappings. |
sdk/nexent/memory/memory_core.py |
Changes cache key derivation to incorporate event loop identity. |
test/sdk/core/tools/test_store_memory_tool.py |
New unit tests for store-memory behavior, limits, preferences, and error handling. |
test/sdk/core/tools/test_search_memory_tool.py |
New unit tests for search-memory behavior, preferences, formatting, and error handling. |
doc/procedural-memory-verification.md |
Adds documentation/report content about procedural memory dependency requirements. |
.gitignore |
Ignores pytest temp output and generated doc assets. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
| levels = ["user_agent", "agent"] | ||
| if self.memory_user_config.agent_share_option == "never": | ||
| levels.remove("agent") | ||
| if self.agent_id in getattr(self.memory_user_config, "disable_user_agent_ids", []): |
| memory_levels = ["tenant", "user", "agent", "user_agent"] | ||
| if self.memory_user_config.agent_share_option == "never": | ||
| memory_levels.remove("agent") | ||
| if self.agent_id in getattr(self.memory_user_config, "disable_agent_ids", []): |
| config_hash = _hash_config(memory_config) | ||
| loop = asyncio.get_event_loop() | ||
| cache_key = f"{config_hash}:{id(loop)}" | ||
|
|
||
| async with _get_cache_lock(): | ||
| if cache_key in _MEMORY_CACHE: |
| def forward(self, query: str, top_k: int = 5) -> str: | ||
| logger.info(f"[ACTIVE MEMORY] SearchMemoryTool invoked: query={query[:200]}, top_k={top_k}, user_id={self.user_id}, agent_id={self.agent_id}") | ||
| if self.observer: | ||
| running_prompt = self.running_prompt_zh if self.observer.lang == "zh" else self.running_prompt_en | ||
| self.observer.add_message("", ProcessType.TOOL, running_prompt) |
* add_greeting_fields_to_agent-develop * feat(knowledge-base): add preserve_source_file and post-index source cleanup Let knowledge bases opt out of keeping uploaded MinIO copies after indexing while retaining Elasticsearch chunks for retrieval. Default behavior remains preserve_source_file=true for backward compatibility. - Add preserve_source_file column (init.sql + v2.2.0_0601 migration) - Accept preserve_source_file on create/update and northbound/vector APIs - Support document DELETE scope=source_only and source_available in listings - Run cleanup_source Celery task when preserve_source_file is false - UI: create-KB toggle, list tag, knowledge-base preview when copy is missing - Update vector-database SDK docs and backend tests * test(data_process): stub knowledge_db, redis_service, and redis in test_worker Align setup_mocks_for_worker with test_tasks so importing backend.data_process.worker loads package __init__ without real DB/redis deps. * test(data_process): shim cleanup_source for submit_process_forward_chain tests * remove duplicate import * fix: update unit tests for greeting_message and example_questions fields * add init.sql to sonar.properites * ♻️ Improvement: API to MCP conversion service supports configuring headers. (#3194) * ♻️ Improvement: API to MCP conversion service supports configuring headers. [Specification Details] 1. Front-end and back-end modifications * ♻️ Improvement: API to MCP conversion service supports configuring headers. [Specification Details] 1. Modify the frontend, after adding, set the HTTP headers to empty. 2. Modify test cases. * ♻️ Improvement: Enhance processing of ES index names in memory banks. (#3196) [Specification Details] 1. Replace all symbols in the index name that do not meet the rules with "_". 2. Modify test cases. * feat: add active memory tools (StoreMemoryTool, SearchMemoryTool) (#3197) - Implement StoreMemoryTool for explicit memory storage during agent reasoning - Implement SearchMemoryTool for on-demand memory retrieval during conversations - Integrate tools into agent creation flow (create_agent_info.py) - Register tools in nexent_agent.py and tools/__init__.py - Add MEMORY_OPERATION tool sign for proper categorization - Fix memory_core.py cache key to include event loop ID (prevents cross-loop conflicts) - Add comprehensive test coverage for both tools - Add procedural memory verification documentation Tools follow existing patterns: lazy imports, observer integration, error handling, and respect user memory preferences (agent_share_option, disabled_agent_ids). Co-authored-by: Dallas98 <40557804+Dallas98@users.noreply.github.com> * 🐛 Bugfix: skill names and descriptions never load to context (#3205) * 🐛 Bugfix: skill names and descriptions never load to context * 🐛 Bugfix: skill names and descriptions never load to context * 🐛 Bugfix: skill names and descriptions never load to context * 🐛 Bugfix: official skills not copied to target directory * 🐛 Bugfix: official skills not copied to target directory * Feat: add selected count badges to tool/skill pool labels (#3206) Co-authored-by: chase <byzhangxin11@126.com> * 🐛 Bugfix: Fix attribution error when tool calling error (#3208) * ✨ Feat: Add support for Word document generation, preview, and download (#3191) * Feat: Add support for Word document generation, preview, and download * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Restrict uploads to a known safe workspace/output directory * 修改单元测试 * 修复单元测试 * Bugfix: Store uploaded files in Minio for conversation messages to enable file visibility in history --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * ✨Feat:Enhance prompt optimization by integrating openjiuwen and fix related bugs (#3190) * ✨Feat:add prompt optimization * 🐛Bugfix: dockerbuild failed when running pipefail in python3_11 * 🔨Optimize: Optimize prompt optimization display page and interaction methods * 🐛Bugfix: fix dependencies replication * 🎨:Optimize frontend prompts and loading interface * 🔧 Refactor: Update imports and remove redundant ENABLE_JIUWEN_SDK import in prompt_service.py * 🔧 Refactor: Correct import path for NexentCapabilityError and enhance test coverage for prompt optimization service * 🔧 Refactor: Update import paths for exception handling and improve logging formatting in prompt_service.py * 🔧 Refactor: Simplify lazy imports in jiuwen_sdk_adapter.py and update import paths in prompt_service.py * 🔧 Refactor: Enhance Jiuwen SDK adapter handling and improve test stubs in prompt_service.py and related test files * 🧪test:Pydantic model for PromptTemplateRequest in test_prompt_template_app.py * 🔧 Refactor: Remove unnecessary dependency exclusions from pyproject.toml * 🔧 Update: Upgrade huggingface_hub dependency version in pyproject.toml * 🔧 Update: Exclude unnecessary transitive dependencies and adjust huggingface_hub version in pyproject.toml * 🔧 Test: Add mock modules for unstructured inference and set up package paths in test files * 🔧 Test: Enhance test setup by adding optional SDK mocks and cleaning up module imports in data processing tests * 🔧 Test: Consolidate mock module setup for unstructured inference across multiple test files * 🔧 Test: Remove unused optional SDK mocks from test configuration * 🔧 Refactor: Clean up imports and enhance dynamic loading of fastmcp components in Docker client * 📦update:sdk dependence update * Add CAS SSO integration and improve logout handling (#3072) * feat: add CAS SSO integration * Skip CAS logout when CAS_LOGOUT_URL is unset * 取消转义 * Improve CAS logout handling and confirm user logout * Disable account deletion for CAS users * Add CAS session init SQL and k8s config * clean code * Remove agent guardrails design doc from tracking * 补充文档 --------- Co-authored-by: hhhhsc <name> * 🐛Bugfix: Remove unnecessary dependency exclusions and upgrade huggingface_hub version in pyproject.toml (#3211) * refactor: move current time from system prompt to user message for prompt cache stability (#3203) Remove {{time}} from all 4 prompt YAML templates (manager/managed × en/zh) and strip time_str from the context_utils pipeline (_format_app_context, build_skeleton_header_component, build_context_components, build_app_context_string). Also remove time from create_agent_info render kwargs and build_context_components call. In CoreAgent.run, prepend [Current time: ...] to self.task so the timestamp travels with the user message instead of being baked into the system prompt. This makes the rendered system prompt fully deterministic per (agent_id, tenant_id, version_no, language) — enabling prompt/KV cache hits across requests for the same agent config. Sync test_context_utils.py: drop time_str= from 3 test cases. Remove unused datetime imports from context_utils.py and create_agent_info.py. * 🐛 Bugfix: Fixed the issue of being unable to add MCP services via containerization. (#3213) [Specification Details] 1. Modify the DEFAULT_NETWORK_NAME when starting the MCP service in the container to match the name in docker-compose. 2. Modify the parameters passed to the add_mcp_service method; custom_headers defaults to None. * 🐛 Bugfix: Fixed the issue where uploaded text files could not be parsed during a session. (#3219) * 🐛 Bugfix: Fixed the issue where uploaded text files could not be parsed during a session. [Specification Details] 1. The return parameter of the file_process method has changed and needs to be unpacked. * 🐛 Bugfix: Fixed the issue where uploaded text files could not be parsed during a session. [Specification Details] 1. Modify test case. * 🐛 Bugfix: Fixed an issue where the MCP service could not be added correctly after updating the FastMCP version. (#3222) [Specification Details] 1. Add `kwargs` to the `create_httpx_client` function to accept all additional parameters. * 🐛 Bugfix: Fix incomplete display of tenant resources page after window resize (#3215) * Move non-shadcn ui component to other folder * Bugfix: Fix incomplete display of tenant resources page after window resize * Bugfix: Fix incomplete display of tenant resources page after window resize * Add agent marketplace repository and version pinning for sub-agents (#3239) * feat: add agent marketplace repository and pin sub-agent versions at publish Introduce ag_agent_repository_t with list/status/publish/import APIs for frozen agent snapshots. Pin selected_agent_version_no on agent relations when publishing so sub-agents resolve to a fixed version at runtime. Extend agent export/import to bundle skills in ZIP payloads and add embedding model fallback when no model name is provided. * feat: add agent marketplace repository and pin sub-agent versions at publish Introduce ag_agent_repository_t with list/status/publish/import APIs for frozen agent snapshots. Pin selected_agent_version_no on agent relations when publishing so sub-agents resolve to a fixed version at runtime. Extend agent export/import to bundle skills in ZIP payloads and add embedding model fallback when no model name is provided. * feat: add agent marketplace repository and pin sub-agent versions at publish Introduce ag_agent_repository_t with list/status/publish/import APIs for frozen agent snapshots. Pin selected_agent_version_no on agent relations when publishing so sub-agents resolve to a fixed version at runtime. Extend agent export/import to bundle skills in ZIP payloads and add embedding model fallback when no model name is provided. * feat: add agent marketplace repository and pin sub-agent versions at publish Introduce ag_agent_repository_t with list/status/publish/import APIs for frozen agent snapshots. Pin selected_agent_version_no on agent relations when publishing so sub-agents resolve to a fixed version at runtime. Extend agent export/import to bundle skills in ZIP payloads and add embedding model fallback when no model name is provided. * feat(agent): add verification configuration for agents and update related components (#3174) * feat(agent): add verification configuration for agents and update related components * feat(model): update model type labels and add monitoring dashboard translations * 🐛 Bugfix: Fix inability to select agent from agent space to edit (#3240) * Move non-shadcn ui component to other folder * Bugfix: Fix incomplete display of tenant resources page after window resize * Bugfix: Fix incomplete display of tenant resources page after window resize * Bugfix: Fix inability to select agent from agent space to edit * Bugfix: Display correct version info when viewing agent details * Update data agent and ME CAS integration documentation (#3242) * 补充dataagent对接文档 * 补充ME cas对接文档 * 补充ME cas对接文档 --------- Co-authored-by: hhhhsc <name> * ✨ Add several northbound apis (#3223) * ✨ Add several northbound apis * ✨ Add several northbound apis * ✨ Add several northbound apis * ✨ Add several northbound apis * ✨ Add several northbound apis * refactor: simplify deployment script by removing unused variables and functions (#3245) * feat(agent): add verification configuration for agents and update related components * feat(model): update model type labels and add monitoring dashboard translations * refactor(build_offline_package): simplify deployment script by removing unused variables and functions * 🐛 Bugfix: Adjust agent detail UI layout to accommodate newly added "self-verification" field (#3246) * Move non-shadcn ui component to other folder * Bugfix: Fix incomplete display of tenant resources page after window resize * Bugfix: Fix incomplete display of tenant resources page after window resize * Bugfix: Fix inability to select agent from agent space to edit * Bugfix: Display correct version info when viewing agent details * Bugfix: Adjust agent detail UI layout to accommodate newly added "self-verification" field * 补充sql (#3248) * 补充sql * 扩大limit限制 * 🐛 Bugfix: Fixed an issue where the MCP service failed to start in a Kubernetes container. (#3254) [Specification Details] 1. Modify the pod naming logic to convert all non-compliant characters to -. 2. Modify test cases. * 🐛 Bugfix: knowledge_base_search_tool called with TypeError: argument of type 'FieldInfo' is not iterable (#3259) * 🐛 Bugfix: Fixed an issue where the one-click rename function failed after importing an agent. (#3258) [Specification Details] 1. The frontend does not pass `agent_id` when calling the `regenerate_name` API. * Bugfix: Exclude attachments from assistant when saving conversation history (#3261) * Bump APP_VERSION from v2.2.0 to v2.2.1 (#3268) The default setting for client-side self-validation is "False". --------- Co-authored-by: chase <byzhangxin11@126.com> Co-authored-by: Chenlifeng <174292121+Lifeng-Chen@users.noreply.github.com> Co-authored-by: Dallas98 <40557804+Dallas98@users.noreply.github.com> Co-authored-by: Jason Wang <56037774+JasonW404@users.noreply.github.com> Co-authored-by: Xia Yichen <iamjasonxia@126.com> Co-authored-by: JeffWu <45140512+jeffwu-1999@users.noreply.github.com> Co-authored-by: WMC001 <46217886+WMC001@users.noreply.github.com> Co-authored-by: xuyaqi <xuyaqist@gmail.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Co-authored-by: DongJiBao2001 <120021235+DongJiBao2001@users.noreply.github.com> Co-authored-by: hhhhsc701 <56435672+hhhhsc701@users.noreply.github.com> Co-authored-by: Dallas98 <990259227@qq.com> Co-authored-by: frr <64584192+wuyuanfr@users.noreply.github.com>
| if "user_agent" in memory_levels: | ||
| memory_levels.remove("user_agent") | ||
|
|
||
| try: |
There was a problem hiding this comment.
asyncio.run() 在 forward() 中调用,但 forward() 可能在已有事件循环的上下文中执行(如 agent 的 async run loop)。asyncio.run() 会创建新的事件循环,如果当前线程已有运行中的事件循环,会抛出 RuntimeError: asyncio.run() cannot be called from a running event loop。建议使用 asyncio.get_event_loop().run_until_complete() 或检查是否已有运行中的循环。
| ) | ||
| tool_list.append(search_tool_config) | ||
| logger.debug("Active memory tools appended to agent tool list") | ||
| except Exception as e: |
There was a problem hiding this comment.
[代码规范] except Exception: 过于宽泛,建议捕获更具体的异常类型,避免掩盖潜在错误。
| lines.append(f"[{i+1}] (score: {score:.2f}, level: {level}) {content}") | ||
| return "\n".join(lines) | ||
|
|
||
| except Exception as e: |
There was a problem hiding this comment.
[代码规范] except Exception: 过于宽泛,建议捕获更具体的异常类型,避免掩盖潜在错误。
| return "The information was already present in memory (no changes needed)." | ||
| return "Stored successfully:\n" + "\n".join(stored_facts) | ||
|
|
||
| except Exception as e: |
There was a problem hiding this comment.
[代码规范] except Exception: 过于宽泛,建议捕获更具体的异常类型,避免掩盖潜在错误。
| storage_client=tool_config.metadata.get("storage_client", []), | ||
| validate_url_access=validate_url_access, | ||
| **params) | ||
| elif class_name in ["StoreMemoryTool", "SearchMemoryTool"]: |
There was a problem hiding this comment.
StoreMemoryTool 和 SearchMemoryTool 通过无参构造 tool_class() 实例化,然后通过手动赋值设置属性(observer, memory_config, tenant_id 等)。这种方式绕过了构造函数验证,如果属性名拼写错误或遗漏,运行时才会暴露问题。建议通过构造函数参数传入这些依赖,或在实例化后添加属性完整性检查。
WMC001
left a comment
There was a problem hiding this comment.
Bug 1 (CRITICAL): asyncio.run() inside sync forward() methods creates event loop per call — defeats get_memory_instance caching
sdk/nexent/core/tools/store_memory_tool.py line ~682 and sdk/nexent/core/tools/search_memory_tool.py line ~569 both call asyncio.run(search_memory_in_levels(...)) from a synchronous forward() method.
Two problems compound:
-
Event loop created per call. smolagents' Tool base class requires
forward()to be synchronous, so each invocation offorward()callsasyncio.run(...), which creates a fresh event loop. TheAsyncMemoryinstance lives inside that loop, so all its socket pools, ES connections, and LLM clients are torn down and recreated on every single tool call. This is catastrophic for latency and connection-pool reuse. -
get_memory_instancecache is keyed byid(loop)(this same PR,memory_core.pyline ~741). Because eachforward()invocation runs in a brand-new event loop,id(loop)is always different, so the cache never hits — every tool call rebuilds the entire memory config (LLM, embedder, vector store) from scratch. The cache change is itself the cause of this issue: if the cache were still keyed by config hash alone, the previous loop's resources could be reused; withid(loop), every call is a cache miss. -
asyncio.run()fails if a loop is already running. Ifforward()is ever invoked from inside an async context (e.g. an async test or a future integration where the tool is awaited from another coroutine),asyncio.run()raisesRuntimeError: asyncio.run() cannot be called from a running event loop.
Fix options:
- Option A: Make the memory service synchronous (use
MemorynotAsyncMemory), and remove theid(loop)cache key. Simplest, no event-loop juggling. - Option B: Move the
asyncio.run()out offorward()entirely. Havecreate_local_toolor the agent runner precompute the sync-wrapped version once at agent init time (e.g.,tool.forward = sync(forward_async)), so the loop is created once per agent run. - Option C: Use a long-lived singleton loop running in a background thread (
loop.run_in_executorstyle) and dispatch async work viaasyncio.run_coroutine_threadsafe.
The current implementation will work but adds 1-2s of cold-start latency on every tool call due to ES + LLM client rebuild, and any future async-context caller will crash.
Bug 2 (HIGH): get_memory_instance cache key now misses every time after this PR
sdk/nexent/memory/memory_core.py line 740-742:
config_hash = _hash_config(memory_config)
loop = asyncio.get_event_loop()
cache_key = f"{config_hash}:{id(loop)}"If get_memory_instance is called from within the per-call asyncio.run(...) (Bug 1), each call gets a different loop id, so the cache never hits. Even ignoring Bug 1, the id(loop) approach is fragile: Python's event loops are recycled (the CPython runtime may reuse the memory address of a freed loop), so two distinct loops over time could collide on id().
Fix: key the cache by config_hash alone (the original behavior), or by (config_hash, tenant_id) if tenant-level isolation is needed.
Summary
Add active memory tools that allow agents to explicitly store and search memories during their ReAct reasoning loop, complementing the existing passive memory injection system.
Changes
New Tools
sdk/nexent/core/tools/store_memory_tool.py): Allows agents to explicitly save important information to long-term memory mid-conversation. Respects user preferences (agent_share_option, disabled_agent_ids), enforces a rate limit of 3 stores per run, and integrates with the observer pattern for UI feedback.sdk/nexent/core/tools/search_memory_tool.py): Allows agents to search long-term memory on-demand when they need context not available in the current conversation. Searches across all enabled memory levels (tenant, user, agent, user_agent) with configurable top_k.Integration
backend/agents/create_agent_info.py: Conditionally injects memory tools into the agent's tool list when memory is enabled. Tools are invisible in the tool management UI (not registered inag_tool_info_t) and auto-injected at runtime.sdk/nexent/core/agents/nexent_agent.py: Addedcreate_local_tool()branches for memory tools and_is_retriever_tool()classification for SearchMemoryTool (uses retriever tracing instead of tool tracing).sdk/nexent/core/tools/__init__.py: Registered both tools in exports.sdk/nexent/core/utils/tools_common_message.py: AddedToolSign.MEMORY_OPERATION = "n"to avoid collision with existing tool signs.Bug Fix
sdk/nexent/memory/memory_core.py: Fixed cache key fromconfig_hashto(config_hash, id(event_loop))to prevent cross-event-loop binding errors when the memory instance is reused across different async contexts.Tests
test/sdk/core/tools/test_store_memory_tool.py: 17 tests covering invocation, rate limiting, preference filtering, error handling.test/sdk/core/tools/test_search_memory_tool.py: 13 tests covering search, preference filtering, error handling, result formatting.Design Decisions
memory_levelinput — system auto-selects levels based on user preferencesforward()to avoid import chain failures in test environmentsTesting