Skip to content

Commit 9faf202

Browse files
beersoccerbeersoccer
authored
chore: release v0.3.0 — checkpoint/lock reliability, provider gate, docs update
chore: release v0.3.0 — checkpoint/lock reliability, provider gate, docs update
2 parents f55a5eb + 45f5d0c commit 9faf202

8 files changed

Lines changed: 193 additions & 41 deletions

File tree

‎.cursor/AGENTS.md‎

Lines changed: 129 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ mem0_dify_plugin/
1313
│ ├── mem0ai.py # Mem0Provider: credential validation, tool interface
1414
│ └── mem0ai.yaml # Provider configuration schema
1515
│
16-
├── tools/ # 10 Dify tools (each: .py implementation + .yaml definition)
16+
├── tools/ # 12 Dify tools (each: .py implementation + .yaml definition)
1717
│ ├── add_memory.py/.yaml # Add/update memories
1818
│ ├── search_memory.py/.yaml # Search with filters (AND/OR, top_k)
1919
│ ├── get_memory.py/.yaml # Get memory by ID
@@ -23,46 +23,150 @@ mem0_dify_plugin/
2323
│ ├── delete_all_memories.py/.yaml # Batch delete with filters
2424
│ ├── get_memory_history.py/.yaml # View change history
2525
│ ├── extract_long_term_memory.py/.yaml # Extract semantic/episodic/procedural memories from Dify history (async task)
26-
│ └── check_extraction_status.py/.yaml # Check status and progress of async extraction tasks
26+
│ ├── check_extraction_status.py/.yaml # Check status and progress of async extraction tasks
27+
│ ├── get_user_checkpoint.py/.yaml # Inspect extraction checkpoint state for a user/app
28+
│ └── forget_memories.py/.yaml # Forget low-retention memories and clean stale checkpoints
2729
│
2830
└── utils/ # Shared utilities
29-
├── mem0_client.py # Mem0 client adapter (sync/async, connection pooling)
31+
├── mem0_client.py # Mem0 client adapter (sync/async, connection pooling, keepalive)
3032
├── config_builder.py # Builds Mem0 config from Dify credentials
3133
├── constants.py # Timeouts, concurrency limits, result shapes
32-
├── logger.py # Centralized logging (Dify plugin logger)
33-
└── helpers.py # Common utilities (timeout parsing, timestamps)
34+
├── logger.py # Centralized logging (Dify plugin logger, dynamic log level)
35+
├── helpers.py # Common utilities (timeout parsing, timestamps)
36+
├── checkpoint.py # Checkpoint read/write managers (sync + async, add-first-then-delete)
37+
├── distributed_lock.py # Mem0-backed distributed lock (read-after-write verification, TTL cleanup)
38+
├── extraction.py # UserCheckpoint / ConversationCheckpoint data models
39+
├── extraction_task.py # Async task runner and task status persistence
40+
├── extraction_helpers.py # Shared helpers for extraction pipeline
41+
├── mem0_extraction.py # Smart classification + semantic/episodic/procedural extraction pipeline
42+
├── memory_forgetting.py # EWMA quality + Ebbinghaus retention forgetting curve logic
43+
├── access_log.py # Per-user/per-app access log managers (drives forgetting curve)
44+
├── score_utils.py # Score mode inference (distance vs similarity) and normalization
45+
├── task_status.py # Async task status storage and retrieval
46+
├── task_tracker.py # Background task queue monitoring
47+
├── pgvector_config.py # PGVector connection string and pool configuration
48+
├── dify_client.py # Dify REST API client (conversations, messages, retry)
49+
├── message_utils.py # Message format conversion utilities
50+
├── retry.py # Exponential backoff retry decorator
51+
├── resource_cleanup.py # Vector/graph store and pool cleanup helpers
52+
├── connection_keepalive.py # Heartbeat thread preventing TCP silent timeouts
53+
├── background_loop.py # Process-wide asyncio event loop management
54+
├── queue_monitor.py # Background task queue overload monitoring
55+
├── prompts.py # Memory extraction prompts (classification + subtype)
56+
└── mem0_client_llm_compat.py # LLM compatibility shim for structured providers
3457
```
3558

3659
## Key Architectural Patterns
3760

3861
### Tools Layer
3962
- Each tool implements `Tool._invoke()` interface
4063
- Supports both sync and async execution modes
41-
- Handles timeouts gracefully (default: 30s, configurable)
42-
- Returns structured JSON responses with consistent error format
64+
- Write ops (Add/Update/Delete) are non-blocking in async mode; Read ops always wait
65+
- Returns structured JSON responses with consistent format
4366

4467
### Client Management
45-
- `mem0_client.py` manages Mem0 client lifecycle
46-
- Singleton instances per configuration
47-
- Connection pooling for async operations
48-
- Background task queue for write operations
49-
- Graceful shutdown with timeout handling
68+
- `mem0_client.py` manages Mem0 client lifecycle (singleton per configuration)
69+
- Connection pooling for async operations (psycopg3 pool with TCP keepalive)
70+
- Background task queue for write operations with overload protection
71+
- `ConnectionKeepAlive` daemon thread prevents TCP silent timeouts to LLM/vector services
72+
73+
### Checkpoint & Lock
74+
- `checkpoint.py` implements add-first-then-delete saves to prevent data loss on failure
75+
- `AsyncCheckpointManager.load()` restores all resume cursor fields (including `resume_conversation_cursor`, `resume_run_at`, `resume_start_time`)
76+
- `distributed_lock.py` uses read-after-write verification after persisting; earliest `acquired_at` wins on contention
5077

5178
### Configuration Flow
5279
1. Dify passes JSON credentials to tool
53-
2. `config_builder.py` converts to Mem0 config format
54-
3. Client initialization with connection pooling
55-
4. Config validation before first use
80+
2. `config_builder.py` converts to Mem0 config format (provider name must be canonical)
81+
3. `pgvector_config.py` normalizes connection string, adds TCP keepalive params
82+
4. Client initialization with connection pooling
83+
5. Config validation before first use
5684

5785
### Error Handling
58-
- All tools return structured error messages:
59-
```json
60-
{
61-
"success": false,
62-
"error": "Description",
63-
"error_code": "ERROR_TYPE"
64-
}
65-
```
66-
- Operations have mandatory timeouts
86+
- All tools return structured error messages with status/results fields
87+
- Operations have mandatory timeouts (Read: 5s default, Write: 15s default)
6788
- Failed operations log details without exposing secrets
68-
- Partial failures return detailed per-item status
89+
- Partial failures return detailed per-item status
90+
91+
## Test Structure
92+
93+
```
94+
tests/
95+
├── conftest.py # Root conftest (markers, env setup)
96+
├── unit/ # Pure logic tests, no external services (471 tests as of v0.3.0)
97+
│ ├── provider/
98+
│ │ └── test_mem0_provider_validation.py # Provider credential validation paths
99+
│ ├── tools/
100+
│ │ ├── test_add_memory_overload_guard.py
101+
│ │ ├── test_check_extraction_status.py
102+
│ │ ├── test_extract_long_term_memory.py
103+
│ │ ├── test_extraction_async.py
104+
│ │ ├── test_extraction_parameters.py
105+
│ │ ├── test_forget_memories.py # forget_memories tool + lock cleanup
106+
│ │ ├── test_get_user_checkpoint.py
107+
│ │ ├── test_search_with_filters.py
108+
│ │ ├── test_time_range_expansion.py
109+
│ │ ├── test_token_truncation.py
110+
│ │ └── test_update_memory.py
111+
│ └── utils/
112+
│ ├── test_async_local_client_read_timeout.py
113+
│ ├── test_async_memory_init_compat.py
114+
│ ├── test_bg_task_tracking.py
115+
│ ├── test_checkpoint.py # 28+ tests; uses _run_async() thread isolation
116+
│ ├── test_config_builder_providers.py # 117 parametrized tests (provider compat gate + registry validation)
117+
│ ├── test_dify_client.py
118+
│ ├── test_dify_incremental_scan.py
119+
│ ├── test_distributed_lock.py # 28+ tests (acquire, contention, TTL cleanup)
120+
│ ├── test_mem0_client_config_override.py
121+
│ ├── test_mem0_client_llm_compat.py
122+
│ ├── test_mem0_extraction_logging.py
123+
│ ├── test_memory_forgetting.py
124+
│ ├── test_message_segmentation.py
125+
│ ├── test_message_utils.py
126+
│ ├── test_normalize_search_results.py
127+
│ ├── test_overload_guard_preenqueue.py
128+
│ ├── test_performance_user_ids.py
129+
│ ├── test_pgvector_config_defaults.py
130+
│ ├── test_pgvector_pool_max_waiting_default.py
131+
│ ├── test_prompts.py
132+
│ ├── test_retry.py
133+
│ ├── test_score_utils.py
134+
│ ├── test_task_status_async.py
135+
│ └── test_task_status_filters.py
136+
├── integration/ # Tests hitting real Dify API (requires live env)
137+
│ ├── conftest.py
138+
│ ├── test_dify_integration.py
139+
│ ├── test_dify_seed_api.py
140+
│ └── test_time_range_filtering.py
141+
├── acceptance/ # End-to-end workflow acceptance tests
142+
│ ├── conftest.py
143+
│ ├── test_memory_extraction_quality.py
144+
│ ├── test_workflow_memory_lifecycle.py
145+
│ └── test_workflow_plugin_smoke.py
146+
└── helpers/ # Shared test utilities
147+
├── artifacts.py
148+
├── dify_cleanup.py
149+
├── dify_env.py
150+
├── dify_seed.py
151+
├── mem0_cleanup.py
152+
└── workflow_runner.py
153+
```
154+
155+
### Running Tests
156+
157+
```bash
158+
# All unit tests (no external services required)
159+
conda run -n dify pytest tests/unit/ -q
160+
161+
# Integration tests (requires live Dify env)
162+
conda run -n dify pytest tests/integration/ -q
163+
164+
# Full suite
165+
conda run -n dify pytest -q
166+
```
167+
168+
### Test Design Notes
169+
170+
- **Provider compatibility gate** (`test_config_builder_providers.py`): 89 parametrized tests verify canonical provider name strings and critical config fields across all mainstream mem0 providers. 28 additional registry validation tests cross-check every provider name against mem0's live factory maps — these fail on `mem0ai` upgrades that drop or rename a provider.
171+
- **Checkpoint test isolation** (`test_checkpoint.py`): Uses `_run_async()` helper that spawns a dedicated thread and explicitly clears the inherited running-loop via `asyncio.events._set_running_loop(None)` before creating a fresh event loop. This prevents pytest-asyncio AUTO mode's C-level TSS inheritance from causing "Cannot run the event loop while another loop is running" when async tests run before these tests.
172+
- **Distributed lock tests** (`test_distributed_lock.py`): 28+ tests covering acquire, contention resolution (earliest `acquired_at` wins), TTL expiry, and `_clean_expired_locks()` behaviour.

‎.cursor/SPEC.md‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -200,3 +200,25 @@
200200
- README.md/CONFIG.md 已更新工具列表（10个工具）并补充 `extract_long_term_memory` 和 `check_extraction_status` 使用说明
201201
- AGENTS.md 已同步工具清单（10个工具）
202202
- CHANGELOG.md 和 manifest.yaml version 已更新到 v0.2.3
203+
204+
---
205+
206+
### 已实现（v0.3.0 — 2026-04-22）
207+
208+
- ✅ **Checkpoint 可靠性**：
209+
- `AsyncCheckpointManager.load()` 已恢复所有三个 resume cursor 字段（`resume_conversation_cursor`、`resume_run_at`、`resume_start_time`），修复了 async 模式下 `max_conversations_reached` 后续重跑无法续点的 P0 问题
210+
- Checkpoint `save()` 改为先 add 后 delete，避免 add 失败时误删旧 checkpoint；`load()` 通过取最新的记录安全处理临时重复条目
211+
- ✅ **Distributed Lock 可靠性**：
212+
- `acquire_lock()` 写入后执行 read-after-write 验证，新增 `_load_all_locks(limit=5)` 方法重读所有活跃锁；最早 `acquired_at` 获胜，失败方自删（解决 Mem0 最终一致性窗口内的竞争问题）
213+
- `forget_memories` 新增 `_clean_expired_locks()` 清理过期分布式锁记录；返回结果增加 `locks_cleaned` 字段
214+
- ✅ **测试隔离**：
215+
- `test_checkpoint.py` 中所有 async 路径改为 sync `def` + `_run_async()` 辅助函数，该函数在独立线程中通过 `asyncio.events._set_running_loop(None)` 清除继承的运行循环，彻底解决 pytest-asyncio AUTO 模式下 C 级 TSS 继承导致的 "Cannot run the event loop while another loop is running" 问题
216+
- ✅ **Provider 兼容门控**：
217+
- 新增 `test_config_builder_providers.py`（117 个参数化测试）：89 个测试覆盖 LLM、Embedder、Vector DB、Reranker 的规范 provider 名称和关键配置字段；28 个测试交叉检验 mem0 工厂注册表，在 `mem0ai` 升级后新增/删除/重命名 provider 时立即失败
218+
- 从 `LLM_CONFIGS` 中移除无效的 `mistral` provider（该 provider 不在 mem0 `LlmFactory` 注册表中）
219+
- ✅ **测试规模**：
220+
- 单元测试从 v0.2.12 的约 380 个增长到 **471 个**，全部通过（`pytest tests/unit/ -q`）
221+
- ✅ **文档更新**：
222+
- AGENTS.md 同步工具清单（12个工具）、完整 utils 模块列表、测试目录结构及设计说明
223+
- SPEC.md 增加 v0.3.0 验收标准章节
224+
- CHANGELOG.md、README.md、CONFIG.md、manifest.yaml、pyproject.toml 版本更新到 v0.3.0

‎CHANGELOG.md‎

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,21 @@
11
# Mem0 Dify Plugin - Changelog
22

3-
## Unreleased
3+
## Version 0.3.0 (2026-04-22)
4+
5+
### 🛠️ Reliability & Compatibility
6+
- **Checkpoint and distributed lock reliability** (PR #44):
7+
- Improved checkpoint write/read sequencing to eliminate rare race conditions under concurrent extraction tasks
8+
- Strengthened distributed lock acquisition and release paths for more predictable behavior under load
9+
- **Async checkpoint test isolation**:
10+
- Isolated async checkpoint tests from pytest-asyncio event loop pollution to prevent cross-test contamination in CI
11+
- Tests now use dedicated per-function event loops, making the suite more deterministic
12+
13+
### ✅ Tests
14+
- **Provider compatibility gate**:
15+
- Added compatibility validation tests for LLMs, embedders, vector DBs, and rerankers to catch unsupported provider configurations before runtime
16+
- **Mem0 registry validation**:
17+
- Added registry smoke tests that validate supported provider names against the mem0 registry
18+
- Removed the invalid `mistral` provider entry that caused silent failures during credential validation
419

520
---
621

‎CONFIG.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Mem0 Dify Plugin - Configuration Guide
22

3-
Last updated: 2026-04-16
3+
Last updated: 2026-04-22
44

55
This guide provides detailed installation and configuration instructions for the Mem0 Dify Plugin.
66

@@ -40,7 +40,7 @@ This guide provides detailed installation and configuration instructions for the
4040

4141
**Option B: Install from Package**
4242
1. Click `Upload Plugin` button
43-
2. Select the `.difypkg` file (e.g., `mem0ai-0.2.12.difypkg`)
43+
2. Select the `.difypkg` file (e.g., `mem0ai-0.3.0.difypkg`)
4444
3. Wait for upload and installation to complete
4545

4646
### Step 3: Verify Installation
@@ -1160,7 +1160,7 @@ For detailed upgrade instructions and field mapping, see [README.md - Upgrade Gu
11601160

11611161
**Problem**: Async validation fails with `object AsyncMemory can't be used in 'await' expression`
11621162
- **Solution**:
1163-
- Upgrade to plugin `v0.2.12` or later, which includes the latest compatibility and workflow-configuration improvements
1163+
- Upgrade to plugin `v0.3.0` or later, which includes the latest compatibility and workflow-configuration improvements
11641164
- Keep `mem0ai` within the documented support range: `>=1.0.2,<=1.0.11`
11651165
- If your Dify environment is offline, ensure the selected `mem0ai` version is available in the daemon cache or installation mirror
11661166

‎PR_TEMPLATE.md‎

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Plugin Submission Form
22

3-
Last updated: 2026-04-16
3+
Last updated: 2026-04-22
44

55
## 1. Metadata
66

@@ -31,13 +31,15 @@ Please provide the following metadata of your plugin to make it easier for the r
3131

3232
<!-- Please briefly describe the purpose of the new plugin or the updates made to the existing plugin -->
3333

34-
This submission updates the Mem0 Dify plugin to **v0.2.12** (self-hosted mode). Key changes are summarized below; detailed release notes and historical context are in [CHANGELOG.md](https://github.com/beersoccer/mem0_dify_plugin/blob/main/CHANGELOG.md).
34+
This submission updates the Mem0 Dify plugin to **v0.3.0** (self-hosted mode). Key changes are summarized below; detailed release notes and historical context are in [CHANGELOG.md](https://github.com/beersoccer/mem0_dify_plugin/blob/main/CHANGELOG.md).
3535

3636
### Key Updates
3737

38-
- **Dynamic Extraction Parameters**: Switched `extract_long_term_memory` inputs to `form: llm` so Dify workflows can bind system and upstream variables more reliably
39-
- **Cohere SDK Dependency**: Bundled `cohere>=6.1.0` by default to reduce manual setup for Cohere reranker users
40-
- **Documentation Refresh**: Updated README, CONFIG, changelog, and related release docs to reflect the new extraction parameter behavior and reranker setup guidance
38+
- **Checkpoint Reliability** (PR #44): Fixed async-mode resume-cursor restoration (`resume_conversation_cursor`, `resume_run_at`, `resume_start_time` now correctly round-trip through `AsyncCheckpointManager.load()`); switched save order to add-first-then-delete to prevent accidental data loss on write failure
39+
- **Distributed Lock Reliability** (PR #44): `acquire_lock()` now performs read-after-write verification using a new `_load_all_locks()` method; earliest `acquired_at` wins on contention, loser self-deletes; `forget_memories` gains `_clean_expired_locks()` for lock record hygiene
40+
- **Provider Compatibility Gate**: 117 parametrized unit tests covering canonical provider name strings and critical config fields across all mainstream mem0 LLMs, Embedders, Vector DBs, and Rerankers; 28 of these cross-check the mem0 factory registry directly to fail early on version-upgrade provider drift; removed invalid `mistral` provider entry
41+
- **Test Isolation Fix**: Async checkpoint tests refactored to sync `def` + `_run_async()` helper that spawns a dedicated thread and explicitly clears inherited running-loop state, eliminating pytest-asyncio AUTO mode cross-test contamination
42+
- **Test Suite Growth**: Unit tests grew from ~380 to **471 passing** tests
4143

4244
All API keys and credentials are stored locally in the user's Dify instance configuration and are not shared with any third parties. The plugin only communicates with services configured by the user (their LLM, embedding, and database services).
4345

0 commit comments

Comments
 (0)