fix(storage): 优化路径锁与语义刷新并发#2029
Merged
Merged
Conversation
Use exact path locks for source commits, tree locks only for lifecycle and schema scopes, and coalesce derived semantic writes to avoid stale summary overwrites under concurrent resource and memory updates.
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
PR Code Suggestions ✨No code suggestions found for the PR. |
19 tasks
This reverts commit d2fb274.
MaojiaSheng
approved these changes
May 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
这次 PR 解决的是本地压测中暴露出来的路径锁粒度问题:同一个目录下并发添加不同资源时,过去会因为父目录级锁过粗而互相阻塞,极端情况下返回 500。核心调整是把“源数据提交”和“语义派生刷新”拆成两件事:
docs/a.md只拿ExactPathLock(docs/a.md),不再因为同目录还有docs/b.md就互斥。TreeLock。.overview.md/.abstract.md这类派生文件不参与前台源数据提交锁,后台刷新通过 coalesce version 淘汰旧任务,最终写回时再短暂拿派生文件自己的ExactPathLock。举例:并发添加
viking://resources/docs/a.md、viking://resources/docs/b.md、viking://resources/docs/c.md时,三个请求分别锁a.md、b.md、c.md,可以并发提交;后台可能产生多个docs/摘要刷新任务,但只有最新版本会写回docs/.overview.md和docs/.abstract.md,旧任务写回前发现过期会丢弃结果。同时补齐了 memory / agent memory 的锁边界。固定文件名 schema,例如
profile.md、identity.md,使用具体文件的ExactPathLock;变量文件名 schema,例如events/{{ name }}.md、experiences/{{ experience_name }}.md,使用 schema 目录的TreeLock。agent experience 的source_trajectories回填也被纳入 experience schema 锁作用域,避免出现“experience 内容已经更新,但来源轨迹 metadata 还没补上”的中间状态被另一个 extraction 读取。Related Issue
N/A
Type of Change
Changes Made
PathLockEngine、ExactPathLock(path)、TreeLock(path),移除业务层对 point/source/subtree 命名的依赖,并统一LockContext/LockManager的 exact/tree/mv 调用方式。add_resource提交流程:TreeBuilder 只构建 final/temp URI 元信息,ResourceProcessor 在短ExactPathLock(final_path)内提交源数据;首次落盘后再交给生命周期TreeLock(final_path)保护后台解析、摘要、向量化期间的资源不被删除或移动。SemanticMsg增加 coalesce 信息,SemanticQueue 维护运行时版本;资源父目录摘要、memory 目录摘要、直接 write 后刷新都会淘汰旧任务,并在写.overview.md/.abstract.md时使用派生文件自己的ExactPathLock。source_trajectories系统 metadata 回填在 schema 锁释放前完成,fallback 路径也用短 exact lock 保护 read-modify-write。Testing
本地执行过的验证:
提交时 pre-commit 也通过了:
Checklist
Screenshots (if applicable)
N/A
Additional Notes
这次 PR 里的“最新版本判断”先使用运行时 coalesce version,没有引入持久化
dir_revision/exact_revision。这样已经可以解决同进程并发刷新中旧.overview.md覆盖新.overview.md的问题;跨重启的持久化版本控制可以后续单独设计。压测脚本默认指向
http://127.0.0.1:1935,用于配合本地ov.conf启动的 OpenViking server。脚本会输出中文 Markdown 报告、JSONL 明细和 CSV 汇总,便于比较 SDK 与 CLI 路径的并发表现。