feat: score adaptive memory promotion

Author: hungrytech

ARCHITECTURE.md

@@ -84,6 +84,27 @@ records rendered into the final `<memory-data>` count. Same-provider/session/day
 replays are idempotent, and manual recalls without a consumer identity do not
 count. Explicit and adaptive memory pages do not feed their own counters, superseded
 sources are excluded, and each source's workspace remains the counting scope.
+Passing the hard gates is necessary but not sufficient. An explainable score must
+also meet `adaptive_memory_min_score` (default `0.65`). Its weighted contributions
+are session diversity `0.30`, UTC-day persistence `0.25`, final-context injection
+recurrence `0.25`, query-backed session ratio `0.10`, and provider diversity
+`0.10`. The first three ratios saturate at twice their configured hard minimum;
+provider diversity saturates at two providers. Search-only rows with
+`injected = 0` contribute to no count or score. Only direct search hits contribute
+to the query-backed ratio; one-hop related and recent-fallback records do not.
+The score, threshold, and weighted components are persisted in both the adaptive
+Markdown evidence block and document metadata, so promotion remains auditable
+without rerunning historical queries. Adaptive publication is first-writer-wins:
+the SQLite transaction selects one winner and only its in-transaction publication
+callback writes the deterministic Markdown path. If publication or commit fails,
+filesystem compensation runs before the SQLite write lock is released; it performs
+filesystem-only work to avoid self-deadlock and prolonged writer starvation. This
+prevents concurrent writers from mixing file evidence and metadata or deleting a
+later winner's file.
+The default score is an initial deterministic policy, not a learned probability.
+Legacy config files receive `0.65`; setting the threshold to `0` restores
+hard-gate-only behavior, and pending candidates are reconsidered on their next use.
+This score measures repeated utility, not truth or correctness.
 A later source supersession is propagated to its adaptive derivative so stale
 retained evidence remains hidden. Old usage rows are pruned as new usage arrives.
 

CHANGELOG.md

@@ -19,6 +19,16 @@ The project follows [Semantic Versioning](https://semver.org/).
 - English, Korean, Japanese, and Simplified Chinese documentation distinguishing
   90-day short-term evidence, adaptive long-term memory, and explicit long-term
   memory.
+- Explainable adaptive-promotion scoring with a default `0.65` threshold across
+  session diversity, UTC-day persistence, final-context injection recurrence,
+  query-backed use, and provider diversity. Hard safety gates remain mandatory;
+  non-injected search hits contribute nothing, and each promoted Markdown page
+  and document metadata records the score, threshold, and weighted components.
+- Direct-search provenance for query-backed scoring; related and recent-fallback
+  context no longer receives search credit.
+- First-writer-wins adaptive publication keeps concurrent Markdown evidence and
+  SQLite promotion metadata from different attempts from being mixed; rollback
+  compensation runs before releasing the SQLite writer lock.
 
 ### Changed
 

README.ja.md

@@ -121,6 +121,19 @@ handoff が異なる UTC 日付で 3 日以上、異なる consumer provider/ses
 利用回数を合算することもありません。昇格後に source が superseded された場合、
 派生した adaptive memory も recall から隠します。
 
+これらのハードゲートを満たすだけでは昇格しません。WikiBrain は、説明可能な昇格
+スコアがデフォルトの `0.65` 以上であることも要求します。スコアは session の多様性
+30%、異なる UTC 日にわたる継続性 25%、最終コンテキストへの反復注入 25%、明示的な
+query に基づく consumer session の割合 10%、consumer provider の多様性 10%を
+合算します。反復要素は設定したハード最小値の 2 倍で飽和し、provider の多様性は
+2 provider で飽和します。最終コンテキストに注入されなかった検索結果は寄与しません。
+昇格ページと document metadata にはスコア、しきい値、重み付き要素を記録し、
+`adaptive_memory_min_score` で 0 から 1 のしきい値を調整できます。query-backed として
+数えるのは明示的な検索の direct hit だけで、related と recent fallback は数えません。
+デフォルト値は学習済み確率ではなく、決定的な初期ポリシーです。既存 config にも `0.65`
+が適用されます。以前の hard-gate-only 動作を維持するには、しきい値を `0` にします。
+pending candidate は次回の利用時に再評価されます。
+
 昇格では source で確認した証拠を最大 2,000 文字だけ新しい Markdown ページへ
 保存し、source 文書 ID、利用回数、昇格時刻、`memory_kind: adaptive` を記録します。
 これは繰り返し有用だったコンテキストであり、内容が真実だという自動判定では

README.ko.md

@@ -139,6 +139,19 @@ identity가 없는 수동 `brainctl recall`은 세지 않습니다. 최종 `<mem
 다른 workspace의 사용량도 합치지 않습니다. 승격 뒤 source가 superseded되면 파생된
 adaptive memory도 recall에서 숨깁니다.
 
+이 hard gate를 통과하는 것만으로는 충분하지 않습니다. WikiBrain은 설명 가능한 승격
+점수가 기본 `0.65` 이상이어야 한다고 추가로 요구합니다. 점수는 session 다양성 30%,
+서로 다른 UTC 날짜에 걸친 지속성 25%, 최종 맥락의 반복 주입 25%, 명시적 query에
+기반한 consumer session 비율 10%, consumer provider 다양성 10%를 합산합니다. 반복
+항목은 설정된 hard minimum의 두 배에서 포화하고 provider 다양성은 두 provider에서
+포화합니다. 최종 맥락에 주입되지 않은 검색 결과는 점수에 기여하지 않습니다. 승격된
+페이지와 document metadata에는 점수·threshold·가중 component를 함께 기록하며,
+`adaptive_memory_min_score`로 0부터 1 사이의 threshold를 조정할 수 있습니다. 명시적
+검색의 direct hit만 query-backed로 세며 related·recent fallback record는 세지 않습니다.
+기본값은 학습된 확률이 아니라 결정적인 초기 정책입니다. 기존 config도 `0.65`를 적용하며,
+이전 hard-gate-only 동작을 유지하려면 threshold를 `0`으로 설정합니다. pending candidate는
+다음 사용 시 다시 평가됩니다.
+
 승격 시 source에서 확인한 근거를 최대 2,000자만 새 Markdown 페이지에 저장하고,
 source 문서 ID, 사용 횟수, 승격 시각, `memory_kind: adaptive`를 함께 기록합니다.
 이는 반복해서 유용했던 맥락이지 내용이 참이라는 자동 판정이 아닙니다. 원래의

README.md

@@ -140,6 +140,20 @@ not count toward their own promotion, superseded evidence is ineligible, and
 workspace counters never mix. If a promoted source is superseded later, its
 adaptive derivative is hidden from recall too.
 
+Meeting those hard gates is necessary but not sufficient. WikiBrain also requires
+an explainable promotion score of at least `0.65`. The score combines session
+diversity (30%), persistence across UTC days (25%), repeated final-context
+injection (25%), the share of consumer sessions backed by an explicit query
+(10%), and consumer-provider diversity (10%). Repetition components saturate at
+twice their configured hard minimum, and provider diversity saturates at two
+providers. Non-injected search hits contribute nothing. The score, threshold,
+and weighted components are written to the promoted page and document metadata;
+`adaptive_memory_min_score` configures the threshold from 0 to 1. Only direct
+search hits count as query-backed; related and recent-fallback records do not.
+The default is a deterministic initial policy, not a learned probability. Existing
+config files adopt `0.65`; set the threshold to `0` to retain the former hard-gate-only
+behavior. Pending candidates are reconsidered on their next use.
+
 Promotion writes at most 2,000 characters of the source-verified evidence to a
 new Markdown page with the source document ID, usage counts, promotion time,
 and `memory_kind: adaptive`. It is retained context, not a declaration that the

README.zh-CN.md

@@ -131,6 +131,17 @@ UTC 日期、3 个不同的 consumer provider/session pair 中实际进入最终
 superseded 证据没有资格，使用量也不会跨 workspace 合并。若 source 在提升后被
 superseded，其派生的 adaptive memory 也会从 recall 中隐藏。
 
+仅满足这些硬门槛还不足以提升。WikiBrain 还要求可解释的提升分数达到默认值
+`0.65`。该分数由 session 多样性（30%）、跨不同 UTC 日期的持续性（25%）、最终
+上下文中的重复注入（25%）、由显式 query 支持的 consumer session 比例（10%）和
+consumer provider 多样性（10%）组成。重复项在配置的硬门槛两倍处饱和，provider
+多样性在两个 provider 处饱和。未进入最终上下文的搜索结果不贡献分数。提升后的页面
+和 document metadata 会记录分数、阈值及加权分项；可通过
+`adaptive_memory_min_score` 在 0 到 1 之间调整阈值。只有显式搜索的 direct hit
+计入 query-backed，related 和 recent fallback 记录不计入。默认值是确定性的初始策略，
+并非学习得到的概率。已有 config 也会采用 `0.65`；若要保留此前仅使用硬门槛的行为，
+请将阈值设为 `0`。pending candidate 会在下次使用时重新评估。
+
 提升时只把经 source 验证的证据中最多 2,000 个字符写入新的 Markdown 页面，并
 记录 source 文档 ID、使用次数、提升时间及 `memory_kind: adaptive`。它表示反复
 有用的上下文，不代表系统自动断言其内容为真。原始的 90 天证据过期后，这个较小

src/wikibrain/config.py

@@ -74,6 +74,7 @@ class BrainConfig:
     adaptive_memory_min_sessions: int = 3
     adaptive_memory_min_days: int = 3
     adaptive_memory_min_injections: int = 2
+    adaptive_memory_min_score: float = 0.65
     adaptive_memory_max_chars: int = 2_000
     wikimap_command: str = "wikimap"
     update_on_stop: bool = True

src/wikibrain/curation.py

@@ -321,9 +321,25 @@ def remember(
 """
         target_path = self.config.vault_path / relative
         previous_content = (
-            target_path.read_text(encoding="utf-8") if target_path.exists() else None
+            target_path.read_text(encoding="utf-8")
+            if memory_kind != "adaptive" and target_path.exists()
+            else None
         )
-        path = self._write(relative, content)
+        adaptive_previous_content: str | None = None
+        path = target_path if memory_kind == "adaptive" else self._write(relative, content)
+
+        def publish_adaptive() -> None:
+            nonlocal adaptive_previous_content
+            adaptive_previous_content = (
+                path.read_text(encoding="utf-8") if path.exists() else None
+            )
+            self._write(relative, content)
+
+        def rollback_adaptive_publish() -> None:
+            if adaptive_previous_content is None:
+                path.unlink(missing_ok=True)
+            else:
+                atomic_write_text(path, adaptive_previous_content)
 
         def restore_failed_write() -> None:
             current = self.store.document(document_id)
@@ -350,9 +366,15 @@ def restore_failed_write() -> None:
                     **(registration_metadata or {}),
                 },
                 relations=relations,
+                insert_only=memory_kind == "adaptive",
+                publish=publish_adaptive if memory_kind == "adaptive" else None,
+                rollback_publish=(
+                    rollback_adaptive_publish if memory_kind == "adaptive" else None
+                ),
             )
         except Exception:
-            restore_failed_write()
+            if memory_kind != "adaptive":
+                restore_failed_write()
             raise
         if not registered:
             restore_failed_write()
@@ -391,13 +413,31 @@ def promote_adaptive(
         )
         title = f"Retained context: {title_line[:100]}"
         promoted_at = datetime.now(UTC).isoformat(timespec="milliseconds")
+        promotion_score = float(usage.get("promotion_score", 0.0))
+        promotion_score_threshold = float(
+            usage.get("promotion_score_threshold", 0.0)
+        )
+        raw_components = usage.get("promotion_score_components", {})
+        score_components = (
+            {str(key): float(value) for key, value in raw_components.items()}
+            if isinstance(raw_components, Mapping)
+            else {}
+        )
+        component_lines = "".join(
+            f"  - {name.replace('_', ' ').title()}: {value:.3f}\n"
+            for name, value in score_components.items()
+        )
         body = (
             bounded
             + "\n\n## Automatic retention evidence\n\n"
             + f"- Distinct sessions: {int(usage.get('distinct_sessions', 0))}\n"
+            + f"- Distinct providers: {int(usage.get('distinct_providers', 0))}\n"
             + f"- Distinct days: {int(usage.get('distinct_days', 0))}\n"
             + f"- Search sessions: {int(usage.get('search_sessions', 0))}\n"
             + f"- Context injections: {int(usage.get('context_injections', 0))}\n"
+            + f"- Promotion score: {promotion_score:.3f} / "
+            + f"{promotion_score_threshold:.3f}\n"
+            + ("- Score components:\n" + component_lines if component_lines else "")
             + f"- Source document: `{source_document_id}`\n"
         )
         created_id, path = self.remember(
@@ -418,8 +458,12 @@ def promote_adaptive(
                 "adaptive_source_document_id": source_document_id,
                 "promoted_at": promoted_at,
                 "last_used_at": str(usage.get("last_used_at") or promoted_at),
+                "promotion_score": promotion_score,
+                "promotion_score_threshold": promotion_score_threshold,
+                "promotion_score_components": score_components,
                 "promotion_reason": {
                     "distinct_sessions": int(usage.get("distinct_sessions", 0)),
+                    "distinct_providers": int(usage.get("distinct_providers", 0)),
                     "distinct_days": int(usage.get("distinct_days", 0)),
                     "search_sessions": int(usage.get("search_sessions", 0)),
                     "context_injections": int(

src/wikibrain/recall.py

@@ -77,6 +77,7 @@ def _render_record(
     captured_at: str,
     evidence: str,
     relations: list[tuple[str, str]] | None = None,
+    retrieval_source: str | None = None,
     truncated: bool = False,
 ) -> str:
     line_attribute = f' line="{line}"' if line is not None else ""
@@ -241,6 +242,7 @@ def context(
                             snippet=hit.snippet,
                             query=query,
                         ),
+                        "retrieval_source": "search",
                         "relations": [
                             (
                                 str(relation["relation_type"]),
@@ -298,6 +300,7 @@ def context(
                         "line": None,
                         "captured_at": str(related_row["created_at"]),
                         "evidence": snippet,
+                        "retrieval_source": "related",
                         "relations": [
                             (
                                 str(relation["relation_type"]),
@@ -339,6 +342,7 @@ def context(
                         "line": None,
                         "captured_at": str(row["created_at"]),
                         "evidence": snippet,
+                        "retrieval_source": "recent",
                         "relations": [
                             (
                                 str(relation["relation_type"]),
@@ -416,12 +420,13 @@ def context(
                         str(record["document_id"]),
                         consumer_provider=provider,
                         consumer_session_id=session_id,
-                        searched=bool(query),
+                        searched=record.get("retrieval_source") == "search",
                         injected=True,
                         window_days=self.config.adaptive_memory_window_days,
                         min_sessions=self.config.adaptive_memory_min_sessions,
                         min_days=self.config.adaptive_memory_min_days,
                         min_injections=self.config.adaptive_memory_min_injections,
+                        min_score=self.config.adaptive_memory_min_score,
                     )
                     if usage["eligible"]:
                         curator = curator or Curator(