feat(learning): measure backend sufficiency query telemetry

Jacobinwwey · Jacobinwwey · commit cd6778177424 · 2026-04-21T01:35:50.000-05:00
diff --git a/docs/diataxis/en/explanation/development-progress-dashboard.md b/docs/diataxis/en/explanation/development-progress-dashboard.md
@@ -55,6 +55,10 @@ Execution anchor:
   - `scripts/backend-baseline-sufficiency-utils.js`
   - `scripts/verify-backend-baseline-sufficiency.js`
   - `src/backend.baseline.sufficiency.contract.test.ts`
+- Upgraded the first real measured observation path:
+  - runtime query traffic now promotes sufficiency query-latency observation from `policy_default` to `measured`,
+  - the warm-query threshold now carries `observedValue` when runtime telemetry exists,
+  - docs/verifier output now expose query observation mode and latest persistence event timestamps.
 - Locked the current architectural posture to an evidence-first rule:
   - local sqlite + deterministic ANN remains the default backend baseline,
   - heavier backend work now needs explicit escalation triggers,
@@ -64,7 +68,7 @@ Execution anchor:
   - `mkdocs.yml`
   - EN/ZH Diataxis overview progress entry points.
 - Changed the next bounded backend-governance slice:
-  - replace `policy_default` thresholds with measured capture rather than redoing initial route/report wiring.
+  - extend measurement beyond query latency into rebuild/recovery duration capture rather than redoing initial route/report wiring.
 
 ## Latest Mainline Increment (2026-04-14 M1 Baseline)
 
diff --git a/docs/diataxis/en/explanation/local-backend-sufficiency-and-escalation-plan.md b/docs/diataxis/en/explanation/local-backend-sufficiency-and-escalation-plan.md
@@ -36,7 +36,7 @@ This decision is no longer prose-only.
 
 Important limitation:
 
-- current evidence mode is still `policy_default`, not measured performance capture.
+- top-level evidence mode is still `policy_default`, but query-latency observation now upgrades to `measured` when runtime query history exists.
 
 That means the next backend-governance step is **measurement hardening**, not backend replacement.
 
@@ -223,8 +223,8 @@ That is done.
 
 The remaining slice is:
 
-- replace `policy_default` thresholds with measured capture where feasible,
-- add explicit rebuild / recovery / query-latency observation fields,
+- replace the remaining `policy_default` thresholds with measured capture where feasible,
+- extend measurement beyond query latency into rebuild / recovery duration fields,
 - keep escalation tied to measured pressure instead of architectural anxiety.
 
 ## Recommended Next Lane
diff --git a/docs/diataxis/zh/explanation/development-progress-dashboard.md b/docs/diataxis/zh/explanation/development-progress-dashboard.md
@@ -55,6 +55,10 @@
   - `scripts/backend-baseline-sufficiency-utils.js`
   - `scripts/verify-backend-baseline-sufficiency.js`
   - `src/backend.baseline.sufficiency.contract.test.ts`
+- 已补上第一条真实 measured observation 路径：
+  - 运行时 query 流量会把 sufficiency query-latency observation 从 `policy_default` 升级为 `measured`，
+  - warm-query threshold 在存在运行时 telemetry 时会携带 `observedValue`，
+  - docs/verifier 输出现在会暴露 query observation mode 与最近 persistence event 时间戳。
 - 已把当前架构姿态锁定为证据优先规则：
   - local sqlite + 确定性 ANN 继续作为默认 backend baseline，
   - 更重 backend 工作必须提供显式 escalation trigger，
@@ -64,7 +68,7 @@
   - `mkdocs.yml`
   - EN/ZH Diataxis overview 的进度入口。
 - 已更新下一段有界 backend governance 切片：
-  - 把 `policy_default` 阈值升级为 measured capture，而不是重复补第一版 route/report。
+  - 把测量范围从 query latency 继续扩展到 rebuild/recovery duration，而不是重复补第一版 route/report。
 
 ## 主线最新增量（2026-04-14 M1 基线）
 
diff --git a/docs/diataxis/zh/explanation/local-backend-sufficiency-and-escalation-plan.md b/docs/diataxis/zh/explanation/local-backend-sufficiency-and-escalation-plan.md
@@ -36,7 +36,7 @@
 
 当前限制：
 
-- 当前 evidence mode 仍然是 `policy_default`，还不是测量驱动的性能采集。
+- 顶层 evidence mode 仍然是 `policy_default`，但 query-latency observation 已可在运行时产生查询历史后升级为 `measured`。
 
 所以，下一步 backend governance 工作应是**测量加固**，不是 backend replacement。
 
@@ -222,8 +222,8 @@
 
 剩余切片是：
 
-- 在可行处用测量采集替换 `policy_default` 阈值，
-- 新增明确的 rebuild / recovery / query-latency observation 字段，
+- 在剩余维度上继续用测量采集替换 `policy_default` 阈值，
+- 把测量范围从 query latency 扩展到 rebuild / recovery duration 字段，
 - 保证升级依然由测量压力驱动，而不是由架构焦虑驱动。
 
 ## 推荐下一条主线
diff --git a/scripts/backend-baseline-sufficiency-utils.js b/scripts/backend-baseline-sufficiency-utils.js
@@ -29,6 +29,10 @@ function fileExists(filePath) {
   }
 }
 
+function isNonEmptyString(value) {
+  return typeof value === 'string' && value.trim().length > 0;
+}
+
 function createThreshold(thresholdId, unit, target, summary) {
   return {
     thresholdId,
@@ -64,6 +68,14 @@ function evaluateBackendBaselineSufficiency(options = {}) {
     repoRoot,
     provenance,
   });
+  const runtimeState = options.runtimeState || {
+    queryCount: 0,
+    queryP95Ms: undefined,
+    queryAverageMs: undefined,
+    queryMaxMs: undefined,
+    evidenceMode: 'policy_default',
+  };
+  const storeDiagnostics = options.storeDiagnostics || {};
   const packageJson = readJsonIfExists(path.join(repoRoot, 'package.json'));
   const scripts = packageJson.scripts || {};
 
@@ -116,14 +128,27 @@ function evaluateBackendBaselineSufficiency(options = {}) {
     'Local backup, restore, and deterministic rebuild remain acceptable operator tactics.',
     'Product value is currently gated more by conversation/memory/operator surfaces than by backend distribution.',
   ];
+  const measuredQueryLatency =
+    Number(runtimeState.queryCount || 0) > 0
+    && Number.isFinite(runtimeState.queryP95Ms)
+    && Number(runtimeState.queryP95Ms) >= 0;
+  const queryLatencyObservationMode = measuredQueryLatency ? 'measured' : 'policy_default';
+  const persistenceObservationMode =
+    isNonEmptyString(storeDiagnostics.lastLoadAt) || isNonEmptyString(storeDiagnostics.lastSaveAt)
+      ? 'observed_event'
+      : 'policy_default';
 
   const thresholds = [
-    createThreshold(
-      'warm_query_p95_ms',
-      'ms',
-      300,
-      'Hold local while warm interactive query latency stays within a 300 ms p95 budget on a reference device.'
-    ),
+    {
+      ...createThreshold(
+        'warm_query_p95_ms',
+        'ms',
+        300,
+        'Hold local while warm interactive query latency stays within a 300 ms p95 budget on a reference device.'
+      ),
+      observationStatus: queryLatencyObservationMode,
+      observedValue: measuredQueryLatency ? Number(runtimeState.queryP95Ms) : undefined,
+    },
     createThreshold(
       'cold_query_p95_ms',
       'ms',
@@ -149,6 +174,7 @@ function evaluateBackendBaselineSufficiency(options = {}) {
     foundationReadiness.status === 'integrated'
     && foundationReadiness.decision === 'go'
     && foundationReadiness.promotionCriteriaPassed === foundationReadiness.promotionCriteriaTotal;
+  const queryLatencyWithinThreshold = !measuredQueryLatency || Number(runtimeState.queryP95Ms) <= 300;
 
   const dimensions = [
     createDimension(
@@ -159,8 +185,12 @@ function evaluateBackendBaselineSufficiency(options = {}) {
     ),
     createDimension(
       'query_latency',
-      foundationReadiness.baseline.vectorAdapterIndependent && foundationReadiness.baseline.queryBackendDefaultMode === 'local_hybrid',
-      'Local hybrid query + ANN acceleration is present; no measured repo evidence currently forces backend escalation.',
+      foundationReadiness.baseline.vectorAdapterIndependent
+        && foundationReadiness.baseline.queryBackendDefaultMode === 'local_hybrid'
+        && queryLatencyWithinThreshold,
+      measuredQueryLatency
+        ? `Measured runtime query p95 is ${Number(runtimeState.queryP95Ms).toFixed(4)} ms across ${Number(runtimeState.queryCount || 0)} recorded queries.`
+        : 'Local hybrid query + ANN acceleration is present; no measured repo evidence currently forces backend escalation.',
       'Escalate when measured warm or cold p95 latency stays above target after local tuning.'
     ),
     createDimension(
@@ -219,6 +249,15 @@ function evaluateBackendBaselineSufficiency(options = {}) {
     );
   }
 
+  if (measuredQueryLatency && Number(runtimeState.queryP95Ms) > 300) {
+    activeTriggers.push(
+      createTrigger(
+        'query_latency_threshold_exceeded',
+        `Measured runtime query p95 ${Number(runtimeState.queryP95Ms).toFixed(4)} ms exceeds the current warm-query target of 300 ms.`
+      )
+    );
+  }
+
   const sufficient = activeTriggers.length === 0;
 
   if (sufficient) {
@@ -268,6 +307,24 @@ function evaluateBackendBaselineSufficiency(options = {}) {
     packageScripts: {
       sufficiencyVerifierPresent,
     },
+    observations: {
+      queryLatency: {
+        queryCount: Number(runtimeState.queryCount || 0),
+        queryP95Ms: measuredQueryLatency ? Number(runtimeState.queryP95Ms) : undefined,
+        queryAverageMs: measuredQueryLatency && Number.isFinite(runtimeState.queryAverageMs)
+          ? Number(runtimeState.queryAverageMs)
+          : undefined,
+        queryMaxMs: measuredQueryLatency && Number.isFinite(runtimeState.queryMaxMs)
+          ? Number(runtimeState.queryMaxMs)
+          : undefined,
+        evidenceMode: queryLatencyObservationMode,
+      },
+      persistence: {
+        lastLoadAt: isNonEmptyString(storeDiagnostics.lastLoadAt) ? storeDiagnostics.lastLoadAt : undefined,
+        lastSaveAt: isNonEmptyString(storeDiagnostics.lastSaveAt) ? storeDiagnostics.lastSaveAt : undefined,
+        evidenceMode: persistenceObservationMode,
+      },
+    },
     provenance,
     mandatoryChecks: [
       {
@@ -357,6 +414,16 @@ function formatBackendBaselineSufficiencyMarkdown(result, reportPaths) {
   lines.push('', '## Package Scripts', '');
   lines.push(`- Sufficiency verifier present: ${result.packageScripts.sufficiencyVerifierPresent ? 'yes' : 'no'}`);
 
+  lines.push('', '## Observations', '');
+  lines.push(`- Query latency evidence mode: ${result.observations.queryLatency.evidenceMode}`);
+  lines.push(`- Query count: ${result.observations.queryLatency.queryCount}`);
+  lines.push(`- Query p95 ms: ${typeof result.observations.queryLatency.queryP95Ms === 'number' ? result.observations.queryLatency.queryP95Ms : 'unmeasured'}`);
+  lines.push(`- Query average ms: ${typeof result.observations.queryLatency.queryAverageMs === 'number' ? result.observations.queryLatency.queryAverageMs : 'unmeasured'}`);
+  lines.push(`- Query max ms: ${typeof result.observations.queryLatency.queryMaxMs === 'number' ? result.observations.queryLatency.queryMaxMs : 'unmeasured'}`);
+  lines.push(`- Persistence evidence mode: ${result.observations.persistence.evidenceMode}`);
+  lines.push(`- Last load at: ${result.observations.persistence.lastLoadAt || 'unobserved'}`);
+  lines.push(`- Last save at: ${result.observations.persistence.lastSaveAt || 'unobserved'}`);
+
   lines.push('', '## Provenance', '');
   lines.push(`- Repo root source: ${result.provenance.repoRootSource}`);
   lines.push(`- Runtime project root aligned: ${result.provenance.runtimeProjectRootAligned ? 'yes' : 'no'}`);
diff --git a/scripts/verify-backend-baseline-sufficiency.js b/scripts/verify-backend-baseline-sufficiency.js
@@ -33,6 +33,17 @@ function printHumanReport(result, reportPaths) {
   console.log(
     `[Backend Sufficiency] Sufficiency verifier present: ${result.packageScripts.sufficiencyVerifierPresent ? 'yes' : 'no'}`
   );
+  console.log(`[Backend Sufficiency] Query latency evidence mode: ${result.observations.queryLatency.evidenceMode}`);
+  console.log(`[Backend Sufficiency] Query count: ${result.observations.queryLatency.queryCount}`);
+  console.log(
+    `[Backend Sufficiency] Query p95 ms: ${typeof result.observations.queryLatency.queryP95Ms === 'number' ? result.observations.queryLatency.queryP95Ms : 'unmeasured'}`
+  );
+  console.log(
+    `[Backend Sufficiency] Last load at: ${result.observations.persistence.lastLoadAt || 'unobserved'}`
+  );
+  console.log(
+    `[Backend Sufficiency] Last save at: ${result.observations.persistence.lastSaveAt || 'unobserved'}`
+  );
   console.log(`[Backend Sufficiency] Repo root source: ${result.provenance.repoRootSource}`);
   console.log(
     `[Backend Sufficiency] Runtime project root aligned: ${result.provenance.runtimeProjectRootAligned ? 'yes' : 'no'}`
diff --git a/src/backend.baseline.sufficiency.contract.test.ts b/src/backend.baseline.sufficiency.contract.test.ts
@@ -9,7 +9,7 @@ type PackageJson = {
 type BackendBaselineSufficiencyResult = {
   status: 'sufficient' | 'escalation_required';
   decision: 'hold_local' | 'escalate';
-  evidenceMode: 'policy_default' | 'measured';
+  evidenceMode: 'policy_default' | 'measured' | 'observed_event';
   baseline: {
     foundationStatus: 'planned' | 'in_progress' | 'integrated';
     foundationDecision: 'go' | 'no-go';
@@ -32,7 +32,9 @@ type BackendBaselineSufficiencyResult = {
     comparator: '<=';
     unit: 'ms' | 'minutes';
     target: number;
-    observationStatus: 'policy_default' | 'measured';
+    observationStatus: 'policy_default' | 'measured' | 'observed_event';
+    observedValue?: number;
+    observedAt?: string;
   }>;
   dimensions: Array<{
     dimensionId:
@@ -43,10 +45,14 @@ type BackendBaselineSufficiencyResult = {
       | 'recovery_ops'
       | 'observability';
     sufficient: boolean;
-    evidenceMode: 'policy_default' | 'measured';
+    evidenceMode: 'policy_default' | 'measured' | 'observed_event';
   }>;
   activeTriggers: Array<{
-    triggerId: 'foundation_not_integrated' | 'sufficiency_docs_not_aligned' | 'sufficiency_verifier_missing';
+    triggerId:
+      | 'foundation_not_integrated'
+      | 'sufficiency_docs_not_aligned'
+      | 'sufficiency_verifier_missing'
+      | 'query_latency_threshold_exceeded';
     summary: string;
   }>;
   documents: {
@@ -56,6 +62,20 @@ type BackendBaselineSufficiencyResult = {
   packageScripts: {
     sufficiencyVerifierPresent: boolean;
   };
+  observations: {
+    queryLatency: {
+      queryCount: number;
+      queryP95Ms?: number;
+      queryAverageMs?: number;
+      queryMaxMs?: number;
+      evidenceMode: 'policy_default' | 'measured' | 'observed_event';
+    };
+    persistence: {
+      lastLoadAt?: string;
+      lastSaveAt?: string;
+      evidenceMode: 'policy_default' | 'measured' | 'observed_event';
+    };
+  };
   provenance: {
     repoRootSource: 'explicit' | 'project_root_hint' | 'cwd' | 'module_root';
     runtimeProjectRootAligned: boolean;
@@ -185,6 +205,18 @@ describe('backend baseline sufficiency contract', () => {
     expect(result.packageScripts).toEqual({
       sufficiencyVerifierPresent: true,
     });
+    expect(result.observations.queryLatency).toEqual({
+      queryCount: 0,
+      queryP95Ms: undefined,
+      queryAverageMs: undefined,
+      queryMaxMs: undefined,
+      evidenceMode: 'policy_default',
+    });
+    expect(result.observations.persistence).toEqual({
+      lastLoadAt: undefined,
+      lastSaveAt: undefined,
+      evidenceMode: 'policy_default',
+    });
     expect(result.nextLaneRecommendation).toBe('product_memory_operator');
     expect(result.recommendations).toContain(
       'Keep sqlite + local ANN as the default backend baseline while the sufficiency report remains trigger-free.'
@@ -259,6 +291,8 @@ describe('backend baseline sufficiency contract', () => {
     expect(result.packageScripts).toEqual({
       sufficiencyVerifierPresent: false,
     });
+    expect(result.observations.queryLatency.evidenceMode).toBe('policy_default');
+    expect(result.observations.persistence.evidenceMode).toBe('policy_default');
     expect(result.nextLaneRecommendation).toBe('backend_escalation');
   });
 });
diff --git a/src/learning/KnowledgeLearningPlatform.test.ts b/src/learning/KnowledgeLearningPlatform.test.ts
@@ -1404,12 +1404,52 @@ describe('KnowledgeLearningPlatform', () => {
                 'observability',
             ]);
             expect(sufficiency.activeTriggers).toEqual([]);
+            expect(sufficiency.observations.queryLatency).toEqual({
+                queryCount: 0,
+                queryP95Ms: undefined,
+                queryAverageMs: undefined,
+                queryMaxMs: undefined,
+                evidenceMode: 'policy_default',
+            });
+            expect(sufficiency.observations.persistence).toEqual({
+                lastLoadAt: undefined,
+                lastSaveAt: undefined,
+                evidenceMode: 'policy_default',
+            });
             expect(sufficiency.nextLaneRecommendation).toBe('product_memory_operator');
         } finally {
             fs.rmSync(tempRoot, { recursive: true, force: true });
         }
     });
 
+    test('backend baseline sufficiency upgrades query latency observation to measured after runtime queries', async () => {
+        await platform.ingestKnowledge({
+            incremental: true,
+            documents: [
+                {
+                    documentId: 'doc_backend_query_measure',
+                    sourcePath: 'Knowledge_Base/backend_query_measure.md',
+                    language: 'en',
+                    content: '# Backend Query Measure\nLocal ANN query telemetry should become measurable after runtime traffic.',
+                },
+            ],
+        });
+
+        await platform.queryKnowledge({
+            query: 'local ann telemetry',
+            topK: 2,
+        });
+
+        const sufficiency = await platform.getBackendBaselineSufficiency();
+        const warmThreshold = sufficiency.thresholds.find((entry) => entry.thresholdId === 'warm_query_p95_ms');
+
+        expect(sufficiency.observations.queryLatency.queryCount).toBeGreaterThan(0);
+        expect(sufficiency.observations.queryLatency.evidenceMode).toBe('measured');
+        expect(sufficiency.observations.queryLatency.queryP95Ms).toBeGreaterThanOrEqual(0);
+        expect(warmThreshold?.observationStatus).toBe('measured');
+        expect(warmThreshold?.observedValue).toBeGreaterThanOrEqual(0);
+    });
+
     test('tutor action uses misconception context for targeted guidance', async () => {
         const ingest = await platform.ingestKnowledge({
             incremental: true,
diff --git a/src/learning/KnowledgeLearningPlatform.ts b/src/learning/KnowledgeLearningPlatform.ts
diff --git a/src/learning/types.ts b/src/learning/types.ts
diff --git a/src/server.migration.test.ts b/src/server.migration.test.ts
