feat(agent-workspace): add notification health cleanup

Author: Jacobinwwey

docs/brainstorms/2026-04-16-mainline-ci-stabilization-and-m7-direction-requirements.md

@@ -587,6 +587,32 @@ Deliverables:
   - `npm run test:agent-workspace:contracts`
   - `npm run verify:agent-workspace:runtime`
 
+### M7.19 (Now): Notification Delivery Health Auditing and Stale Cleanup (Lane Ops Bridge)
+
+Deliverables:
+
+- add bounded notification-health route for operator delivery auditing.
+- remove stale reminder notification records whose escalation is no longer active.
+- expose cleanup counts so retention behavior is auditable instead of implicit.
+
+#### M7.19 Progress Note (2026-04-16)
+
+- [Done] expanded `src/server.ts` with notification health route:
+  - `GET /api/knowledge/operator/agent-workspace-diagnostics/triage/remediation/escalation/notification-health?limit=...`.
+- [Done] added deterministic stale-notification cleanup behavior:
+  - reminder-channel delivery records are pruned when their `escalationId` no longer exists in active escalation governance state.
+- [Done] added bounded health helper stack:
+  - `cleanupStaleAgentWorkspaceDiagnosticsRemediationEscalationNotificationTrail(...)`,
+  - `buildAgentWorkspaceDiagnosticsRemediationEscalationNotificationHealthReport(...)`.
+- [Done] expanded evidence coverage:
+  - `src/server.migration.test.ts` now validates notification health route semantics, cleanup counts, stale-record pruning, and persisted file cleanup.
+  - `src/knowledge.api.contract.test.ts` now fail-fast checks notification-health route contract.
+  - `src/agent_workspace.verification.contract.test.ts` + `scripts/verify-agent-workspace-runtime.js` now fail fast on notification-health route and cleanup-helper drift.
+- [Done] verification evidence:
+  - `npm test -- src/server.migration.test.ts --runInBand --testNamePattern "escalation notification health audit and stale cleanup stay deterministic"`
+  - `npm run test:agent-workspace:contracts`
+  - `npm run verify:agent-workspace:runtime`
+
 ## Success Criteria
 
 - CI failure mode that previously blocked the three agent-workspace suites is eliminated on mainline.
@@ -596,4 +622,4 @@ Deliverables:
 
 ## Next Step
 
-Proceed to `/prompts:ce-plan` using this document as the source for `M7.19` decomposition (notification delivery health auditing and stale-notification cleanup), while preserving M7 lane boundary constraints.
+Proceed to `/prompts:ce-plan` using this document as the source for `M7.20` decomposition (notification anomaly surfacing and retention-policy governance), while preserving M7 lane boundary constraints.

docs/diataxis/en/explanation/development-progress-dashboard.md

@@ -562,6 +562,24 @@ Execution anchor:
   - `npm run test:agent-workspace:contracts`
   - `npm run verify:agent-workspace:runtime`
 
+## Latest Mainline Increment (2026-04-16 M7.19 Notification Delivery Health Auditing and Stale Cleanup Lane)
+
+- Expanded `src/server.ts` with notification health route:
+  - `GET /api/knowledge/operator/agent-workspace-diagnostics/triage/remediation/escalation/notification-health?limit=...`.
+- Added deterministic stale-notification cleanup behavior:
+  - reminder-channel notification records are pruned when their `escalationId` no longer exists in active escalation governance state.
+- Added bounded delivery-health auditing helpers:
+  - cleanup helper for stale reminder records,
+  - health-report helper exposing active escalation count, tracked notifications, cleanup delta, and by-channel/by-status counters.
+- Expanded executable evidence:
+  - `src/server.migration.test.ts` now validates notification health route semantics, stale-record removal, cleanup counts, and persisted notification-log cleanup.
+- Hardened runtime verification gate:
+  - `src/knowledge.api.contract.test.ts`, `src/agent_workspace.verification.contract.test.ts`, and `scripts/verify-agent-workspace-runtime.js` now fail fast on notification-health route and cleanup-helper drift.
+- Verification evidence:
+  - `npm test -- src/server.migration.test.ts --runInBand --testNamePattern \"escalation notification health audit and stale cleanup stay deterministic\"`
+  - `npm run test:agent-workspace:contracts`
+  - `npm run verify:agent-workspace:runtime`
+
 ## Mainline vs Working-Branch Snapshot (2026-04-14)
 
 | Capability Slice | Working Branch (`feat/learning-multi-tutor-adapter`) | Mainline (`origin/main`) | Integration Status |
@@ -610,7 +628,7 @@ This dashboard aligns against the following requirement chain:
 | L2 Retrieval | explainable hybrid/vector retrieval + governance | Expanded in branch-oriented plans | Mainline file-backed baseline only (`src/learning/store.ts`) | Re-enter lane after concrete module evidence lands on mainline |
 | L3 Learning | mastery diagnostics + path/session loop | Expanded in branch | Partially integrated | Contract and integration parity |
 | L4 Interaction | agent conversation + focus/path pane runtime | Implemented in branch | M1-M4 baseline integrated on mainline | Expand capability surface via typed contract only |
-| L5 Governance | runbook, diagnostics, replay/autonomy controls | Expanded in branch | Operator diagnostics persistence/triage/history/threshold governance + runbook automation/audit + adaptive simulation/remediation + remediation backtest/approval-gate + approval-policy hardening/regression-alarms + approval-policy drift/escalation + escalation acknowledgement lifecycle/audit + escalation SLA/reminder baseline + notification digest/suppression baseline + delivery-log observability integrated | M7.19: delivery-health auditing and stale-notification cleanup |
+| L5 Governance | runbook, diagnostics, replay/autonomy controls | Expanded in branch | Operator diagnostics persistence/triage/history/threshold governance + runbook automation/audit + adaptive simulation/remediation + remediation backtest/approval-gate + approval-policy hardening/regression-alarms + approval-policy drift/escalation + escalation acknowledgement lifecycle/audit + escalation SLA/reminder baseline + notification digest/suppression baseline + delivery-log observability + stale-cleanup health auditing integrated | M7.20: anomaly surfacing and retention-policy governance |
 
 ## Verification Baseline
 

docs/diataxis/zh/explanation/development-progress-dashboard.md

@@ -564,6 +564,24 @@
   - `npm run test:agent-workspace:contracts`
   - `npm run verify:agent-workspace:runtime`
 
+## 主线最新增量（2026-04-16 M7.19 通知交付健康审计与陈旧通知清理链路）
+
+- 已在 `src/server.ts` 增加通知健康审计路由：
+  - `GET /api/knowledge/operator/agent-workspace-diagnostics/triage/remediation/escalation/notification-health?limit=...`。
+- 已新增确定性陈旧通知清理行为：
+  - reminder channel 通知记录在其 `escalationId` 不再存在于当前 active escalation 治理状态时会被清理。
+- 已新增有界交付健康 helper：
+  - 陈旧 reminder 记录清理 helper，
+  - 输出 active escalation 数、tracked notification 数、cleanup delta 与 by-channel/by-status 聚合的 health helper。
+- 已补可执行证据：
+  - `src/server.migration.test.ts` 新增通知健康路由语义、陈旧记录移除、cleanup 计数与通知日志落盘清理断言。
+- 已加固 runtime 门禁：
+  - `src/knowledge.api.contract.test.ts`、`src/agent_workspace.verification.contract.test.ts` 与 `scripts/verify-agent-workspace-runtime.js` 新增 notification-health 路由与 cleanup helper 的 fail-fast 断言。
+- 验证证据：
+  - `npm test -- src/server.migration.test.ts --runInBand --testNamePattern \"escalation notification health audit and stale cleanup stay deterministic\"`
+  - `npm run test:agent-workspace:contracts`
+  - `npm run verify:agent-workspace:runtime`
+
 ## 主线 vs 工作分支快照（2026-04-14）
 
 | 能力切片 | 工作分支（`feat/learning-multi-tutor-adapter`） | 主线（`origin/main`） | 集成状态 |
@@ -612,7 +630,7 @@
 | L2 检索层 | 可解释混合/向量检索 + 治理 | 分支规划增强中 | 主线当前为 file-backed 基线（`src/learning/store.ts`） | 待主线出现对应模块证据后再收敛 |
 | L3 学习层 | 掌握诊断 + 路径/会话闭环 | 分支增强中 | 主线部分集成 | 契约与集成一致性 |
 | L4 交互层 | agent 对话 + focus/path pane 运行时 | 分支已实现 | 主线 M1-M4 已落入基线 | 继续通过 typed contract 扩展动作面 |
-| L5 治理层 | runbook/诊断/回放与自动化 | 分支增强中 | 主线已集成运维诊断持久化/分级/趋势历史/阈值治理 + runbook 自动化/阈值审计 + 自适应模拟/自动修复 + 回测/批准门禁 + 批准策略硬化/回归告警 + 批准策略漂移/升级 + 升级确认生命周期/审计 + 升级 SLA/提醒基线 + 通知摘要/抑制基线 + 交付日志可观测性 | M7.19：通知交付健康审计与陈旧通知清理 |
+| L5 治理层 | runbook/诊断/回放与自动化 | 分支增强中 | 主线已集成运维诊断持久化/分级/趋势历史/阈值治理 + runbook 自动化/阈值审计 + 自适应模拟/自动修复 + 回测/批准门禁 + 批准策略硬化/回归告警 + 批准策略漂移/升级 + 升级确认生命周期/审计 + 升级 SLA/提醒基线 + 通知摘要/抑制基线 + 交付日志可观测性 + 陈旧通知健康审计 | M7.20：异常通知浮现与 retention policy 治理 |
 
 ## 验证基线
 

scripts/verify-agent-workspace-runtime.js

@@ -164,6 +164,10 @@ function verifyAgentWorkspaceRuntime(repoRoot = path.resolve(__dirname, '..')) {
     serverSource.includes('/api/knowledge/operator/agent-workspace-diagnostics/triage/remediation/escalation/notifications'),
     'Missing diagnostics remediation escalation notifications route in src/server.ts'
   );
+  assert(
+    serverSource.includes('/api/knowledge/operator/agent-workspace-diagnostics/triage/remediation/escalation/notification-health'),
+    'Missing diagnostics remediation escalation notification health route in src/server.ts'
+  );
   assert(
     serverSource.includes('/api/knowledge/operator/agent-workspace-diagnostics/triage/remediation/escalation/digest'),
     'Missing diagnostics remediation escalation digest route in src/server.ts'
@@ -288,6 +292,14 @@ function verifyAgentWorkspaceRuntime(repoRoot = path.resolve(__dirname, '..')) {
     serverSource.includes('buildAgentWorkspaceDiagnosticsRemediationEscalationNotificationSummary'),
     'Missing remediation escalation notification summary helper in src/server.ts'
   );
+  assert(
+    serverSource.includes('cleanupStaleAgentWorkspaceDiagnosticsRemediationEscalationNotificationTrail'),
+    'Missing remediation escalation stale notification cleanup helper in src/server.ts'
+  );
+  assert(
+    serverSource.includes('buildAgentWorkspaceDiagnosticsRemediationEscalationNotificationHealthReport'),
+    'Missing remediation escalation notification health helper in src/server.ts'
+  );
   assert(
     serverSource.includes('applyAgentWorkspaceDiagnosticsRemediationEscalationReminderSuppressionPolicy'),
     'Missing remediation escalation reminder suppression policy helper in src/server.ts'
@@ -355,14 +367,14 @@ function verifyAgentWorkspaceRuntime(repoRoot = path.resolve(__dirname, '..')) {
       'diagnostics remediation policy drift route exists',
       'diagnostics remediation escalation route exists',
       'diagnostics remediation escalation SLA/reminder routes exist',
-      'diagnostics remediation escalation notification policy, log, and digest routes exist',
+      'diagnostics remediation escalation notification policy, log, health, and digest routes exist',
       'diagnostics remediation escalation acknowledgement routes exist',
       'diagnostics remediation approval routes exist',
       'diagnostics triage remediation route exists',
       'diagnostics retention governance exists',
       'diagnostics alert-threshold governance helpers exist',
       'diagnostics threshold simulation, drift, remediation, escalation, and SLA helpers exist',
-      'diagnostics escalation notification, digest cadence, reminder suppression, and delivery-log helpers exist',
+      'diagnostics escalation notification, digest cadence, reminder suppression, delivery-log, and health helpers exist',
       'diagnostics remediation policy and alarm helpers exist',
       'diagnostics threshold audit helpers exist',
       'diagnostics remediation approval trail helpers exist',

src/agent_workspace.verification.contract.test.ts

@@ -64,6 +64,7 @@ describe('agent workspace verification script contracts', () => {
     expect(runtimeSource).toContain('/api/knowledge/operator/agent-workspace-diagnostics/triage/remediation/escalation/reminders');
     expect(runtimeSource).toContain('/api/knowledge/operator/agent-workspace-diagnostics/triage/remediation/escalation/notification-policy');
     expect(runtimeSource).toContain('/api/knowledge/operator/agent-workspace-diagnostics/triage/remediation/escalation/notifications');
+    expect(runtimeSource).toContain('/api/knowledge/operator/agent-workspace-diagnostics/triage/remediation/escalation/notification-health');
     expect(runtimeSource).toContain('/api/knowledge/operator/agent-workspace-diagnostics/triage/remediation/escalation/digest');
     expect(runtimeSource).toContain('/api/knowledge/operator/agent-workspace-diagnostics/triage/remediation/escalation/acknowledge');
     expect(runtimeSource).toContain('/api/knowledge/operator/agent-workspace-diagnostics/triage/remediation/approvals');
@@ -93,6 +94,8 @@ describe('agent workspace verification script contracts', () => {
     expect(runtimeSource).toContain('readAgentWorkspaceDiagnosticsRemediationEscalationNotificationTrail');
     expect(runtimeSource).toContain('appendAgentWorkspaceDiagnosticsRemediationEscalationNotificationEntries');
     expect(runtimeSource).toContain('buildAgentWorkspaceDiagnosticsRemediationEscalationNotificationSummary');
+    expect(runtimeSource).toContain('cleanupStaleAgentWorkspaceDiagnosticsRemediationEscalationNotificationTrail');
+    expect(runtimeSource).toContain('buildAgentWorkspaceDiagnosticsRemediationEscalationNotificationHealthReport');
     expect(runtimeSource).toContain('applyAgentWorkspaceDiagnosticsRemediationEscalationReminderSuppressionPolicy');
     expect(runtimeSource).toContain('buildAgentWorkspaceDiagnosticsRemediationEscalationGovernanceContext');
     expect(runtimeSource).toContain('acknowledgeAgentWorkspaceDiagnosticsRemediationEscalation');

src/knowledge.api.contract.test.ts

@@ -28,6 +28,7 @@ describe('Knowledge mastery API contract wiring', () => {
             '/api/knowledge/operator/agent-workspace-diagnostics/triage/remediation/escalation/reminders',
             '/api/knowledge/operator/agent-workspace-diagnostics/triage/remediation/escalation/notification-policy',
             '/api/knowledge/operator/agent-workspace-diagnostics/triage/remediation/escalation/notifications',
+            '/api/knowledge/operator/agent-workspace-diagnostics/triage/remediation/escalation/notification-health',
             '/api/knowledge/operator/agent-workspace-diagnostics/triage/remediation/escalation/digest',
             '/api/knowledge/operator/agent-workspace-diagnostics/triage/remediation/escalation/acknowledge',
             '/api/knowledge/operator/agent-workspace-diagnostics/triage/remediation/approvals',

src/server.migration.test.ts

@@ -2310,6 +2310,144 @@ describe('server migration settings routes', () => {
     ).toBe(true);
   });
 
+  test('escalation notification health audit and stale cleanup stay deterministic', async () => {
+    const diagnosticsDir = path.join(runtimeDataDir, 'agent_workspace_diagnostics');
+    const approvalTrailPath = path.join(diagnosticsDir, 'triage_remediation_approvals.v1.json');
+    const escalationNotificationPath = path.join(diagnosticsDir, 'triage_remediation_escalation_notifications.v1.json');
+    await fs.promises.mkdir(diagnosticsDir, { recursive: true });
+    await fs.promises.rm(escalationNotificationPath, { force: true });
+    await fs.promises.writeFile(
+      approvalTrailPath,
+      JSON.stringify(
+        [
+          {
+            approvalId: 'awdr-m719-health-seed',
+            approvedAt: '2026-04-10T00:00:00.000Z',
+            approvedBy: 'op',
+            reason: 'm7.19 deterministic seed',
+            strategy: 'conservative',
+            window: 8,
+            expiresAt: '2020-01-01T00:00:00.000Z',
+            consumedAt: '',
+            simulation: {
+              generatedAt: '2026-04-10T00:00:00.000Z',
+              changedKeys: ['topReplayLimit'],
+              recommendedThresholds: {
+                replayRateMedium: 0.2,
+                replayRateHigh: 0.5,
+                includeFailureAsHighRisk: true,
+                topReplayLimit: 5,
+                historyWindow: 20,
+                source: 'seed',
+                updatedAt: '2026-04-10T00:00:00.000Z'
+              }
+            }
+          }
+        ],
+        null,
+        2
+      ),
+      'utf8'
+    );
+    await fs.promises.writeFile(
+      escalationNotificationPath,
+      JSON.stringify(
+        [
+          {
+            notificationId: 'stale-reminder-1',
+            generatedAt: '2020-01-01T00:00:00.000Z',
+            channel: 'reminder',
+            status: 'emitted',
+            escalationId: 'stale:resolved-escalation',
+            reminderId: 'reminder:stale:resolved-escalation',
+            digestId: '',
+            severity: 'warning',
+            title: 'stale reminder',
+            reason: '',
+            forceDispatch: false,
+            nextEligibleAt: '',
+            overdueHours: 4,
+            previousOverdueHours: 0,
+            currentOverdueHours: 4,
+            breaches: 0,
+            emittedReminders: 1,
+            suppressedReminders: 0,
+            runbookLinkIds: ['development-progress-dashboard']
+          },
+          {
+            notificationId: 'recent-digest-1',
+            generatedAt: '2026-04-16T00:00:00.000Z',
+            channel: 'digest',
+            status: 'emitted',
+            escalationId: '',
+            reminderId: '',
+            digestId: 'digest:recent',
+            severity: '',
+            title: 'recent digest',
+            reason: '',
+            forceDispatch: false,
+            nextEligibleAt: '',
+            overdueHours: 0,
+            previousOverdueHours: 0,
+            currentOverdueHours: 0,
+            breaches: 2,
+            emittedReminders: 1,
+            suppressedReminders: 0,
+            runbookLinkIds: ['development-progress-dashboard']
+          }
+        ],
+        null,
+        2
+      ),
+      'utf8'
+    );
+
+    const healthResponse = await requestJson(
+      port,
+      'GET',
+      '/api/knowledge/operator/agent-workspace-diagnostics/triage/remediation/escalation/notification-health?limit=10'
+    );
+    expect(healthResponse.status).toBe(200);
+    expect(healthResponse.body.success).toBe(true);
+    expect(healthResponse.body.cleanup).toEqual(
+      expect.objectContaining({
+        removed: expect.any(Number),
+        retained: expect.any(Number)
+      })
+    );
+    expect(healthResponse.body.cleanup.removed).toBeGreaterThan(0);
+    expect(healthResponse.body.health).toEqual(
+      expect.objectContaining({
+        activeEscalations: expect.any(Number),
+        trackedNotifications: expect.any(Number),
+        staleNotificationsRemoved: expect.any(Number),
+        byChannel: expect.objectContaining({
+          reminder: expect.any(Number),
+          digest: expect.any(Number)
+        }),
+        byStatus: expect.objectContaining({
+          emitted: expect.any(Number),
+          suppressed: expect.any(Number),
+          throttled: expect.any(Number)
+        })
+      })
+    );
+    expect(healthResponse.body.health.staleNotificationsRemoved).toBeGreaterThan(0);
+    expect(healthResponse.body.notifications).toEqual(expect.any(Array));
+    expect(
+      healthResponse.body.notifications.some((entry: any) => entry.notificationId === 'stale-reminder-1')
+    ).toBe(false);
+
+    const persistedNotificationTrail = JSON.parse(await fs.promises.readFile(escalationNotificationPath, 'utf8'));
+    expect(Array.isArray(persistedNotificationTrail)).toBe(true);
+    expect(
+      persistedNotificationTrail.some((entry: any) => entry.notificationId === 'stale-reminder-1')
+    ).toBe(false);
+    expect(
+      persistedNotificationTrail.some((entry: any) => entry.notificationId === 'recent-digest-1')
+    ).toBe(true);
+  });
+
   test('server runtime path avoids synchronous filesystem APIs', () => {
     const serverSourcePath = path.join(__dirname, 'server.ts');
     const serverSource = fs.readFileSync(serverSourcePath, 'utf8');