Skip to content

Commit 8588eff

Browse files
tclaude
andcommitted
feat(triage): how_to lane staleness governance + notifications FAQ (PP-4831)
A how_to answer names UI ("go to Settings → Libraries") and has no fix version to expire against, so it goes stale silently when the app's navigation moves — exactly the Palace-icon → Settings reroute we just did. This adds the governance to catch that. - KBEntry gains `ui_surface` (the screen a how_to depends on) + `reviewed_at` (when the answer was last verified). Both nil for known_issue entries. - All four how_to entries are now anchored + dated; HT-003 switch-library and the add-library answers point at `settings-libraries`. - HowToGovernanceTests adds a UI-surface CHANGE LOG (surface → last-changed date) and a staleness lint: any how_to whose `reviewed_at` predates its surface's last-changed date fails — the screen moved and nobody re-checked the directions. `settings-libraries` is dated 2026-07-20 (today's reroute); the answers were re-reviewed the same day, so they pass — had they not been, the lint would have caught it. Teeth proven on the real catalog: rolling a review date back to before the change fails with the drift message. - New FAQ entry HT-2026-004-notifications (hold-ready + due-soon reminders — enable notifications; from HelpSpot 18103), with a benchmark case. Note on FAQ growth: several "common questions" (loan period, borrow limit, hold window) have LIBRARY-SPECIFIC answers (they vary per library's config), so they're poor canned FAQs — better routed to the library than answered with a hardcoded number. v1 adds the universal one (notifications); the rest want product input before authoring. TriageBotCore suite 248/248 green via `swift test`. **Scope:** TriageBotCore local package only — KBEntry anchor/date fields, catalog metadata + one new how_to entry, governance tests. No classifier/reducer/escalation change (that's PP-4832). No app/DRM/sign-in code. **Not done:** branched off the unmerged PP-4825 branch; re-stack after #1305 + PP-4825 land. Applying the UI anchor to known_issue text (which can also drift) is deferred. Holding PR until tomorrow per request — pushed for backup only. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 07b58cd commit 8588eff

5 files changed

Lines changed: 202 additions & 4 deletions

File tree

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
---
2+
name: pp-4831-howto-governance
3+
created: 2026-07-20
4+
author: claude-opus-4-8
5+
---
6+
7+
**ADR refs:** none — ForgeOS governance is off in this environment.
8+
9+
**Ticket:** PP-4831 (offshoot of PP-4825). Branched off
10+
`feat/pp-4825-triage-corpus-generalhelp` (independent of PP-4832). Re-stack after
11+
#1305 + PP-4825 land.
12+
13+
**Why:** a how_to answer references UI ("go to Settings → Libraries") and has no fix
14+
version to expire against, so it goes stale silently when the UI moves — exactly what
15+
happened with the Palace-icon → Settings library-switch reroute. This adds the
16+
governance the how_to lane needs to stay trustworthy, plus one new FAQ entry.
17+
18+
## Claims
19+
20+
- adds field `uiSurface` to `KBEntry` (the UI screen a how_to answer depends on)
21+
- adds field `reviewedAt` to `KBEntry` (the date the answer was last verified)
22+
- adds `ui_surface` + `reviewed_at` to the three existing how_to entries in
23+
`catalog.json` (HT-001 renewals, HT-002 return-early, HT-003 switch-library)
24+
- adds a how_to entry `HT-2026-004-notifications` to `catalog.json` (hold-ready +
25+
due-soon reminders — enable notifications; from HelpSpot 18103)
26+
- adds a benchmark shouldMatch case for HT-004 to `ResponseQualityTests`
27+
- adds `HowToGovernanceTests` — a UI-surface change-log registry + a staleness lint
28+
that fails any how_to whose `reviewed_at` predates its surface's last-changed date,
29+
requires every how_to to carry a known `ui_surface` + `reviewed_at`, and proves the
30+
lint has teeth on a synthetic drifted entry
31+
32+
## Anti-claims
33+
34+
- does NOT change the classifier, scorer, normalization, version gate, or reducer
35+
- does NOT change escalation behavior (that is PP-4832)
36+
- does NOT alter the existing how_to keywords or answer text (only adds anchor/date
37+
metadata) — except adding the new HT-004 entry
38+
- does NOT apply the ui_surface anchor to known_issue entries (how_to only for v1;
39+
KI text-drift is a separate concern)
40+
- does NOT touch sign-in, borrow, download, DRM, or audiobook production code
41+
42+
## Files in scope
43+
44+
- Palace/Packages/PalaceTriageBot/Sources/TriageBotCore/Models/KBEntry.swift
45+
- Palace/Packages/PalaceTriageBot/Sources/TriageBotCore/Resources/catalog.json
46+
- Palace/Packages/PalaceTriageBot/Tests/TriageBotCoreTests/ResponseQualityTests.swift
47+
- Palace/Packages/PalaceTriageBot/Tests/TriageBotCoreTests/HowToGovernanceTests.swift

Palace/Packages/PalaceTriageBot/Sources/TriageBotCore/Models/KBEntry.swift

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -95,6 +95,14 @@ public struct KBEntry: Codable, Equatable, Identifiable, Sendable {
9595
public let helpspotTag: String?
9696
public let trustLevel: KBTrustLevel
9797
public let visibility: KBVisibility
98+
/// The UI screen a `how_to` answer depends on (e.g. "settings-libraries").
99+
/// A how_to answer references the app's navigation and has no fix version to
100+
/// expire against, so it goes stale silently when the UI moves. This anchor
101+
/// lets a governance lint flag entries whose screen changed after they were
102+
/// last reviewed. Nil for known_issue entries (PP-4831).
103+
public let uiSurface: String?
104+
/// The date (YYYY-MM-DD) a how_to answer was last verified against the UI.
105+
public let reviewedAt: String?
98106

99107
/// `kind` with its default applied — `.knownIssue` when the JSON omitted it.
100108
public var resolvedKind: KBKind { kind ?? .knownIssue }
@@ -118,6 +126,8 @@ public struct KBEntry: Codable, Equatable, Identifiable, Sendable {
118126
case helpspotTag = "helpspot_tag"
119127
case trustLevel = "trust_level"
120128
case visibility = "visibility"
129+
case uiSurface = "ui_surface"
130+
case reviewedAt = "reviewed_at"
121131
}
122132

123133
public init(
@@ -138,7 +148,9 @@ public struct KBEntry: Codable, Equatable, Identifiable, Sendable {
138148
escalateAnyway: Bool = false,
139149
helpspotTag: String? = nil,
140150
trustLevel: KBTrustLevel = .authoritative,
141-
visibility: KBVisibility = .userFacing
151+
visibility: KBVisibility = .userFacing,
152+
uiSurface: String? = nil,
153+
reviewedAt: String? = nil
142154
) {
143155
self.id = id
144156
self.category = category
@@ -158,6 +170,8 @@ public struct KBEntry: Codable, Equatable, Identifiable, Sendable {
158170
self.helpspotTag = helpspotTag
159171
self.trustLevel = trustLevel
160172
self.visibility = visibility
173+
self.uiSurface = uiSurface
174+
self.reviewedAt = reviewedAt
161175
}
162176
}
163177

Palace/Packages/PalaceTriageBot/Sources/TriageBotCore/Resources/catalog.json

Lines changed: 38 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -754,7 +754,9 @@
754754
"escalate_anyway": false,
755755
"helpspot_tag": "how-to-renewals",
756756
"trust_level": "authoritative",
757-
"visibility": "user_facing"
757+
"visibility": "user_facing",
758+
"ui_surface": "catalog",
759+
"reviewed_at": "2026-07-20"
758760
},
759761
{
760762
"id": "HT-2026-002-return-early",
@@ -783,7 +785,9 @@
783785
"escalate_anyway": false,
784786
"helpspot_tag": "how-to-return-early",
785787
"trust_level": "authoritative",
786-
"visibility": "user_facing"
788+
"visibility": "user_facing",
789+
"ui_surface": "my-books",
790+
"reviewed_at": "2026-07-20"
787791
},
788792
{
789793
"id": "HT-2026-003-switch-library",
@@ -810,7 +814,38 @@
810814
"escalate_anyway": false,
811815
"helpspot_tag": "how-to-switch-library",
812816
"trust_level": "authoritative",
813-
"visibility": "user_facing"
817+
"visibility": "user_facing",
818+
"ui_surface": "settings-libraries",
819+
"reviewed_at": "2026-07-20"
820+
},
821+
{
822+
"id": "HT-2026-004-notifications",
823+
"category": "other",
824+
"kind": "how_to",
825+
"symptom_keywords": [
826+
"get notified",
827+
"notify me when",
828+
"hold ready notification",
829+
"remind me when",
830+
"due date reminder",
831+
"reminder when",
832+
"turn on notifications",
833+
"enable notifications",
834+
"when my book is due"
835+
],
836+
"user_facing_workaround": "Turn on notifications and Palace will alert you when a hold you placed is ready to borrow, and again when a loan is about to expire. You can enable them when you first install the app, or later in your phone's Settings under Palace.",
837+
"internal_reference": {
838+
"helpspot": [
839+
18103
840+
]
841+
},
842+
"confidence_threshold": 0.1,
843+
"escalate_anyway": false,
844+
"helpspot_tag": "how-to-notifications",
845+
"trust_level": "authoritative",
846+
"visibility": "user_facing",
847+
"ui_surface": "notifications-settings",
848+
"reviewed_at": "2026-07-20"
814849
}
815850
]
816851
}
Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
import XCTest
2+
@testable import TriageBotCore
3+
4+
/// PP-4831 — governance for the general-help (how_to) lane. A how_to answer names
5+
/// UI ("go to Settings → Libraries") and has no fix version to expire against, so
6+
/// it goes stale silently when the app's navigation moves. This pins that every
7+
/// how_to entry is anchored to a UI surface + carries a review date, and flags any
8+
/// answer that was last reviewed BEFORE its surface changed — i.e. the screen moved
9+
/// and nobody re-checked the directions.
10+
final class HowToGovernanceTests: XCTestCase {
11+
12+
/// The UI-surface change log. Bump a surface's date whenever its screen/nav
13+
/// changes; any how_to reviewed before that date then fails the staleness check
14+
/// until a human re-verifies the answer and bumps its `reviewed_at`.
15+
///
16+
/// `settings-libraries` last changed 2026-07-20 — the Palace-icon library
17+
/// switcher moved into Settings (PP-4825). The switch/add-library answers were
18+
/// re-reviewed the same day, so they pass; had they not been, this would fail.
19+
static let uiSurfaceChangeLog: [String: String] = [
20+
"settings-libraries": "2026-07-20",
21+
"my-books": "2026-05-01",
22+
"catalog": "2026-05-01",
23+
"notifications-settings": "2026-07-20",
24+
]
25+
26+
private func loadEntries() throws -> [KBEntry] {
27+
try BundledCatalogSource.loadCatalogSync().entries
28+
}
29+
30+
/// Entries whose answer was last reviewed before their anchored surface changed.
31+
/// ISO dates compare correctly as strings. Shared by the catalog test and the
32+
/// teeth test so both exercise the same logic.
33+
static func staleHowToEntries(_ entries: [KBEntry], changeLog: [String: String]) -> [String] {
34+
var stale: [String] = []
35+
for entry in entries where entry.resolvedKind == .howTo {
36+
guard let surface = entry.uiSurface,
37+
let reviewed = entry.reviewedAt,
38+
let changed = changeLog[surface] else { continue }
39+
if reviewed < changed {
40+
stale.append("\(entry.id): reviewed \(reviewed) < surface '\(surface)' changed \(changed)")
41+
}
42+
}
43+
return stale
44+
}
45+
46+
// MARK: - Structure: every how_to is anchored + dated + known
47+
48+
func testEveryHowTo_isAnchoredAndDated() throws {
49+
for entry in try loadEntries() where entry.resolvedKind == .howTo {
50+
XCTAssertNotNil(entry.uiSurface, "\(entry.id): how_to must declare a ui_surface it depends on")
51+
XCTAssertNotNil(entry.reviewedAt, "\(entry.id): how_to must carry a reviewed_at date")
52+
}
53+
}
54+
55+
func testEveryHowToSurface_isInTheChangeLog() throws {
56+
for entry in try loadEntries() where entry.resolvedKind == .howTo {
57+
guard let surface = entry.uiSurface else { continue }
58+
XCTAssertNotNil(Self.uiSurfaceChangeLog[surface],
59+
"\(entry.id): ui_surface '\(surface)' is not a known UI surface. Add it to the change log (and remove a surface that no longer exists).")
60+
}
61+
}
62+
63+
// MARK: - Drift: no how_to reviewed before its surface changed
64+
65+
func testNoHowTo_wasReviewedBeforeItsSurfaceChanged() throws {
66+
let stale = Self.staleHowToEntries(try loadEntries(), changeLog: Self.uiSurfaceChangeLog)
67+
XCTAssertTrue(stale.isEmpty,
68+
"how_to answers whose screen changed after they were last reviewed — re-verify the directions and bump reviewed_at:\n \(stale.joined(separator: "\n "))")
69+
}
70+
71+
// MARK: - The lint has teeth
72+
73+
func testStalenessLint_catchesADriftedEntry() {
74+
// A how_to reviewed in January, anchored to a surface that changed in July,
75+
// MUST be flagged — proof the check would fire when the UI moves under an
76+
// un-re-reviewed answer.
77+
let drifted = KBEntry(
78+
id: "HT-STALE", category: .library, kind: .howTo,
79+
symptomKeywords: ["switch library"],
80+
userFacingWorkaround: "Tap the (now-removed) top-left logo to switch libraries.",
81+
confidenceThreshold: 0.1, uiSurface: "settings-libraries", reviewedAt: "2026-01-01"
82+
)
83+
let stale = Self.staleHowToEntries([drifted], changeLog: Self.uiSurfaceChangeLog)
84+
XCTAssertEqual(stale.count, 1, "a Jan-reviewed answer on a July-changed surface must be flagged")
85+
}
86+
87+
func testFreshlyReviewedEntry_isNotFlagged() {
88+
let fresh = KBEntry(
89+
id: "HT-FRESH", category: .library, kind: .howTo,
90+
symptomKeywords: ["switch library"],
91+
userFacingWorkaround: "Go to Settings → Libraries and tap the one you want.",
92+
confidenceThreshold: 0.1, uiSurface: "settings-libraries", reviewedAt: "2026-07-20"
93+
)
94+
XCTAssertTrue(Self.staleHowToEntries([fresh], changeLog: Self.uiSurfaceChangeLog).isEmpty)
95+
}
96+
}

Palace/Packages/PalaceTriageBot/Tests/TriageBotCoreTests/ResponseQualityTests.swift

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -413,6 +413,12 @@ final class ResponseQualityTests: XCTestCase {
413413
expect: .shouldMatch(entryId: "HT-2026-003-switch-library"),
414414
source: "common patron question — multiple libraries in one app"),
415415

416+
// === HT-2026-004 notifications / reminders (how_to) ===
417+
Case(userText: "how do I get notified when my hold is ready",
418+
category: .other, context: nil,
419+
expect: .shouldMatch(entryId: "HT-2026-004-notifications"),
420+
source: "HelpSpot 18103 — hold-ready + due-soon reminders; enable notifications"),
421+
416422
// how_to negative — no FAQ answer exists; must still escalate, not
417423
// grab a loosely-related how_to.
418424
Case(userText: "how do I delete my account permanently",

0 commit comments

Comments
 (0)