docs(observability): ApeRAG 可观测性需求对齐文档 v1

[earayu]

Summary

per earayu2 飞书 directive (via @飞书 DM msg=d3042add): 汇总近期可观测性相关需求, 整理详细文档, 跟其他人讨论并对齐 — 先不考虑如何实现, 讨论完成发 PR 合并.

核心驱动场景

• 太钢 token 消耗追问 (谭怀远 飞书群): PG 无 usage 表 / LiteLLM callback 禁用 / worker 不记 token → 答不上来
• 私有大模型 cost 担忧 (ou_1d75b5... 飞书): 客户私有部署 cost 是对外可见的运维必需品
• Singapore 大 PDF 现场: per-window 耗时 / entity-relation 计数无可查询数据
• 财务分摊 + 容量预测

文档定位

• ✅ alignment 指标 capability + 字段 + 维度 + 口径
• ❌ 不写实现 (实施 spec 见 task-89-telemetry-spec-v1.md PR docs(task-89): ApeRAG 全路径埋点 spec v1 #1951 P0 + future P1/P2 spec)

全路径 4 layer capability map

• Layer A: API 层 (P3 defer)
• Layer B: 索引 worker 层 (P1 worker lane lifecycle)
• Layer C: 业务执行层 (P0 graph extraction 2 metric + P1 vector/fulltext + P2 retrieval)
• Layer D: LLM 调用层 cross-cutting (P2 LLM ledger 独立 event family per Weston msg=94df61b3 — 优先级最高 P2 跟 P1 几乎同时启动)

LLM 调用账本字段对齐

llm_call_event 独立 event family:

• model_name / provider / prompt_tokens / completion_tokens / total_tokens / cost_usd / latency_ms / status / error_code
• call_purpose: chunking/extraction/embedding/answer/rerank/summarization
• 关联维度: collection_id / document_id / window_id / run_id / call_id

Privacy hard gate (跨 phase 共同基线)

• 永远不进 attrs: chunk_text / query_text / prompt_text / completion_text / error_message / traceback
• AST data-flow gate (NOT 全文 grep) 防误伤合法 extraction 路径
• error 类仅 error_code/error_type whitelist enum
• DISABLE_TELEMETRY 部署级 opt-out

Test plan / 跨域协作方对齐

• @符炫炜 spec author
• @不穷 PM 实施时机 + dispatch
• @earayu2 P0/P1/P2 trigger 决策 ratify
• @Planetegg SRE 部署可视化 + retention 容量
• @weston 架构师 cross-CR LLM ledger 独立 event family 设计
• @ziang index/worker producer 接入点
• @huangzhangshu testing privacy boundary scope + fail-safe
• @dongdong FE dashboard typed schema 三层区分
• @cuiwenbo FE typed schema sync 一致性

🤖 Generated with Claude Code

[earayu]

Testing / acceptance CR: LGTM ✅

I reviewed this as the macro requirements alignment doc, not an implementation spec. The important acceptance boundaries are clear enough:

• PR docs(task-89): ApeRAG 全路径埋点 spec v1 #1951 P0 remains scoped to graph extraction window/document metrics; this doc does not pull LLM ledger or dashboards into P0.
• The cross-layer correlation keys are sufficient for future tests and SQL examples: collection_id, document_id, parse_version, document_index_id, window_id, run_id, call_id, and call_purpose.
• LLM ledger is correctly treated as a separate P2 event family / ledger table and must cover both API process calls and indexing-worker calls, instead of relying on LiteLLM callback availability.
• Privacy hard gate is consistent with P0: metadata-only, no prompt/completion/chunk/query/entity-description text, and no raw error messages or tracebacks.
• The acceptance path is testable by phase: P0 via SQL/query + tests, P1 via admin aggregate schemas, P2 via ledger SQL + cost attribution examples.

Non-blocking NIT for future P2 spec: §3.2 lists LLM ledger fields as first-class fields, while §5 SQL examples read them from llc.attrs->>.... That is fine for a macro alignment doc, but the P2 implementation spec should choose one canonical shape (column vs JSON attrs) before tests are written, otherwise acceptance SQL and schema tests may drift.

No blocker from my lane.