Paper Reference
- Title: InconLens: Interactive Visual Diagnosis of Behavioral Inconsistencies in LLM-based Agentic Systems
- Authors: Shuo Yan, Xiaolin Wen, Shaolun Ruan, et al.
- Year: 2026
- URL: https://arxiv.org/abs/2603.28106
- Venue: arXiv preprint
Paper Summary
A visual analytics system for interactive diagnosis of LLM-based agentic systems with focus on cross-run behavioral analysis. Introduces "information nodes" as intermediate abstractions capturing canonical informational milestones across executions, enabling semantic alignment of reasoning trajectories for comparison.
Proposed Feature
Implement behavioral divergence visualization that compares multiple agent sessions to identify where and why behaviors diverge:
Core Capabilities
- Session Comparison View: Side-by-side or overlay view of multiple agent sessions aligned by semantic milestones (not just time)
- Divergence Highlighting: Automatically detect and highlight points where agent behavior diverges between runs
- Information Node Abstraction: Extract canonical milestones from each session (e.g., "retrieved context", "made decision", "called tool") for alignment
- Divergence Metrics: Quantify behavioral similarity between sessions with drill-down into specific divergence causes
Technical Approach
- Add session comparison endpoint to the API
- Implement semantic milestone extraction in the SDK
- Build comparison visualization component in the React frontend
- Integrate with existing drift detection feature for enhanced cross-session analysis
Impact
Directly enhances Peaky Peek's existing drift detection with principled visualization, making it much easier to understand why agent behavior changed between versions or configurations.
Labels
enhancement, paper-inspired, frontend
Paper Reference
Paper Summary
A visual analytics system for interactive diagnosis of LLM-based agentic systems with focus on cross-run behavioral analysis. Introduces "information nodes" as intermediate abstractions capturing canonical informational milestones across executions, enabling semantic alignment of reasoning trajectories for comparison.
Proposed Feature
Implement behavioral divergence visualization that compares multiple agent sessions to identify where and why behaviors diverge:
Core Capabilities
Technical Approach
Impact
Directly enhances Peaky Peek's existing drift detection with principled visualization, making it much easier to understand why agent behavior changed between versions or configurations.
Labels
enhancement, paper-inspired, frontend