Skip to content

Support GitHub Copilot CLI and improve CLI accounting#141

Open
zhangzqs wants to merge 5 commits intoPiebald-AI:mainfrom
zhangzqs:copilot/support-copilot-cli-statistics
Open

Support GitHub Copilot CLI and improve CLI accounting#141
zhangzqs wants to merge 5 commits intoPiebald-AI:mainfrom
zhangzqs:copilot/support-copilot-cli-statistics

Conversation

@zhangzqs
Copy link
Copy Markdown

@zhangzqs zhangzqs commented Apr 9, 2026

Summary

  • split GitHub Copilot (VS Code) and GitHub Copilot CLI into separate analyzers/applications so they are tracked as distinct tools
  • improve Copilot CLI accounting by using shutdown model metrics for completed sessions and better cumulative-context estimation for active sessions
  • fill in Copilot CLI reasoning token tracking, plus add several built-in model prices and aliases observed in local data

What changed

  • added a dedicated GitHub Copilot CLI analyzer and Application::CopilotCli
  • kept the VS Code Copilot analyzer focused on chat session files only
  • made Copilot CLI parsing turn-aware and applied exact session.shutdown.data.modelMetrics usage to completed sessions
  • improved live-session estimates using accumulated context and compaction prompt overhead
  • populated Copilot CLI Reason Tks from reasoningText without changing the existing output/cost semantics
  • updated aggregation/session rollups so assistant messages are counted by role even when the model field is temporarily missing
  • added pricing entries/aliases for locally observed models, including reordered Claude names and extra built-in model mappings

Validation

  • cargo fmt --all --quiet
  • cargo build --quiet
  • cargo test --quiet
  • cargo clippy --quiet -- -D warnings
  • cargo doc --quiet

Notes

  • this PR was put together with a lot of vibe coding on real local Copilot CLI data, but the regression tests and verification steps are real and usable
  • pricing/alias coverage was also expanded for additional built-in models seen locally

Summary by CodeRabbit

Release Notes

  • New Features

    • Added support for GitHub Copilot CLI session logs as a data source
    • Added support for additional AI models: GPT-5.4-Pro, Kimi-K2.5, DuBao Seed 2.0, MiniMax M2.1, and LongCat Flash Lite
  • Documentation

    • Updated documentation and UI to distinguish GitHub Copilot CLI from GitHub Copilot (VS Code)

Copilot AI and others added 5 commits April 8, 2026 03:04
Agent-Logs-Url: https://github.com/zhangzqs/splitrail/sessions/4ee476d9-1122-4625-b054-0d5921bb214e

Co-authored-by: zhangzqs <34616640+zhangzqs@users.noreply.github.com>
… precise token tracking

Separate Copilot CLI parsing from the VS Code analyzer into its own
module, add exact output token counts, context compaction handling,
and model-based cost calculation.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Apr 9, 2026

📝 Walkthrough

Walkthrough

This PR introduces GitHub Copilot CLI analyzer support by implementing a new analyzer that discovers and parses Copilot CLI session event logs, updates model definitions with new canonical models and aliases, refines message-processing logic to gate AI-message aggregation on message role, and integrates the new analyzer into the registry.

Changes

Cohort / File(s) Summary
Documentation Updates
README.md, vscode-splitrail/README.md, src/tui.rs, src/mcp/server.rs
Updated tool lists and descriptions to include "GitHub Copilot CLI" alongside GitHub Copilot and other supported tools.
Copilot CLI Analyzer Implementation
src/analyzers/copilot_cli.rs
New 1131-line analyzer module that discovers, parses, and reconstructs GitHub Copilot CLI session events from JSONL logs, with session/project identity hashing, token counting (reusable/cache/visible), tool call tracking, and cost attribution logic using shutdown metrics or static prompt overhead.
Copilot Analyzer Refactoring
src/analyzers/copilot.rs
Increased crate-visibility for helper functions (count_tokens, is_probably_tool_json_text, extract_model_from_model_id), replaced local vscode_forks array with module-level constant reference, and added explicit type annotation for sources vector.
Analyzer Module Integration
src/analyzers/mod.rs, src/main.rs
Registered new copilot_cli module and re-exported CopilotCliAnalyzer, added analyzer instantiation and registry integration in main.
Message Processing Logic
src/contribution_cache/single_session.rs, src/tui/logic.rs, src/utils.rs
Refactored AI-message counting and per-model aggregation to gate on MessageRole::Assistant instead of model presence alone, ensuring user messages are not counted as AI messages.
Model Registry Expansion
src/models.rs
Added 6 new canonical models (gpt-5.4-pro, kimi-k2.5, doubao-seed-2.0-code, minimax-m2.1, longcat-flash-lite, auto) with pricing/caching configuration and expanded MODEL_ALIASES with dotted, provider-prefixed, and alternative forms.
Application Type Extension
src/types.rs
Added CopilotCli variant to Application enum for application identification.
Test Suite
src/analyzers/tests/copilot.rs, src/analyzers/tests/copilot_cli.rs, src/analyzers/tests/mod.rs, src/utils/tests.rs
Added comprehensive tests for Copilot CLI analyzer (321 lines: registry exposure, path recognition, JSONL parsing, event reconstruction, token/cost calculation), negative assertions for Copilot VS Code exclusion, and AI-message aggregation behavior.

Sequence Diagram(s)

sequenceDiagram
    participant FileSystem as File System
    participant Discovery as Data Discovery
    participant Parser as JSONL Parser
    participant Reconstructor as Event Reconstructor
    participant Calculator as Token & Cost Calculator
    participant Output as Message Output

    FileSystem->>Discovery: Discover ~/.copilot/{session-state,history-session-state}
    Discovery->>Discovery: Filter events.jsonl files
    Discovery->>Parser: Load & read JSONL events
    Parser->>Parser: Parse JSON objects sequentially
    Parser->>Reconstructor: Feed parsed events
    Reconstructor->>Reconstructor: Correlate user.message with assistant/tool events
    Reconstructor->>Reconstructor: Build conversation turns
    Reconstructor->>Calculator: Extract session ID & workspace identity
    Calculator->>Calculator: Compute session/project hashes
    Calculator->>Calculator: Estimate tokens (reusable/cache/visible)
    Calculator->>Calculator: Accumulate tool call stats
    Calculator->>Calculator: Apply shutdown metrics or static overhead
    Calculator->>Output: Emit ConversationMessage entries
    Output->>Output: Generate deterministic per-message hashes
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

Suggested reviewers

  • mike1858
  • bl-ue

Poem

🐰 A CLI's tale unfolds so bright,
With session logs parsed through the night,
Events reconstructed, tokens aglow,
Copilot's wisdom now we know!
New models join the vibrant crew,
And costs are calculated true. ✨

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Title check ✅ Passed The title directly and accurately summarizes the two main changes: adding support for GitHub Copilot CLI as a separate analyzer and improving CLI accounting logic.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@zhangzqs
Copy link
Copy Markdown
Author

zhangzqs commented Apr 9, 2026

image

coderabbitai[bot]
coderabbitai bot previously requested changes Apr 9, 2026
Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
src/analyzers/copilot_cli.rs (2)

541-624: Consider grouping parameters into a context struct.

The flush_copilot_cli_turn function has 9 parameters. While the #[allow(clippy::too_many_arguments)] annotation acknowledges this, grouping related parameters (like conversation_hash, project_hash, session_name) into a context struct would improve readability and make future modifications easier.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/analyzers/copilot_cli.rs` around lines 541 - 624, The function
flush_copilot_cli_turn currently takes many related params—group
conversation_hash, project_hash, session_name (and optionally
user_index/assistant_index and live_context) into a small context struct to
reduce argument count and improve readability; create a CopilotCliFlushContext
(or similar) containing conversation_hash: String (or &str reference type
matching callers), project_hash: String, session_name: Option<String>, and if
helpful mutable indices user_index: usize and assistant_index: usize (or keep
indices separate if you prefer), update flush_copilot_cli_turn signature to
accept &mut CopilotCliFlushContext (and keep entries, current_turn,
live_context, pending_user as before), then update all call sites to build/pass
that context struct and adjust usage inside flush_copilot_cli_turn to reference
context.conversation_hash, context.project_hash, context.session_name, and
context.assistant_index (or &mut assistant_index) accordingly.

751-755: Using Utc::now() for missing timestamps may affect reproducibility.

When an event lacks a timestamp, the current time is used as a fallback. This means parsing the same file at different times could produce different date values for those messages. Consider logging a warning or using a sentinel value if reproducibility is a concern.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/analyzers/copilot_cli.rs` around lines 751 - 755, The code sets
pending_user.date via event_timestamp.unwrap_or_else(Utc::now), which makes
parsing non-deterministic; change CopilotCliPendingUser.date to use a
reproducible sentinel or optional timestamp instead of calling Utc::now(), and
emit a warning when the event timestamp is missing; specifically, update the
creation of pending_user (the pending_user assignment and CopilotCliPendingUser
struct) to accept either Option<DateTime<Utc>> or a fixed sentinel (e.g., epoch)
and add a log call when event_timestamp.is_none() so missing timestamps are
recorded for debugging and reproducibility.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/tui.rs`:
- Line 1177: Update the no-data screen label that currently reads "GitHub
Copilot" to "GitHub Copilot (VS Code)" so it matches the analyzer split and
README wording; locate the string literal in src/tui.rs (the quoted message
shown) and replace that occurrence, and also search for other user-facing
occurrences of the plain "GitHub Copilot" label in the UI/strings to make the
naming consistent with "GitHub Copilot (VS Code)".

---

Nitpick comments:
In `@src/analyzers/copilot_cli.rs`:
- Around line 541-624: The function flush_copilot_cli_turn currently takes many
related params—group conversation_hash, project_hash, session_name (and
optionally user_index/assistant_index and live_context) into a small context
struct to reduce argument count and improve readability; create a
CopilotCliFlushContext (or similar) containing conversation_hash: String (or
&str reference type matching callers), project_hash: String, session_name:
Option<String>, and if helpful mutable indices user_index: usize and
assistant_index: usize (or keep indices separate if you prefer), update
flush_copilot_cli_turn signature to accept &mut CopilotCliFlushContext (and keep
entries, current_turn, live_context, pending_user as before), then update all
call sites to build/pass that context struct and adjust usage inside
flush_copilot_cli_turn to reference context.conversation_hash,
context.project_hash, context.session_name, and context.assistant_index (or &mut
assistant_index) accordingly.
- Around line 751-755: The code sets pending_user.date via
event_timestamp.unwrap_or_else(Utc::now), which makes parsing non-deterministic;
change CopilotCliPendingUser.date to use a reproducible sentinel or optional
timestamp instead of calling Utc::now(), and emit a warning when the event
timestamp is missing; specifically, update the creation of pending_user (the
pending_user assignment and CopilotCliPendingUser struct) to accept either
Option<DateTime<Utc>> or a fixed sentinel (e.g., epoch) and add a log call when
event_timestamp.is_none() so missing timestamps are recorded for debugging and
reproducibility.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: fc410277-c73f-425d-ab4b-eed5041508f1

📥 Commits

Reviewing files that changed from the base of the PR and between f5ef1f4 and c3ab1b3.

📒 Files selected for processing (17)
  • README.md
  • src/analyzers/copilot.rs
  • src/analyzers/copilot_cli.rs
  • src/analyzers/mod.rs
  • src/analyzers/tests/copilot.rs
  • src/analyzers/tests/copilot_cli.rs
  • src/analyzers/tests/mod.rs
  • src/contribution_cache/single_session.rs
  • src/main.rs
  • src/mcp/server.rs
  • src/models.rs
  • src/tui.rs
  • src/tui/logic.rs
  • src/types.rs
  • src/utils.rs
  • src/utils/tests.rs
  • vscode-splitrail/README.md

@zhangzqs zhangzqs changed the title Split GitHub Copilot CLI from VS Code Copilot and improve CLI accounting Support GitHub Copilot CLI and improve CLI accounting Apr 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants