Gemini Tokens Usage in Comment (/Stats)

[vivekkairi]

TL;DR

Token data on Vertex AI is quite bad and it takes too much effort to check the consumption. Use the /stats command in gemini-cli to get the token information and output it at the end of review comment and maybe if we have some pricing api can use to get exact price as well.

I can pick this up, if it sounds good.

Detailed design

No response

Additional information

No response

[jerop]

@vivekkairi we'd appreciate your contribution here 🙏🏾

1. let's add this as a markdown dropdown so it doesn't clutter the comments
2. it would be interesting to consider how we add this to other workflows as well
3. adding pricing information is interesting, but i think that would go in /stats upstream, and on this surface we just rely on /stats
4. are you planning to use the slash commands directly in your implementation? that's not supported in interactive mode yet - Commands Should work in Non-Interactive Mode google-gemini/gemini-cli#5435 (cc @abhipatel12 @NTaylorMullen @jackwotherspoon - here's another use case, this time for in-built slash commands)

[vivekkairi]

@vivekkairi we'd appreciate your contribution here 🙏🏾

1. let's add this as a markdown dropdown so it doesn't clutter the comments
2. it would be interesting to consider how we add this to other workflows as well
3. adding pricing information is interesting, but i think that would go in /stats upstream, and on this surface we just rely on /stats
4. are you planning to use the slash commands directly in your implementation? that's not supported in interactive mode yet - Slash Commands (Custom Commands) Fail in Non-Interactive/Headless Mode google-gemini/gemini-cli#5435 (cc @abhipatel12 @NTaylorMullen @jackwotherspoon - here's another use case, this time for in-built slash commands)

Checked the 4th issue, it does make it a lot harder to do. I am doing some experiments with expect if I can make it work but doesn't seem like we can without non interactive mode.

Also we are running gemini --yolo --prompt {prompt}, the session will expire after this so need a way to hook it in this command only.

[jerop]

@vivekkairi it sounds like we may need some changes upstream? if so, happy to help there as well

cc @leehagoodjames

[leehagoodjames]

@vivekkairi this would be great if /stats was provided in a json object to the invoker, for various downstream use cases

[vivekkairi]

@jerop @leehagoodjames I forgot gemini-cli is also open source, will check the implementation and come back with any queries than will proceed with the upstream changes.

[vivekkairi]

~/Projects/gemini-cli main !3 ?1 ❯ npm start -- --prompt "hi" --stats                                                                                6s 12:00:02

> @google/gemini-cli@0.1.21 start
> node scripts/start.js --prompt hi --stats

Checking build status...
Build is up-to-date.
(node:96624) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
Hi! How can I help you today?


--- Session Stats ---
=== Session Stats ===

Session ID: 37eb4da9-a654-494c-9664-439fd6814459
Wall Time: 6.3s
Agent Active: 6.0s

Interaction Summary:
  Tool Calls: 0 (✓ 0 × 0)
  Success Rate: 0.0%

Performance:
  API Time: 6.0s (100.0%)
  Tool Time: 0s (0.0%)

Model Usage:
Model                        Reqs   Input Tokens  Output Tokens
---------------------------------------------------------------
gemini-2.0-flash                2         22,062             76

Savings: 9,966 (45.2%) input tokens served from cache

@jerop @leehagoodjames what do you feel about this type of implementation? Let me know if we want to move this discussion to gemini-cli repo. Can have it as json for easier downstream implementations.

I am chaining it with --prompt because each non-interactive commands executes a different session so doing --prompt and --stats seperately leads to 0 token output