Skip to content

API_MONITORING.md

Latest commit

 

History

History
138 lines (101 loc) · 3.46 KB

API_MONITORING.md

File metadata and controls

138 lines (101 loc) · 3.46 KB
Raw

API Monitoring

The partnership CLI now includes built-in API monitoring to track rate limits, errors, token usage, and performance.

Features

  • Real-time monitoring: All API calls are logged as they happen
  • Rate limit detection: Automatically detects rate limit errors
  • Token tracking: Counts prompt, completion, and total tokens
  • Performance metrics: Tracks latency, calls/minute, tokens/second
  • Persistent logs: All data saved to api_monitor.jsonl

Usage

During Runs

API monitoring is now automatic. Every call to the partnership CLI will log API metrics:

python partnership_cli.py design "Your problem"

At the end of the run, you'll see stats:

======================================================================
📊 API MONITOR STATS
======================================================================
Total calls:       12
Errors:            0 (0.0%)
Rate limits:       0
Total tokens:      45,234
Elapsed:           196.2s
Calls/minute:      3.67
Tokens/second:     230.58
Avg tokens/call:   3,770
======================================================================

View Recent API Calls

Show the last 20 API calls:

python partnership_cli.py design --show-log 20

Output:

📝 Last 20 API calls:
✅ 2026-02-07T21:15:30 | anthropic:claude-sonnet-4-20250514 | 4123 tokens | 12.3s | success
✅ 2026-02-07T21:15:45 | anthropic:claude-sonnet-4-20250514 | 3890 tokens | 11.8s | success
❌ 2026-02-07T21:16:02 | anthropic:claude-sonnet-4-20250514 | 0 tokens | 0.5s | rate_limited
   Error: rate_limit_error: Request rate limit exceeded

View Summary Stats

Get stats from the full log history:

python partnership_cli.py design --log-stats

Understanding the Logs

The api_monitor.jsonl file contains one JSON object per line:

{
  "timestamp": "2026-02-07T21:15:30.123456",
  "call_number": 1,
  "model": "anthropic:claude-sonnet-4-20250514",
  "prompt_tokens": 1234,
  "completion_tokens": 2890,
  "total_tokens": 4124,
  "latency": 12.3,
  "status": "success",
  "error": null,
  "prompt_length": 4936,
  "response_length": 11560
}

Status Values

  • success: API call completed successfully
  • error: API call failed (non-rate-limit error)
  • rate_limited: API returned a rate limit error (429)

Detecting Issues

Rate Limiting

The monitor detects rate limiting by looking for:

  • HTTP 429 status codes
  • Error messages containing "rate limit", "too many requests", "quota exceeded"

If you see rate limits:

  1. Check the calls_per_minute metric
  2. Add delays between calls if needed
  3. Consider batching or reducing parallelism

High Latency

If tokens_per_second is low or individual call latency is high:

  • May indicate API performance issues
  • Could be network latency
  • Might be model load (some models are slower)

Token Usage

Monitor avg_tokens_per_call to:

  • Optimize prompt size
  • Understand cost per operation
  • Plan capacity/budgets

Integration

The monitoring system can be used in other scripts:

from api_monitor import APIMonitor
from monitored_agent import MonitoredAgent

monitor = APIMonitor("my_log.jsonl")
agent = MonitoredAgent("You are helpful", "anthropic:claude-sonnet-4-20250514", monitor)

result = agent.run_sync("Hello!")

monitor.print_stats()

File Management

  • Logs append to api_monitor.jsonl (won't overwrite)
  • To reset, delete the file: rm api_monitor.jsonl
  • File can get large over time - consider archiving old logs