feat: add insights page, analytics API, enhanced dashboard with spend trends

- Add /insights page with DAU, model adoption, efficiency rankings, MCP usage, file extensions, client versions
- Add /api/analytics, /api/team-spend, /api/model-costs, /api/groups endpoints
- Enhance dashboard with time range picker, billing cycle progress, daily spend chart, spend breakdown, model cost table
- Enhance user detail page with activity profile and improved charts
- Add Analytics API support to collector (DAU, models, agent edits, tabs, MCP, file extensions, client versions)
- Add billing groups collection from /teams/groups endpoint with cycle dates
- Add model name shortening (format-utils) and date formatting (date-utils)
- Improve anomaly detection with refined thresholds and trend analysis
- Update cursor-client with Analytics API endpoints and groups endpoint
- Update README with complete setup guide including detect step
- Update cursor rules with current architecture and conventions
- Add VS Code tasks for common workflows

Co-authored-by: Cursor <cursoragent@cursor.com>

Author: ofershap

.cursor/rules/coding-conventions.mdc

@@ -13,12 +13,8 @@ alwaysApply: false
 - Use `import type` for type-only imports
 
 ```typescript
-// CORRECT
 import { getDb } from "@/lib/db";
 import type { Anomaly } from "@/lib/types";
-
-// WRONG — will break Next.js build
-import { getDb } from "./db.js";
 ```
 
 ## Database
@@ -27,25 +23,29 @@ import { getDb } from "./db.js";
 - Use `getDb()` singleton — it initializes schema on first call
 - Use transactions for batch inserts: `db.transaction(() => { ... })()`
 - SQLite column names use snake_case; TypeScript types use camelCase
+- Key-value metadata stored in `metadata` table via `setMetadata()`/`getMetadata()`
 
 ## API Routes
 
 - All API routes use `export const dynamic = "force-dynamic"` (data is always fresh from SQLite)
 - Cron endpoint requires `x-cron-secret` header or `?secret=` query param
 - Use Next.js App Router async params pattern: `{ params }: { params: Promise<{ id: string }> }`
+- Most GET endpoints support `?days=N` query param for time range filtering
 
 ## Components
 
 - Dashboard pages: server component (page.tsx) fetches data, passes to client component (*-client.tsx)
 - Charts are always client components ("use client")
 - Use Tailwind CSS with zinc color palette (dark theme)
 - No shadcn/ui installed — components are hand-rolled
+- Model names displayed via `shortModel()` from `src/lib/format-utils.ts` with full name in tooltip
 
 ## Anomaly Detection
 
 - Three layers: thresholds → zscore → trends (all in `src/lib/anomaly/`)
 - `detector.ts` orchestrates all three, deduplicates by `userEmail:type:metric` key
 - New anomalies get inserted; existing ones that no longer fire get auto-resolved
+- Run via `npm run detect` CLI or as part of `POST /api/cron`
 
 ## Testing
 

.cursor/rules/cursor-api.mdc

@@ -42,6 +42,10 @@ Poll at most once per hour (data aggregated hourly). Max 30 days per request.
 ### POST /teams/user-spend-limit
 Body: `{ email, hardLimitDollars }`
 
+### POST /teams/groups
+Returns `{ groups: [{ id, name, memberCount, spendCents, members: [{ email, joinedAt }] }], subscriptionCycleStart, subscriptionCycleEnd }`
+Also provides billing cycle dates (cycle_start, cycle_end) stored in the metadata table.
+
 ## Analytics API Endpoints (separate key: CURSOR_ANALYTICS_API_KEY)
 
 All use `startDate`/`endDate` params (formats: `YYYY-MM-DD`, `7d`, `30d`, `today`, `yesterday`). Max 30 days. Default: last 7 days.

.cursor/rules/project-context.mdc

@@ -11,7 +11,7 @@ Open-source Cursor IDE usage monitoring, anomaly detection, and alerting for ent
 
 - Next.js (App Router, Turbopack) + TypeScript strict
 - SQLite via better-sqlite3 (zero-config, file at `data/tracker.db`)
-- Recharts for charts, Tailwind CSS for styling
+- Recharts for charts, Tailwind CSS for styling (zinc dark theme)
 - Vitest for tests, ESLint 9 flat config, Prettier
 - Docker (multi-stage Dockerfile + docker-compose.yml)
 
@@ -27,37 +27,60 @@ Cursor Enterprise APIs → Collector (src/lib/collector.ts) → SQLite (src/lib/
                                                     Dashboard (src/app/ pages)
 ```
 
-Single cron endpoint `POST /api/cron` does: collect → detect → alert in one call.
+Two CLI entry points:
+- `npm run collect` — fetch data from Cursor APIs into SQLite
+- `npm run detect` — run anomaly detection + alerting on stored data
+
+Single cron endpoint `POST /api/cron` does both: collect → detect → alert in one call.
 
 ## Key Files
 
 | File | Purpose |
 |------|---------|
 | `src/lib/types.ts` | All shared types + DetectionConfig + DEFAULT_CONFIG |
 | `src/lib/cursor-client.ts` | Cursor API client (Admin + Analytics), pagination, rate-limit retry |
-| `src/lib/db.ts` | SQLite schema, all queries, dashboard stats, user stats |
-| `src/lib/collector.ts` | Data collection pipeline (members, daily usage, spending, events) |
+| `src/lib/db.ts` | SQLite schema, all queries, dashboard stats, analytics, user stats |
+| `src/lib/collector.ts` | Data collection pipeline (members, daily usage, spending, events, analytics, groups) |
 | `src/lib/anomaly/detector.ts` | Orchestrator — runs all 3 detection layers, deduplicates |
 | `src/lib/anomaly/thresholds.ts` | Layer 1: static limits (spend, requests, tokens) |
 | `src/lib/anomaly/zscore.ts` | Layer 2: statistical z-score vs team mean |
 | `src/lib/anomaly/trends.ts` | Layer 3: personal spikes, drift above P75, model shift |
 | `src/lib/incidents.ts` | Incident lifecycle: create, alert, acknowledge, resolve + MTTD/MTTI/MTTR |
 | `src/lib/alerts/slack.ts` | Slack webhook with block-kit messages |
 | `src/lib/alerts/email.ts` | SMTP email with HTML templates |
+| `src/lib/date-utils.ts` | Date formatting helpers (formatDateShort, formatDateTick, formatDateLabel) |
+| `src/lib/format-utils.ts` | Model name shortening (MODEL_MAP regex → short labels) |
 | `src/app/api/cron/route.ts` | Main cron endpoint (collect + detect + alert) |
 
 ## Dashboard Pages
 
-- `/` — Team overview: stat cards, spend bar chart, usage line chart, members table
-- `/users/[email]` — Per-user: token timeline, model pie chart, feature breakdown, anomaly history
+- `/` — Team overview: stat cards, spend bar chart, daily spend trend, spend breakdown by user, members table with search/sort, time range picker (24h/3d/7d/14d/30d), billing cycle progress
+- `/insights` — Analytics: DAU chart, model adoption, model efficiency rankings, MCP tool usage, file extensions, client versions
+- `/users/[email]` — Per-user: token timeline, model pie chart, feature breakdown, activity profile, anomaly history
 - `/anomalies` — MTTD/MTTI/MTTR metrics, open incidents (acknowledge/resolve), anomaly table
 - `/settings` — Configurable detection thresholds
 
+## API Endpoints
+
+| Endpoint | Method | Purpose |
+|----------|--------|---------|
+| `/api/cron` | POST | Collect + detect + alert (requires x-cron-secret header) |
+| `/api/stats` | GET | Full dashboard data (supports `?days=N`) |
+| `/api/analytics` | GET | Analytics data: DAU, models, agent edits, tabs, MCP, etc. |
+| `/api/team-spend` | GET | Daily team spend breakdown |
+| `/api/model-costs` | GET | Model cost breakdown (users, avg/total spend, requests) |
+| `/api/groups` | GET | Billing groups with member counts and spend |
+| `/api/anomalies` | GET | Anomaly timeline (supports `?days=N`) |
+| `/api/users/[email]` | GET | Per-user statistics (supports `?days=N`) |
+| `/api/incidents/[id]` | PATCH | Acknowledge or resolve incident |
+| `/api/settings` | GET/PUT | Detection configuration |
+
 ## Database Tables
 
-members, daily_usage, spending, usage_events, anomalies, incidents, config, collection_log
+members, daily_usage, spending, usage_events, anomalies, incidents, config, collection_log, metadata, daily_spend, billing_groups, billing_group_members, analytics_dau, analytics_model_usage, analytics_agent_edits, analytics_tabs, analytics_mcp, analytics_file_extensions, analytics_client_versions
 
 ## Important Caveats
 
 - API response shapes may not exactly match the live Cursor API. If real responses differ, update `src/lib/types.ts` and `src/lib/cursor-client.ts`.
 - Trend detection can produce duplicate dedup keys (spike + drift both emit `trend:tokens` for same user) — first one wins, second is silently dropped.
+- CLI scripts (`npm run collect`, `npm run detect`) do NOT auto-load `.env` — use `source .env && export CURSOR_ADMIN_API_KEY` or run via the Next.js dev server cron endpoint instead.

.env.example

@@ -1,5 +1,4 @@
 CURSOR_ADMIN_API_KEY=key_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
-CURSOR_ANALYTICS_API_KEY=
 
 SLACK_WEBHOOK_URL=
 

.vscode/tasks.json

@@ -0,0 +1,98 @@
+{
+  "version": "2.0.0",
+  "tasks": [
+    {
+      "label": "Dev Server",
+      "type": "shell",
+      "command": "npm run dev -- --turbopack -p 3456",
+      "problemMatcher": [],
+      "isBackground": true,
+      "presentation": {
+        "group": "cursor-tracker",
+        "panel": "dedicated",
+        "showReuseMessage": false,
+        "clear": true,
+        "reveal": "always",
+        "focus": false
+      },
+      "icon": {
+        "id": "globe",
+        "color": "terminal.ansiCyan"
+      }
+    },
+    {
+      "label": "Collect Data",
+      "type": "shell",
+      "command": "npm run collect",
+      "problemMatcher": [],
+      "presentation": {
+        "group": "cursor-tracker",
+        "panel": "dedicated",
+        "showReuseMessage": false,
+        "clear": true,
+        "reveal": "always"
+      },
+      "icon": {
+        "id": "cloud-download",
+        "color": "terminal.ansiGreen"
+      }
+    },
+    {
+      "label": "Run Anomaly Detection",
+      "type": "shell",
+      "command": "npm run detect",
+      "problemMatcher": [],
+      "presentation": {
+        "group": "cursor-tracker",
+        "panel": "dedicated",
+        "showReuseMessage": false,
+        "clear": true,
+        "reveal": "always"
+      },
+      "icon": {
+        "id": "warning",
+        "color": "terminal.ansiYellow"
+      }
+    },
+    {
+      "label": "Collect + Detect",
+      "dependsOrder": "sequence",
+      "dependsOn": ["Collect Data", "Run Anomaly Detection"],
+      "problemMatcher": [],
+      "icon": {
+        "id": "zap",
+        "color": "terminal.ansiMagenta"
+      }
+    },
+    {
+      "label": "Type Check",
+      "type": "shell",
+      "command": "npm run typecheck",
+      "problemMatcher": ["$tsc"],
+      "presentation": {
+        "reveal": "always",
+        "panel": "shared"
+      },
+      "group": "build",
+      "icon": {
+        "id": "check",
+        "color": "terminal.ansiBlue"
+      }
+    },
+    {
+      "label": "Run Tests",
+      "type": "shell",
+      "command": "npm test",
+      "problemMatcher": [],
+      "presentation": {
+        "reveal": "always",
+        "panel": "shared"
+      },
+      "group": "test",
+      "icon": {
+        "id": "beaker",
+        "color": "terminal.ansiGreen"
+      }
+    }
+  ]
+}

README.md

@@ -80,12 +80,13 @@ Anomaly Detected ──→ Alert Sent ──→ Acknowledged ──→ Resolved
 
 ### Web Dashboard
 
-| Page               | What you see                                                             |
-| ------------------ | ------------------------------------------------------------------------ |
-| **Team Overview**  | Total spend, requests, tokens, top consumers, daily trends               |
-| **User Drilldown** | Per-user token timeline, model breakdown, feature usage, anomaly history |
-| **Anomalies**      | Open incidents, MTTD/MTTI/MTTR metrics, full anomaly timeline            |
-| **Settings**       | Configurable detection thresholds — no code changes needed               |
+| Page               | What you see                                                                                                                             |
+| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------- |
+| **Team Overview**  | Stat cards, spend by user, daily spend trend, spend breakdown, members table with search/sort, billing cycle progress, time range picker |
+| **Insights**       | DAU chart, model adoption trends, model efficiency rankings (cost/precision), MCP tool usage, file extensions, client versions           |
+| **User Drilldown** | Per-user token timeline, model breakdown, feature usage, activity profile, anomaly history                                               |
+| **Anomalies**      | Open incidents, MTTD/MTTI/MTTR metrics, full anomaly timeline                                                                            |
+| **Settings**       | Configurable detection thresholds — no code changes needed                                                                               |
 
 ---
 
@@ -123,16 +124,27 @@ CURSOR_ADMIN_API_KEY=your_admin_api_key
 SLACK_WEBHOOK_URL=https://hooks.slack.com/services/T.../B.../xxx
 
 # Optional
-CURSOR_ANALYTICS_API_KEY=your_analytics_key   # for DAU and model breakdowns
+CURSOR_ANALYTICS_API_KEY=your_analytics_key   # for Insights page (DAU, model breakdowns, MCP)
 CRON_SECRET=your_secret_here                  # protects the cron endpoint
-SMTP_HOST=smtp.gmail.com                      # for email alerts
+DASHBOARD_PASSWORD=your_password              # optional basic auth for the dashboard
+
+# Email alerts (optional)
+SMTP_HOST=smtp.gmail.com
 SMTP_PORT=587
 SMTP_USER=you@gmail.com
 SMTP_PASS=app_password
+SMTP_FROM=cursor-tracker@yourcompany.com
 ALERT_EMAIL_TO=team-lead@company.com
 ```
 
-### 3. Collect your first data
+### 3. Start the dashboard
+
+```bash
+npm run dev
+# Open http://localhost:3000
+```
+
+### 4. Collect your first data
 
 ```bash
 npm run collect
@@ -148,14 +160,19 @@ You should see:
   Usage events: 12,847
 ```
 
-### 4. Start the dashboard
+### 5. Run anomaly detection
+
+After collecting data, run detection separately:
 
 ```bash
-npm run dev
-# Open http://localhost:3000
+npm run detect
 ```
 
-### 5. Set up recurring collection
+This analyzes the stored data against all three detection layers and sends alerts for any anomalies found.
+
+> **Note:** `npm run collect` only fetches data. `npm run detect` only runs detection. The cron endpoint (`POST /api/cron`) does both in one call.
+
+### 6. Set up recurring collection
 
 Trigger the cron endpoint hourly (via crontab, GitHub Actions, or any scheduler):
 
@@ -206,7 +223,9 @@ Cursor Enterprise APIs
   ├── /teams/members
   ├── /teams/spend
   ├── /teams/daily-usage-data
-  └── /teams/filtered-usage-events
+  ├── /teams/filtered-usage-events
+  ├── /teams/groups
+  └── /analytics/team/*
           │
           ▼
     ┌─────────────┐     ┌──────────┐     ┌───────────────────┐
@@ -232,26 +251,30 @@ All detection thresholds are configurable via the Settings page or the API:
 
 | Setting              | Default | What it does                                      |
 | -------------------- | ------- | ------------------------------------------------- |
-| Max spend per cycle  | $50.00  | Alert when a user exceeds this in a billing cycle |
-| Max requests per day | 500     | Alert on excessive daily request count            |
+| Max spend per cycle  | $200    | Alert when a user exceeds this in a billing cycle |
+| Max requests per day | 200     | Alert on excessive daily request count            |
 | Max tokens per day   | 5M      | Alert on excessive daily token consumption        |
-| Z-score multiplier   | 2.0     | How many standard deviations above mean to flag   |
-| Z-score window       | 14 days | Historical window for statistical comparison      |
+| Z-score multiplier   | 2.5     | How many standard deviations above mean to flag   |
+| Z-score window       | 7 days  | Historical window for statistical comparison      |
 | Spike multiplier     | 3.0x    | Alert when today > N× user's personal average     |
 | Drift days above P75 | 3       | Consecutive days above team P75 to flag           |
 
 ---
 
 ## API Endpoints
 
-| Endpoint              | Method  | Description                                   |
-| --------------------- | ------- | --------------------------------------------- |
-| `/api/cron`           | POST    | Collect + detect + alert (use with scheduler) |
-| `/api/stats`          | GET     | Dashboard statistics                          |
-| `/api/anomalies`      | GET     | Anomaly timeline                              |
-| `/api/users/[email]`  | GET     | Per-user statistics                           |
-| `/api/incidents/[id]` | PATCH   | Acknowledge or resolve incident               |
-| `/api/settings`       | GET/PUT | Detection configuration                       |
+| Endpoint              | Method  | Description                                         |
+| --------------------- | ------- | --------------------------------------------------- |
+| `/api/cron`           | POST    | Collect + detect + alert (use with scheduler)       |
+| `/api/stats`          | GET     | Dashboard statistics (`?days=7`)                    |
+| `/api/analytics`      | GET     | Analytics data: DAU, models, MCP, etc. (`?days=30`) |
+| `/api/team-spend`     | GET     | Daily team spend breakdown                          |
+| `/api/model-costs`    | GET     | Model cost breakdown by users and spend             |
+| `/api/groups`         | GET     | Billing groups with member counts                   |
+| `/api/anomalies`      | GET     | Anomaly timeline (`?days=30`)                       |
+| `/api/users/[email]`  | GET     | Per-user statistics (`?days=30`)                    |
+| `/api/incidents/[id]` | PATCH   | Acknowledge or resolve incident                     |
+| `/api/settings`       | GET/PUT | Detection configuration                             |
 
 ---
 
@@ -286,15 +309,16 @@ npm run lint         # Lint + format check
 
 Requires a **Cursor Enterprise** plan. The tool uses these endpoints:
 
-| Endpoint                            | Auth              | What it provides                            |
-| ----------------------------------- | ----------------- | ------------------------------------------- |
-| `GET /teams/members`                | Admin API key     | Team member list                            |
-| `POST /teams/spend`                 | Admin API key     | Per-user spending data                      |
-| `POST /teams/daily-usage-data`      | Admin API key     | Daily usage metrics                         |
-| `POST /teams/filtered-usage-events` | Admin API key     | Detailed usage events with model/token info |
-| `GET /analytics/team/*`             | Analytics API key | DAU, model usage breakdowns (optional)      |
+| Endpoint                            | Auth              | What it provides                             |
+| ----------------------------------- | ----------------- | -------------------------------------------- |
+| `GET /teams/members`                | Admin API key     | Team member list                             |
+| `POST /teams/spend`                 | Admin API key     | Per-user spending data                       |
+| `POST /teams/daily-usage-data`      | Admin API key     | Daily usage metrics                          |
+| `POST /teams/filtered-usage-events` | Admin API key     | Detailed usage events with model/token info  |
+| `POST /teams/groups`                | Admin API key     | Billing groups + cycle dates                 |
+| `GET /analytics/team/*`             | Analytics API key | DAU, model usage, MCP, tabs, etc. (optional) |
 
-Rate limit: 250 requests/minute. The collector handles rate limiting with automatic retry.
+Rate limit: 20 requests/minute (Admin API), 100 requests/minute (Analytics API). The collector handles rate limiting with automatic retry.
 
 ---