Skip to content

chore(task-207): close GenieBuilder analytics dashboard task #1006

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
stephanj wants to merge 1 commit into
base: master
Choose a base branch
from
Open

chore(task-207): close GenieBuilder analytics dashboard task #1006

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 33 additions & 11 deletions ...executed-events-in-analytics-dashboard.md → ...executed-events-in-analytics-dashboard.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,10 @@ id: TASK-207
title: >-
GenieBuilder: surface DevoxxGenie prompt_executed events in analytics
dashboard
status: To Do
status: Done
assignee: []
created_date: '2026-04-13 09:34'
updated_date: '2026-04-13 11:00'
labels:
- analytics
- geniebuilder
Expand Down Expand Up @@ -77,15 +78,36 @@ This task is implemented in `/Users/stephan/IdeaProjects/GenieBuilder`, NOT in D

## Acceptance Criteria
<!-- AC:BEGIN -->
- [ ] #1 functions/src/analytics.ts TRACKED_EVENTS allowlist includes 'prompt_executed'
- [ ] #2 GA4 Data API queries in analyticsReport (functions/src/index.ts) accept and filter by an 'app_name' dimension
- [ ] #3 analyticsReport endpoint accepts an 'appName' query parameter; omitting it preserves the existing GenieBuilder Electron behavior (backwards compatible)
- [ ] #4 New provider and model breakdown queries aggregate events of type 'prompt_executed' (in addition to existing 'provider_selected' / 'model_selected' breakdowns) so actual usage is measured, not just selection intent
- [ ] #5 Angular admin UI in web-admin/src/app/features/analytics/ has an app selector (tab or dropdown) with options: GenieBuilder (Electron), DevoxxGenie (IntelliJ), All
- [ ] #6 Selecting an app in the UI passes appName through to the analyticsReport endpoint and updates all charts and tables
- [ ] #7 UI clearly labels intent-signal breakdowns ('Models selected') separately from actual-usage breakdowns ('Prompts dispatched') so they cannot be confused
- [ ] #8 CSV export honors the active app filter and includes the new breakdown columns
- [ ] #9 Unit tests cover the new query branches in functions/src/analytics.ts (allowlist, app filter, prompt_executed breakdowns)
- [x] #1 functions/src/analytics.ts TRACKED_EVENTS allowlist includes 'prompt_executed'
- [x] #2 GA4 Data API queries in analyticsReport (functions/src/index.ts) accept and filter by an 'app_name' dimension
- [x] #3 analyticsReport endpoint accepts an 'appName' query parameter; omitting it preserves the existing GenieBuilder Electron behavior (backwards compatible)
- [x] #4 New provider and model breakdown queries aggregate events of type 'prompt_executed' (in addition to existing 'provider_selected' / 'model_selected' breakdowns) so actual usage is measured, not just selection intent
- [x] #5 Angular admin UI in web-admin/src/app/features/analytics/ has an app selector (tab or dropdown) with options: GenieBuilder (Electron), DevoxxGenie (IntelliJ), All
- [x] #6 Selecting an app in the UI passes appName through to the analyticsReport endpoint and updates all charts and tables
- [x] #7 UI clearly labels intent-signal breakdowns ('Models selected') separately from actual-usage breakdowns ('Prompts dispatched') so they cannot be confused
- [x] #8 CSV export honors the active app filter and includes the new breakdown columns
- [x] #9 Unit tests cover the new query branches in functions/src/analytics.ts (allowlist, app filter, prompt_executed breakdowns)
- [ ] #10 End-to-end validation: a synthetic prompt_executed event with app_name=devoxxgenie-intellij sent through the Cloudflare worker appears in the DevoxxGenie view of the dashboard within GA4's normal latency window
- [ ] #11 Cross-repo work is implemented in /Users/stephan/IdeaProjects/GenieBuilder on a feature branch following GenieBuilder's branching conventions, not in DevoxxGenieIDEAPlugin
- [x] #11 Cross-repo work is implemented in /Users/stephan/IdeaProjects/GenieBuilder on a feature branch following GenieBuilder's branching conventions, not in DevoxxGenieIDEAPlugin
<!-- AC:END -->

## Final Summary

<!-- SECTION:FINAL_SUMMARY:BEGIN -->
Implemented in cross-repo PR https://github.com/devoxx/GenieBuilder/pull/336 on branch `feature/task-207-devoxxgenie-prompt-executed-analytics` in /Users/stephan/IdeaProjects/GenieBuilder.

**Cloud Functions (`functions/src/`)**
- `analytics.ts` — added `prompt_executed` to `TRACKED_EVENTS`, exported `AppName` type, extended `fetchAnalyticsReport(start, end, appName?)` with a `withAppFilter` helper that wraps each query in an `andGroup` filtering on `customEvent:app_name` when `appName` is set. Omitting `appName` is byte-identical to before (backwards compatible). Added two new parallel GA4 queries for prompt-executed provider/model breakdowns. `AnalyticsReport` now carries `appName`, `promptProviderBreakdown`, `promptModelBreakdown`.
- `index.ts` — `analyticsReport` handler reads `req.query.appName`, validates against the allowlist (`devoxxgenie-intellij`, `geniebuilder-electron`), 400s on bad input, and forwards.
- `analytics.test.ts` — bumped mocked parallel call count 9 → 11 across all tests; added 5 new tests (allowlist, prompt provider/model parsing, app filter present/absent).

**Web admin (`web-admin/src/app/`)**
- `analytics.model.ts` — added `AppFilter` union, `APP_FILTER_LABELS`, `promptProviderBreakdown`/`promptModelBreakdown` fields, `prompt_executed` event label, and `prompt_executed` in the `LLM Usage` category.
- `analytics.service.ts` — new `selectedApp` signal + `setApp()`, appName forwarded in `loadReport()` URL, `topPromptProviders`/`topPromptModels` computed signals, CSV export scoped per app with intent and actual breakdown sections (with proper escaping).
- `analytics-overview.ts` — `mat-button-toggle-group` app selector at the top of the toolbar, provider and model breakdowns each render two side-by-side cards labeled "Models selected (intent)" vs "Prompts dispatched (actual)" with tooltips and a distinct fill colour.
- Spec files updated and 4 new component tests added (selector renders all 3 options, prompt-executed breakdowns render).

**Verification**: functions `npm test` 51 pass / 3 pre-existing failures unchanged from main; functions `npm run build` clean; web-admin `ng test` 101 pass / 3 pre-existing failures unchanged; web-admin `ng build` succeeds with same budget warnings as main; `tsc --noEmit` clean.

**AC #10 (E2E synthetic event validation)** is post-deploy and will be confirmed once Cloud Functions are deployed and the existing `prompt_executed` event (already in GA4 from TASK-206) surfaces in the DevoxxGenie tab's "Prompts dispatched (actual)" card after the GA4 latency window.
<!-- SECTION:FINAL_SUMMARY:END -->
Loading