feat: Redis cache + pre-warm for dashboard summary endpoints (Phase C of #20) (#28)

* feat: add duration/participants/issues summary endpoints (Phase 2 + 3)

Five new endpoints for the remaining dashboard charts that fetch raw data:

- GET /v1/conferences/duration-summary
    Returns conference counts bucketed by duration range (< 1m, 1-3m, etc.)
- GET /v1/conferences/participant-count-summary
    Returns distribution of conferences by participant count
- GET /v1/issues/summary
    Returns issue counts grouped by code with titles
- GET /v1/issues/gum-summary
    Returns getusermedia_error issue counts grouped by error name

Also adds three new filter params to /v1/conferences for click-to-detail
modals on these charts:
- duration_gte, duration_lt (for duration chart)
- issue_code (for most-common-issues chart)

All endpoints accept appId, created_at_gte, created_at_lte and handle
both Python native ISO format and JavaScript's toISOString Z suffix.

Phases 2 and 3 of #20 — eliminates the need for the dashboard to
download all conferences (~38MB) and all issues (~73MB).

* feat: add connections + sessions summary endpoints (Phase 4 + 5 of #20)

Adds three new aggregation endpoints that let the dashboard stop
downloading full /connections and /sessions payloads to build charts
client-side:

- GET /v1/connections/summary — relay vs direct connection counts
  (replaces the Relayed-connections pie chart's client-side reduce)
- GET /v1/connections/setup-time-summary — connection setup-time
  buckets with per-bucket conference_ids for click-to-detail
- GET /v1/sessions/summary — browsers, OS, country, and city/geo
  aggregates (powers Browsers, OS, and Map charts in one roundtrip)

Also accepts `conference_ids=a,b,c` on /conferences so the setup-time
chart can page through matched conferences on click.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: 60s Redis cache + pre-warm for dashboard summary endpoints (Phase C of #20)

With Phases 0-5 merged, every dashboard chart reads from a server-side
aggregation endpoint. The SQL is fast with indexes, but the same ~8
queries run on every page load, and the heavy ones (sessions.summary,
connections.setup_time_summary) still cost 400-800ms on a live tenant.

Adds a thin caching layer in front of each summary view:

- `app/summary_cache.py` — `cached_json(endpoint, request, compute)`
  hashes (endpoint + filter params) into a short key, reads Redis,
  falls through to `compute()` on miss, and writes back with a 60s TTL.
  Redis failures are tolerated (settings already has IGNORE_EXCEPTIONS).
- Each of the eight summary views moves its existing compute body into
  a local `compute()` closure and returns through the helper. No change
  to the JSON shape, query logic, or error handling.
- `manage.py prewarm_summaries` — scheduled command that iterates apps
  with recent traffic (default: any conference in the last 2 days) and
  runs every summary view with the 30d-window filters the dashboard
  sends by default. Intended to run every ~30s as an ECS scheduled
  task so first visitors never see a cold miss.

Measured locally against a 7-day Production clone (~18k conferences /
38k sessions / 38k connections):

  endpoint                               cold      warm
  conferences/summary                    391ms  →   12ms   (33x)
  sessions/summary                       748ms  →   11ms   (68x)
  connections/setup_time_summary         373ms  →   11ms   (34x)
  conferences/participant_count_summary  216ms  →    7ms   (31x)
  issues/gum_summary                     107ms  →    6ms   (18x)
  connections/summary                     57ms  →    6ms   (9.5x)
  issues/summary                          45ms  →   86ms   (noise; both <100ms)
  conferences/duration_summary            19ms  →    8ms   (2.3x)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: dedupe conferences for issue_code; harden gum-summary

* test: summary_cache LocMem tests and prewarm smoke

- Unit-test cache key rules, hit/miss, TTL override, and soft-fail on get/set errors.
- Smoke-test prewarm_summaries for zero apps and one recent app (8 views).

Made-with: Cursor

* fix: bucket created_at_gte/created_at_lte to the minute in cache key

The dashboard sends `new Date().toISOString()` minus 30 days as
created_at_gte (web app.vue:189), which is millisecond-precise. With the
unrounded value going straight into _make_key, every page load — even
back-to-back reloads — produced a unique SHA1 digest and a fresh cache
miss, so the warm path never served real users:

  flush redis -> 0 keys
  load dashboard -> 8 keys
  reload         -> 16 keys (8 stale + 8 fresh)

prewarm_summaries had the same problem on the write side: its own
since_window = utcnow() - 30d advanced every run, so the entries it
populated never matched what the dashboard requested.

Truncate ISO timestamps to the minute (YYYY-MM-DDTHH:MM) before hashing,
so two requests in the same wall-clock minute share an entry. Correctness
still holds because the 60s TTL bounds staleness regardless of bucket
size.

After the fix, two same-minute dashboard loads both produce 8 keys (no
growth), and the 2nd load's slow endpoints serve from cache:

                                       before    after
  /v1/sessions/summary                 2314ms ->  411ms
  /v1/connections/setup-time-summary   1117ms ->  229ms
  /v1/conferences/summary               994ms ->   48ms

Two regression tests added covering the bucket and the minute boundary.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: agonza1 <albertogontras@gmail.com>

Author: 3 people

README.md

@@ -987,6 +987,8 @@ Private endpoints are used by the web interface to query data. They require user
   - `GET`, query parameters:
     - `appId`: Filter by app
     - `participantId`: Filter by participant
+    - `issue_code`: Filter conferences that contain at least one active issue with this code
+      - Returns each conference once even if multiple matching issues exist
 
 - `/conferences/<uuid:pk>`: Get a specific conference
   - `GET`

app/management/commands/prewarm_summaries.py

@@ -0,0 +1,98 @@
+"""
+Pre-warm the dashboard-summary Redis cache so first visitors don't pay
+the cold-query tax. Intended to run every ~30s via ECS scheduled task.
+
+For each active app that has seen recent data, walks every summary view
+with the same (appId, created_at_gte) filter the dashboard sends by
+default (last 30 days). The views themselves populate the cache on miss.
+"""
+import datetime
+import logging
+import time
+
+from django.core.management.base import BaseCommand
+from django.test.client import RequestFactory
+
+from app.models.app import App
+from app.models.conference import Conference
+
+from app.views.conference_summary_view import ConferenceSummaryView
+from app.views.conference_duration_summary_view import ConferenceDurationSummaryView
+from app.views.conference_participant_count_summary_view import ConferenceParticipantCountSummaryView
+from app.views.issue_summary_view import IssueSummaryView, GetUserMediaSummaryView
+from app.views.connection_summary_view import ConnectionSummaryView, ConnectionSetupTimeSummaryView
+from app.views.session_summary_view import SessionSummaryView
+
+logger = logging.getLogger(__name__)
+
+# (label, view)
+VIEWS = [
+    ('conferences.summary',                  ConferenceSummaryView),
+    ('conferences.duration_summary',         ConferenceDurationSummaryView),
+    ('conferences.participant_count_summary', ConferenceParticipantCountSummaryView),
+    ('issues.summary',                       IssueSummaryView),
+    ('issues.gum_summary',                   GetUserMediaSummaryView),
+    ('connections.summary',                  ConnectionSummaryView),
+    ('connections.setup_time_summary',       ConnectionSetupTimeSummaryView),
+    ('sessions.summary',                     SessionSummaryView),
+]
+
+
+class Command(BaseCommand):
+    help = 'Pre-compute dashboard summary responses and cache them'
+
+    def add_arguments(self, parser):
+        parser.add_argument(
+            '--days',
+            type=int,
+            default=30,
+            help='Window to warm (default: 30 days, matches dashboard default)',
+        )
+        parser.add_argument(
+            '--active-within-days',
+            type=int,
+            default=2,
+            help='Only warm apps that saw a conference in the last N days (default: 2)',
+        )
+
+    def handle(self, *args, **options):
+        window_days = options['days']
+        active_within = options['active_within_days']
+
+        since_window = datetime.datetime.utcnow() - datetime.timedelta(days=window_days)
+        active_since = datetime.datetime.utcnow() - datetime.timedelta(days=active_within)
+
+        # Apps with any conference in the recent window — skip tenants with
+        # no traffic so warming doesn't scan their cold tables.
+        recent_app_ids = (Conference.objects
+            .filter(created_at__gte=active_since)
+            .values_list('app_id', flat=True)
+            .distinct())
+        apps = App.objects.filter(id__in=list(recent_app_ids), is_active=True)
+        count = apps.count()
+        self.stdout.write(f'Warming {count} active apps ({window_days}d window)')
+
+        rf = RequestFactory()
+        created_at_gte = since_window.isoformat() + 'Z'
+        warmed = 0
+        failed = 0
+
+        for app in apps:
+            for label, view_cls in VIEWS:
+                req = rf.get('/v1/' + label, {
+                    'appId': str(app.id),
+                    'created_at_gte': created_at_gte,
+                })
+                started = time.monotonic()
+                try:
+                    view_cls.get(req)
+                    warmed += 1
+                except Exception as e:
+                    failed += 1
+                    logger.warning('prewarm %s for app %s failed: %s', label, app.id, e)
+                elapsed_ms = (time.monotonic() - started) * 1000
+                if elapsed_ms > 500:
+                    self.stdout.write(f'  slow: {label} app={app.id} {elapsed_ms:.0f}ms')
+
+        self.stdout.write(f'Warmed {warmed} summary entries across {count} apps' +
+                          (f' ({failed} failed)' if failed else ''))

app/summary_cache.py

@@ -0,0 +1,93 @@
+"""
+Short-TTL Redis cache for dashboard summary endpoints.
+
+The summary endpoints run SQL aggregations that are fast enough on their own
+but get called by every dashboard page load. Cache the computed JSON for
+~60 seconds so concurrent viewers share the same roll-up.
+
+Keys are derived from the endpoint name + all request filters, so different
+date ranges / apps get independent entries. TTL-only — no explicit
+invalidation — because the data is strictly additive (new conferences,
+sessions, issues arrive over time) and a sub-minute staleness window is
+acceptable for a dashboard.
+"""
+import hashlib
+import json
+import logging
+
+from django.conf import settings
+from django.core.cache import cache
+
+logger = logging.getLogger(__name__)
+
+DEFAULT_TTL_SECONDS = 60
+KEY_PREFIX = 'summary'
+
+# The query-string params that factor into the cache key for each endpoint.
+# Anything not listed here is ignored (e.g. trailing slashes, user agent, etc).
+CACHE_KEY_PARAMS = (
+    'appId',
+    'created_at_gte',
+    'created_at_lte',
+    'conferenceId',
+    'participantId',
+)
+
+# Params whose ISO timestamps should be truncated to the minute before hashing,
+# so that the dashboard's millisecond-precise `now - 30d` doesn't produce a
+# unique cache key per page load. Bucketing means two requests in the same
+# wall-clock minute share an entry; correctness still holds because the cache
+# entry's own TTL bounds staleness regardless of bucket size.
+BUCKETED_PARAMS = ('created_at_gte', 'created_at_lte')
+
+
+def _bucket_minute(value):
+    # ISO 8601: 2026-03-28T17:13:18.382Z -> 2026-03-28T17:13Z (first 16 chars
+    # are YYYY-MM-DDTHH:MM). Anything that doesn't match the layout is
+    # passed through unchanged so the key still differentiates malformed input.
+    if not value or len(value) < 16 or value[10] != 'T' or value[13] != ':':
+        return value
+    return value[:16] + 'Z'
+
+
+def _make_key(endpoint, request):
+    parts = [endpoint]
+    for name in CACHE_KEY_PARAMS:
+        val = request.GET.get(name)
+        if val:
+            if name in BUCKETED_PARAMS:
+                val = _bucket_minute(val)
+            parts.append(f'{name}={val}')
+    raw = '|'.join(parts)
+    # Keep key short but unique; include a readable prefix for ops visibility.
+    digest = hashlib.sha1(raw.encode()).hexdigest()[:16]
+    return f'{KEY_PREFIX}:{endpoint}:{digest}'
+
+
+def get_ttl():
+    return getattr(settings, 'SUMMARY_CACHE_TTL', DEFAULT_TTL_SECONDS)
+
+
+def cached_json(endpoint, request, compute):
+    """
+    Returns (payload_dict, was_cached_bool).
+
+    `compute` is called only on cache miss and must return the JSON-serializable
+    dict the endpoint would have returned.
+    """
+    key = _make_key(endpoint, request)
+    try:
+        cached = cache.get(key)
+    except Exception as e:
+        logger.warning('summary cache get failed for %s: %s', key, e)
+        cached = None
+
+    if cached is not None:
+        return cached, True
+
+    payload = compute()
+    try:
+        cache.set(key, payload, timeout=get_ttl())
+    except Exception as e:
+        logger.warning('summary cache set failed for %s: %s', key, e)
+    return payload, False

app/tests/__init__.py

@@ -0,0 +1,143 @@
+import json
+
+from django.test import Client, TestCase
+
+from app.models.app import App
+from app.models.conference import Conference
+from app.models.issue import Issue
+from app.models.organization import Organization
+from app.models.participant import Participant
+from app.models.session import Session
+
+
+class PR26RegressionTests(TestCase):
+    def setUp(self):
+        self.client = Client()
+        self.organization = Organization.objects.create(name="Test Org")
+        self.app = App.objects.create(
+            name="Test App",
+            api_key="a" * 32,
+            organization=self.organization,
+        )
+
+    def _make_conference_graph(self, conference_id):
+        conference = Conference.objects.create(
+            conference_id=conference_id,
+            app=self.app,
+        )
+        participant = Participant.objects.create(
+            participant_id=f"{conference_id}-participant",
+            app=self.app,
+        )
+        participant.conferences.add(conference)
+        session = Session.objects.create(
+            conference=conference,
+            participant=participant,
+        )
+        return conference, participant, session
+
+    def test_conferences_issue_code_filter_returns_distinct_conferences(self):
+        conference, participant, session = self._make_conference_graph("conf-1")
+
+        Issue.objects.create(
+            session=session,
+            conference=conference,
+            participant=participant,
+            type=Issue.TYPES_OF_ISSUES["warning"],
+            code="getusermedia_error",
+            data={"name": "NotFoundError"},
+        )
+        Issue.objects.create(
+            session=session,
+            conference=conference,
+            participant=participant,
+            type=Issue.TYPES_OF_ISSUES["warning"],
+            code="getusermedia_error",
+            data={"name": "NotFoundError"},
+        )
+
+        response = self.client.get(
+            "/v1/conferences",
+            {
+                "appId": str(self.app.id),
+                "issue_code": "getusermedia_error",
+                "limit": "50",
+            },
+        )
+
+        self.assertEqual(response.status_code, 200)
+        payload = json.loads(response.content)
+        self.assertEqual(payload["count"], 1)
+        self.assertEqual(len(payload["results"]), 1)
+        self.assertEqual(payload["results"][0]["id"], str(conference.id))
+
+    def test_conferences_issue_code_filter_ignores_inactive_issues(self):
+        conference_active, participant_active, session_active = self._make_conference_graph("conf-active")
+        conference_inactive, participant_inactive, session_inactive = self._make_conference_graph("conf-inactive")
+
+        Issue.objects.create(
+            session=session_active,
+            conference=conference_active,
+            participant=participant_active,
+            type=Issue.TYPES_OF_ISSUES["warning"],
+            code="getusermedia_error",
+            data={"name": "NotFoundError"},
+            is_active=True,
+        )
+        Issue.objects.create(
+            session=session_inactive,
+            conference=conference_inactive,
+            participant=participant_inactive,
+            type=Issue.TYPES_OF_ISSUES["warning"],
+            code="getusermedia_error",
+            data={"name": "NotReadableError"},
+            is_active=False,
+        )
+
+        response = self.client.get(
+            "/v1/conferences",
+            {
+                "appId": str(self.app.id),
+                "issue_code": "getusermedia_error",
+                "limit": "50",
+            },
+        )
+
+        self.assertEqual(response.status_code, 200)
+        payload = json.loads(response.content)
+        returned_ids = {row["id"] for row in payload["results"]}
+        self.assertEqual(returned_ids, {str(conference_active.id)})
+        self.assertEqual(payload["count"], 1)
+
+    def test_gum_summary_skips_non_dict_issue_data(self):
+        conference, participant, session = self._make_conference_graph("conf-gum")
+
+        Issue.objects.create(
+            session=session,
+            conference=conference,
+            participant=participant,
+            type=Issue.TYPES_OF_ISSUES["warning"],
+            code="getusermedia_error",
+            data={"name": "NotAllowedError", "message": "Permission denied"},
+        )
+        Issue.objects.create(
+            session=session,
+            conference=conference,
+            participant=participant,
+            type=Issue.TYPES_OF_ISSUES["warning"],
+            code="getusermedia_error",
+            data="malformed",
+        )
+
+        response = self.client.get(
+            "/v1/issues/gum-summary",
+            {
+                "appId": str(self.app.id),
+            },
+        )
+
+        self.assertEqual(response.status_code, 200)
+        payload = json.loads(response.content)
+        self.assertEqual(payload["total"], 1)
+        self.assertEqual(payload["data"][0]["name"], "NotAllowedError")
+        self.assertEqual(payload["data"][0]["count"], 1)

app/tests/test_pr26_regressions.py

@@ -0,0 +1,41 @@
+"""
+Smoke tests for prewarm_summaries management command (PR #28).
+"""
+from io import StringIO
+
+from django.core.management import call_command
+from django.test import TestCase
+
+from app.models.app import App
+from app.models.conference import Conference
+from app.models.organization import Organization
+
+
+class PrewarmSummariesSmokeTests(TestCase):
+    def test_runs_with_no_qualifying_apps(self):
+        out = StringIO()
+        err = StringIO()
+        call_command('prewarm_summaries', stdout=out, stderr=err)
+
+        combined = out.getvalue() + err.getvalue()
+        self.assertIn('Warming 0 active apps', combined)
+        self.assertIn('Warmed 0 summary entries across 0 apps', combined)
+
+    def test_runs_for_one_app_with_recent_conference(self):
+        org = Organization.objects.create(name='Prewarm Org')
+        app = App.objects.create(
+            name='Prewarm App',
+            api_key='b' * 32,
+            organization=org,
+        )
+        Conference.objects.create(
+            conference_id='prewarm-conf-1',
+            app=app,
+        )
+
+        out = StringIO()
+        call_command('prewarm_summaries', stdout=out, stderr=StringIO())
+        text = out.getvalue()
+
+        self.assertIn('Warming 1 active apps', text)
+        self.assertIn('Warmed 8 summary entries across 1 apps', text)