Fix/participant count consistency (#29)

* feat: add duration/participants/issues summary endpoints (Phase 2 + 3)

Five new endpoints for the remaining dashboard charts that fetch raw data:

- GET /v1/conferences/duration-summary
    Returns conference counts bucketed by duration range (< 1m, 1-3m, etc.)
- GET /v1/conferences/participant-count-summary
    Returns distribution of conferences by participant count
- GET /v1/issues/summary
    Returns issue counts grouped by code with titles
- GET /v1/issues/gum-summary
    Returns getusermedia_error issue counts grouped by error name

Also adds three new filter params to /v1/conferences for click-to-detail
modals on these charts:
- duration_gte, duration_lt (for duration chart)
- issue_code (for most-common-issues chart)

All endpoints accept appId, created_at_gte, created_at_lte and handle
both Python native ISO format and JavaScript's toISOString Z suffix.

Phases 2 and 3 of #20 — eliminates the need for the dashboard to
download all conferences (~38MB) and all issues (~73MB).

* feat: add connections + sessions summary endpoints (Phase 4 + 5 of #20)

Adds three new aggregation endpoints that let the dashboard stop
downloading full /connections and /sessions payloads to build charts
client-side:

- GET /v1/connections/summary — relay vs direct connection counts
  (replaces the Relayed-connections pie chart's client-side reduce)
- GET /v1/connections/setup-time-summary — connection setup-time
  buckets with per-bucket conference_ids for click-to-detail
- GET /v1/sessions/summary — browsers, OS, country, and city/geo
  aggregates (powers Browsers, OS, and Map charts in one roundtrip)

Also accepts `conference_ids=a,b,c` on /conferences so the setup-time
chart can page through matched conferences on click.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: 60s Redis cache + pre-warm for dashboard summary endpoints (Phase C of #20)

With Phases 0-5 merged, every dashboard chart reads from a server-side
aggregation endpoint. The SQL is fast with indexes, but the same ~8
queries run on every page load, and the heavy ones (sessions.summary,
connections.setup_time_summary) still cost 400-800ms on a live tenant.

Adds a thin caching layer in front of each summary view:

- `app/summary_cache.py` — `cached_json(endpoint, request, compute)`
  hashes (endpoint + filter params) into a short key, reads Redis,
  falls through to `compute()` on miss, and writes back with a 60s TTL.
  Redis failures are tolerated (settings already has IGNORE_EXCEPTIONS).
- Each of the eight summary views moves its existing compute body into
  a local `compute()` closure and returns through the helper. No change
  to the JSON shape, query logic, or error handling.
- `manage.py prewarm_summaries` — scheduled command that iterates apps
  with recent traffic (default: any conference in the last 2 days) and
  runs every summary view with the 30d-window filters the dashboard
  sends by default. Intended to run every ~30s as an ECS scheduled
  task so first visitors never see a cold miss.

Measured locally against a 7-day Production clone (~18k conferences /
38k sessions / 38k connections):

  endpoint                               cold      warm
  conferences/summary                    391ms  →   12ms   (33x)
  sessions/summary                       748ms  →   11ms   (68x)
  connections/setup_time_summary         373ms  →   11ms   (34x)
  conferences/participant_count_summary  216ms  →    7ms   (31x)
  issues/gum_summary                     107ms  →    6ms   (18x)
  connections/summary                     57ms  →    6ms   (9.5x)
  issues/summary                          45ms  →   86ms   (noise; both <100ms)
  conferences/duration_summary            19ms  →    8ms   (2.3x)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: dedupe conferences for issue_code; harden gum-summary

* test: summary_cache LocMem tests and prewarm smoke

- Unit-test cache key rules, hit/miss, TTL override, and soft-fail on get/set errors.
- Smoke-test prewarm_summaries for zero apps and one recent app (8 views).

Made-with: Cursor

* fix: bucket created_at_gte/created_at_lte to the minute in cache key

The dashboard sends `new Date().toISOString()` minus 30 days as
created_at_gte (web app.vue:189), which is millisecond-precise. With the
unrounded value going straight into _make_key, every page load — even
back-to-back reloads — produced a unique SHA1 digest and a fresh cache
miss, so the warm path never served real users:

  flush redis -> 0 keys
  load dashboard -> 8 keys
  reload         -> 16 keys (8 stale + 8 fresh)

prewarm_summaries had the same problem on the write side: its own
since_window = utcnow() - 30d advanced every run, so the entries it
populated never matched what the dashboard requested.

Truncate ISO timestamps to the minute (YYYY-MM-DDTHH:MM) before hashing,
so two requests in the same wall-clock minute share an entry. Correctness
still holds because the 60s TTL bounds staleness regardless of bucket
size.

After the fix, two same-minute dashboard loads both produce 8 keys (no
growth), and the 2nd load's slow endpoints serve from cache:

                                       before    after
  /v1/sessions/summary                 2314ms ->  411ms
  /v1/connections/setup-time-summary   1117ms ->  229ms
  /v1/conferences/summary               994ms ->   48ms

Two regression tests added covering the bucket and the minute boundary.

* Fix conference participant counts for dashboard and list

Count only active non-SFU participants with a session on the conference,
excluding peer stubs without sessions and SFU endpoints. Shared annotation
in conference_query.py used by conferences list and participant-count summary.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Fix participants_count when filtering conferences by participantId

Use a correlated Subquery instead of Count on the outer participants join.
Filtering by participantId narrowed that join to one person, so the
aggregate incorrectly showed 1 for multi-participant calls. The subquery
counts independently. Add regression test.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Clarify participantId filter regression test docstring

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Samson Nkrumah <samsonnkrumah253@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>

Author: 4 people

app/conference_query.py

@@ -0,0 +1,28 @@
+"""
+Query helpers for Conference-related aggregates.
+
+Peers created only for addConnection (P2P remote) are linked on conference.participants
+but often have no Session and no name; they are excluded. SFU endpoints (is_sfu) are
+excluded from participant totals — only human/clients with a session in the conference
+are counted.
+
+participants_count must use a Subquery (not Count on the outer queryset's participants
+relation): when /v1/conferences is filtered by participantId, the outer query joins
+participants and would otherwise restrict the aggregate to that single participant.
+"""
+
+from django.db.models import Count, IntegerField, OuterRef, Subquery
+
+from .models.participant import Participant
+
+PARTICIPANTS_COUNT_SUBQUERY = Subquery(
+    Participant.objects.filter(
+        conferences=OuterRef('pk'),
+        is_active=True,
+        is_sfu=False,
+        sessions__conference_id=OuterRef('pk'),
+    ).order_by().values('conferences').annotate(
+        cnt=Count('id', distinct=True),
+    ).values('cnt')[:1],
+    output_field=IntegerField(),
+)

app/tests/test_pr26_regressions.py

@@ -141,3 +141,41 @@ def test_gum_summary_skips_non_dict_issue_data(self):
         self.assertEqual(payload["total"], 1)
         self.assertEqual(payload["data"][0]["name"], "NotAllowedError")
         self.assertEqual(payload["data"][0]["count"], 1)
+
+    def test_conferences_participant_id_filter_preserves_full_participants_count(self):
+        """
+        When the conference list is filtered to one person (participantId),
+        participants_count must still reflect everyone in that conference, not 1.
+        """
+        conference = Conference.objects.create(
+            conference_id="conf-multi",
+            app=self.app,
+        )
+        participants = []
+        for i in range(3):
+            p = Participant.objects.create(
+                participant_id=f"user-{i}",
+                app=self.app,
+            )
+            p.conferences.add(conference)
+            Session.objects.create(
+                conference=conference,
+                participant=p,
+            )
+            participants.append(p)
+
+        response = self.client.get(
+            "/v1/conferences",
+            {
+                "appId": str(self.app.id),
+                "participantId": str(participants[0].id),
+                "limit": "50",
+            },
+        )
+
+        self.assertEqual(response.status_code, 200)
+        payload = json.loads(response.content)
+        self.assertEqual(payload["count"], 1)
+        self.assertEqual(len(payload["results"]), 1)
+        self.assertEqual(payload["results"][0]["id"], str(conference.id))
+        self.assertEqual(payload["results"][0]["participants_count"], 3)

app/views/conference_participant_count_summary_view.py

@@ -2,13 +2,11 @@
 from collections import Counter
 
 from django.core.exceptions import ValidationError
-from django.db.models import Count, IntegerField, OuterRef, Subquery
-
+from ..conference_query import PARTICIPANTS_COUNT_SUBQUERY
 from ..errors import INVALID_PARAMETERS, MISSING_PARAMETERS, PMError
 from ..summary_cache import cached_json
 from ..utils import JSONHttpResponse
 from ..models.conference import Conference
-from ..models.participant import Participant
 from .generic_view import GenericView
 
 
@@ -50,17 +48,7 @@ def compute():
             try:
                 per_conf = (Conference.objects
                     .filter(**filters)
-                    .annotate(
-                        participants_count=Subquery(
-                            Participant.objects.filter(
-                                conferences=OuterRef('pk'),
-                                is_active=True,
-                            ).order_by().values('conferences').annotate(
-                                cnt=Count('id', distinct=True)
-                            ).values('cnt')[:1],
-                            output_field=IntegerField(),
-                        ),
-                    )
+                    .annotate(participants_count=PARTICIPANTS_COUNT_SUBQUERY)
                     .values_list('participants_count', flat=True))
             except ValidationError:
                 raise PMError(status=400, app_error=INVALID_PARAMETERS)

app/views/conference_view.py

@@ -1,14 +1,14 @@
 import datetime
 
 from django.core.exceptions import ValidationError
-from django.db.models import Count, Exists, IntegerField, OuterRef, Subquery
+from django.db.models import Exists, OuterRef
 
+from ..conference_query import PARTICIPANTS_COUNT_SUBQUERY
 from ..errors import (INVALID_PARAMETERS, CONFERENCE_NOT_FOUND,
                       MISSING_PARAMETERS, PMError)
 from ..utils import JSONHttpResponse, serialize, paginate_and_serialize
 from ..models.conference import Conference
 from ..models.issue import Issue
-from ..models.participant import Participant
 from .generic_view import GenericView
 
 class ConferencesView(GenericView):
@@ -63,15 +63,7 @@ def filter(cls, request):
                 has_warnings=Exists(
                     Issue.objects.filter(conference=OuterRef('pk'), type='w', is_active=True)
                 ),
-                participants_count=Subquery(
-                    Participant.objects.filter(
-                        conferences=OuterRef('pk'),
-                        is_active=True,
-                    ).order_by().values('conferences').annotate(
-                        cnt=Count('id', distinct=True)
-                    ).values('cnt')[:1],
-                    output_field=IntegerField(),
-                ),
+                participants_count=PARTICIPANTS_COUNT_SUBQUERY,
             )
             if filter_by_issue_code:
                 objs = objs.distinct()