Analytics now count the developer portal's synthetic probe as partner traffic (~576 req/day vs ~487 real)

[PavelMakarchuk]

@anth-volk flagging this for you since you know the analytics layer best — it grew out of the developer-portal work and makes #1548 more urgent.

What happened

The developer portal runs a synthetic latency probe against the household API every 5 minutes (both channels), driven by Vercel Cron. Until last week it hit the unauthenticated /us/calculate_demo. On 2026-07-02 we switched it to the authenticated /us/calculate path using a dedicated M2M application, so the latency it measures matches what partners experience:

• Auth0 application: portal-latency-probe, client_id M5laaMnHNbCeZ9azRLdPDFOGLbysRIKR

Side effect: every probe request now lands in the calculate analytics as ordinary authenticated traffic.

Scale of the distortion

From the 90-day client inventory (PR #1602, tools/parity/inventory-report.json): the whole real fleet averages ~487 calculate requests/day across all clients. The probe adds 576/day (2 channels × 288 runs) — more than all real traffic combined. Any consumer of /analytics/calculate/requests (including the portal's staff usage panel) now sees:

• request volume roughly doubled
• error rates diluted ~2× (the probe rarely errors)
• resolved_channel mix pulled toward 50/50 (the probe hits current and frontier equally; real partners are almost entirely on current)
• the probe's canonical payload variables inflating the ?unique=true variable counts by 576/day

Options

1. Land Analytics: expose stored client_id via /analytics/calculate/requests (filter + output) to distinguish callers #1548 (expose + filter client_id on the endpoint). Consumers can then exclude the probe's client_id (and other internal/testing clients) themselves. This is the most general fix and also unblocks the portal's per-partner usage view.
2. Additionally or alternatively, maintain a server-side exclusion list (or an exclude_client_ids= / internal=false parameter) so fleet-level summaries are clean by default rather than relying on every consumer knowing the probe's client_id.
3. Tag synthetic traffic at ingest (e.g. flag requests from known internal client_ids) so it can never be conflated, regardless of query shape.

Happy to contribute the #1548 implementation if that is the preferred direction — the columns and indexes already exist per that issue, so it is an endpoint/serialization change only.

🤖 Generated with Claude Code

[PavelMakarchuk]

Two more data points from the 90-day inventory that the exclusion/attribution design should account for:

1. Interactive-user traffic is attributed by user sub, not application client_id. Staff sandbox runs from the developer portal appear in the analytics as google-oauth2|… rows (e.g. two staff identities with 68 and 52 requests). So "internal traffic" isn't only M2M client_ids — any exclusion list or client_id filter should expect user-sub identifiers too, and portal-driven traffic will grow this category.

2. Dormant applications with zero traffic in 90 days: SBN Household Calculator (wWJ66Cpw…) and CTD production (461tlpIy…) had no requests at all, while CTD development shows 339. If SBN/CTD-prod are no longer active integrations, consider decommissioning the Auth0 apps — dormant M2M credentials can still mint valid tokens. Also three clients with real traffic we couldn't name from the Auth0 app list alone: STaztFgt… (931), WyWSiIJE… (854), and NaQpYc3e… ("External Client Testing App", 299) — worth labeling while you're in there.

🤖 Generated with Claude Code