Skip to content

Commit f9863ab

Browse files
authored
feat(core): add emitPerformanceMetric bridge for runtime telemetry (heygen-com#393)
## Summary Extend the runtime analytics bridge with a numeric performance metric channel. Hosts subscribe via the existing postMessage transport (one bridge, two channels) and aggregate per-session p50 / p95 for scrub latency, sustained fps, dropped frames, decoder count, composition load time, and media sync drift before forwarding to their observability pipeline. This is the foundation other perf tooling sits on — the player itself emits the events; player-side aggregation and flush land in a follow-up. ## Why Step `X-1` of the player perf proposal. Today there is no way for an embedding host to learn that scrub latency spiked, that a composition took 3 s to load, or that the media-sync loop is running 200 ms behind real time. The only signals are anecdotal user reports. A single shared bridge keeps the runtime → host surface area minimal: hosts that already wire up the analytics channel get perf for free, and hosts that don't aren't paying for it. ## What changed - New `emitPerformanceMetric(name, value, tags?)` helper in `@hyperframes/core` that forwards a `{ type: "performance-metric", name, value, tags }` envelope through the existing analytics postMessage transport. - Six initial metric names defined in the proposal: - `scrub_latency_ms` — wall-clock from `seek()` call to first paint at the new frame. - `playback_fps` — sustained rAF cadence during play. - `dropped_frames` — count of >25 ms gaps within a play window. - `decoder_count` — number of concurrently-decoding video elements. - `composition_load_ms` — navigation-start to player-ready. - `media_sync_drift_ms` — drift between expected and actual decoder time. - Each emit also writes a `performance.mark()` with `{ value, tags }` on `detail`, so the same numbers surface in the DevTools Performance panel's User Timing track for local debugging without instrumenting the host. - Zero PostHog (or any other analytics SDK) dependency in `core` — the host decides where to forward the events. ## Test plan - [x] Unit tests cover the envelope shape, the `performance.mark` mirror, and the no-op path when no host has wired up the bridge. - [x] Manual: verified marks appear in the User Timing track when scrubbing the studio preview. ## Stack Step `X-1` of the player perf proposal. Foundation for the perf gate (P0-1a/b/c) — the perf scenarios in this stack instrument these same channels for CI measurement.
1 parent ef26798 commit f9863ab

3 files changed

Lines changed: 179 additions & 12 deletions

File tree

packages/core/src/runtime/analytics.test.ts

Lines changed: 92 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
import { describe, it, expect, vi, beforeEach } from "vitest";
2-
import { initRuntimeAnalytics, emitAnalyticsEvent } from "./analytics";
2+
import { initRuntimeAnalytics, emitAnalyticsEvent, emitPerformanceMetric } from "./analytics";
33

44
describe("runtime analytics", () => {
55
let postMessage: ReturnType<typeof vi.fn>;
@@ -58,3 +58,94 @@ describe("runtime analytics", () => {
5858
expect(postMessage).toHaveBeenCalledTimes(events.length);
5959
});
6060
});
61+
62+
describe("runtime performance metrics", () => {
63+
let postMessage: ReturnType<typeof vi.fn>;
64+
65+
beforeEach(() => {
66+
postMessage = vi.fn();
67+
initRuntimeAnalytics(postMessage);
68+
// Clean up DevTools marks between tests to avoid cross-test interference.
69+
if (typeof performance !== "undefined" && typeof performance.clearMarks === "function") {
70+
performance.clearMarks();
71+
}
72+
});
73+
74+
it("emits a perf metric via postMessage", () => {
75+
emitPerformanceMetric("player_scrub_latency", 12.5);
76+
expect(postMessage).toHaveBeenCalledWith({
77+
source: "hf-preview",
78+
type: "perf",
79+
name: "player_scrub_latency",
80+
value: 12.5,
81+
tags: {},
82+
});
83+
});
84+
85+
it("passes tags through", () => {
86+
emitPerformanceMetric("player_decoder_count", 3, {
87+
composition_id: "abc123",
88+
mode: "isolated",
89+
});
90+
expect(postMessage).toHaveBeenCalledWith({
91+
source: "hf-preview",
92+
type: "perf",
93+
name: "player_decoder_count",
94+
value: 3,
95+
tags: { composition_id: "abc123", mode: "isolated" },
96+
});
97+
});
98+
99+
it("normalizes missing tags to an empty object", () => {
100+
emitPerformanceMetric("player_playback_fps", 60);
101+
expect(postMessage).toHaveBeenCalledWith(expect.objectContaining({ tags: {} }));
102+
});
103+
104+
it("supports zero and negative values", () => {
105+
emitPerformanceMetric("player_dropped_frames", 0);
106+
emitPerformanceMetric("player_media_sync_drift", -8.3);
107+
expect(postMessage).toHaveBeenNthCalledWith(1, expect.objectContaining({ value: 0 }));
108+
expect(postMessage).toHaveBeenNthCalledWith(2, expect.objectContaining({ value: -8.3 }));
109+
});
110+
111+
it("does not throw when postMessage is not set", () => {
112+
initRuntimeAnalytics(null as unknown as (payload: unknown) => void);
113+
expect(() => emitPerformanceMetric("player_load_time", 250)).not.toThrow();
114+
});
115+
116+
it("does not throw when postMessage throws", () => {
117+
postMessage.mockImplementation(() => {
118+
throw new Error("channel closed");
119+
});
120+
expect(() => emitPerformanceMetric("player_scrub_latency", 12)).not.toThrow();
121+
});
122+
123+
it("does not throw when performance.mark throws", () => {
124+
const original = performance.mark;
125+
// Vitest provides a real performance API; replace mark with a thrower for this test.
126+
performance.mark = vi.fn(() => {
127+
throw new Error("mark failed");
128+
}) as typeof performance.mark;
129+
try {
130+
expect(() => emitPerformanceMetric("player_load_time", 100)).not.toThrow();
131+
// Even though performance.mark threw, the bridge should still receive the metric.
132+
expect(postMessage).toHaveBeenCalledWith(
133+
expect.objectContaining({ type: "perf", name: "player_load_time", value: 100 }),
134+
);
135+
} finally {
136+
performance.mark = original;
137+
}
138+
});
139+
140+
it("writes a User Timing mark with detail for DevTools visibility", () => {
141+
if (typeof performance.getEntriesByName !== "function") {
142+
// Older test environments — skip the DevTools assertion but don't fail.
143+
return;
144+
}
145+
emitPerformanceMetric("player_composition_switch", 42, { from: "a", to: "b" });
146+
const entries = performance.getEntriesByName("player_composition_switch", "mark");
147+
expect(entries.length).toBeGreaterThan(0);
148+
const mark = entries[entries.length - 1] as PerformanceMark;
149+
expect(mark.detail).toEqual({ value: 42, tags: { from: "a", to: "b" } });
150+
});
151+
});

packages/core/src/runtime/analytics.ts

Lines changed: 70 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
/**
2-
* Runtime analytics — vendor-agnostic event emission.
2+
* Runtime analytics & performance telemetry — vendor-agnostic event emission.
33
*
44
* The runtime emits structured events via postMessage. The host application
55
* decides what to do with them: forward to PostHog, Mixpanel, Amplitude,
@@ -13,15 +13,18 @@
1313
*
1414
* ```javascript
1515
* window.addEventListener("message", (e) => {
16-
* if (e.data?.source !== "hf-preview" || e.data?.type !== "analytics") return;
17-
* const { event, properties } = e.data;
16+
* if (e.data?.source !== "hf-preview") return;
1817
*
19-
* // PostHog:
20-
* posthog.capture(event, properties);
21-
* // Mixpanel:
22-
* mixpanel.track(event, properties);
23-
* // Custom:
24-
* myLogger.track(event, properties);
18+
* if (e.data.type === "analytics") {
19+
* // discrete lifecycle events: composition_loaded, played, seeked, etc.
20+
* posthog.capture(e.data.event, e.data.properties);
21+
* }
22+
*
23+
* if (e.data.type === "perf") {
24+
* // numeric performance metrics: scrub latency, fps, decoder count, etc.
25+
* // Aggregate per-session (p50/p95) and forward on flush.
26+
* myMetrics.observe(e.data.name, e.data.value, e.data.tags);
27+
* }
2528
* });
2629
* ```
2730
*/
@@ -36,10 +39,22 @@ export type RuntimeAnalyticsEvent =
3639

3740
export type RuntimeAnalyticsProperties = Record<string, string | number | boolean | null>;
3841

42+
/**
43+
* Tags attached to a performance metric — small, low-cardinality identifiers
44+
* (composition id hash, media count bucket, browser version, etc.). Same shape
45+
* as analytics properties so hosts can forward both through one pipeline.
46+
*/
47+
export type RuntimePerformanceTags = Record<string, string | number | boolean | null>;
48+
3949
// Stored reference to the postRuntimeMessage function, set during init.
40-
// Avoids a circular import between analytics ↔ bridge.
50+
// Avoids a circular import between analytics ↔ bridge. Shared by both
51+
// emitAnalyticsEvent and emitPerformanceMetric — one bridge, two channels.
4152
let _postMessage: ((payload: unknown) => void) | null = null;
4253

54+
/**
55+
* Wire the analytics + performance bridge to the runtime's postMessage transport.
56+
* Called once during runtime bootstrap from `init.ts`.
57+
*/
4358
export function initRuntimeAnalytics(postMessage: (payload: unknown) => void): void {
4459
_postMessage = postMessage;
4560
}
@@ -64,3 +79,48 @@ export function emitAnalyticsEvent(
6479
// Never let analytics failures affect the runtime
6580
}
6681
}
82+
83+
/**
84+
* Emit a numeric performance metric through the bridge.
85+
*
86+
* Used for player-perf telemetry — scrub latency, sustained fps, dropped
87+
* frames, decoder count, composition load time, media sync drift. The host
88+
* aggregates per-session values (p50/p95) and forwards to its observability
89+
* pipeline on flush.
90+
*
91+
* Also writes a `performance.mark()` so the metric shows up under the
92+
* DevTools Performance panel's "User Timing" track for local debugging,
93+
* with `value` and `tags` available on the entry's `detail` field.
94+
*
95+
* @param name Metric name, e.g. "player_scrub_latency", "player_playback_fps"
96+
* @param value Numeric value (units are metric-specific: ms for latency, fps for rate, etc.)
97+
* @param tags Optional low-cardinality tags (composition id, media count bucket, etc.)
98+
*/
99+
export function emitPerformanceMetric(
100+
name: string,
101+
value: number,
102+
tags?: RuntimePerformanceTags,
103+
): void {
104+
// Local DevTools breadcrumb. Wrapped because performance.mark() can throw on
105+
// strict CSP, when the document is not yet ready, or when `detail` is non-cloneable.
106+
try {
107+
if (typeof performance !== "undefined" && typeof performance.mark === "function") {
108+
performance.mark(name, { detail: { value, tags: tags ?? {} } });
109+
}
110+
} catch {
111+
// performance API unavailable or rejected — keep going
112+
}
113+
114+
if (!_postMessage) return;
115+
try {
116+
_postMessage({
117+
source: "hf-preview",
118+
type: "perf",
119+
name,
120+
value,
121+
tags: tags ?? {},
122+
});
123+
} catch {
124+
// Never let telemetry failures affect the runtime
125+
}
126+
}

packages/core/src/runtime/types.ts

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -170,6 +170,21 @@ export type RuntimeAnalyticsMessage = {
170170
properties: Record<string, string | number | boolean | null>;
171171
};
172172

173+
/**
174+
* Numeric performance metrics emitted by the runtime — scrub latency, sustained
175+
* fps, dropped frames, decoder count, composition load time, media sync drift.
176+
* The host aggregates per-session values (p50/p95) and forwards to its
177+
* observability pipeline. Distinct from `analytics` events because perf data
178+
* is continuous and numeric, not discrete.
179+
*/
180+
export type RuntimePerformanceMessage = {
181+
source: "hf-preview";
182+
type: "perf";
183+
name: string;
184+
value: number;
185+
tags: Record<string, string | number | boolean | null>;
186+
};
187+
173188
export type RuntimeOutboundMessage =
174189
| RuntimeStateMessage
175190
| RuntimeTimelineMessage
@@ -181,7 +196,8 @@ export type RuntimeOutboundMessage =
181196
| RuntimePickerCancelledMessage
182197
| RuntimeStageSizeMessage
183198
| RuntimeMediaAutoplayBlockedMessage
184-
| RuntimeAnalyticsMessage;
199+
| RuntimeAnalyticsMessage
200+
| RuntimePerformanceMessage;
185201

186202
export type RuntimePlayer = {
187203
_timeline: RuntimeTimelineLike | null;

0 commit comments

Comments
 (0)