Skip to content

Commit 2283a39

Browse files
author
iexitdev
committed
test(release): surface performance story metrics in qa tooling
1 parent 3aaf2eb commit 2283a39

5 files changed

Lines changed: 93 additions & 32 deletions

‎docs/release/native-qa-checklists.md‎

Lines changed: 20 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -158,26 +158,26 @@ Source: [docs/release/native-performance-benchmark.md](native-performance-benchm
158158

159159
### Performance Rows
160160

161-
| Row | Target | Scenario | Data Size | Interaction | Showcase Story | Deep Link | Status | Evidence |
162-
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
163-
| `ios-svg-small-line-initial-render` | iOS / svg | Small line initial render | 100 points | initial render | `v2-perf-line-100` | chartkitshowcase://showcase?view=charts&story=v2-perf-line-100 | pending | None |
164-
| `ios-svg-standard-line-scrub` | iOS / svg | Standard line scrub | 1,000 points | initial render and scrub | `v2-perf-line-1000-scrub` | chartkitshowcase://showcase?view=charts&story=v2-perf-line-1000-scrub | pending | None |
165-
| `ios-svg-dense-line-decimated-overview` | iOS / svg | Dense line decimated overview | 10,000 total points | decimated overview | `v2-perf-line-10000-overview` | chartkitshowcase://showcase?view=charts&story=v2-perf-line-10000-overview | pending | None |
166-
| `ios-svg-multi-line-shared-tooltip-scrub` | iOS / svg | Multi-line shared tooltip scrub | 5 series x 1,000 points | shared tooltip scrub | `v2-perf-line-5x1000-tooltip` | chartkitshowcase://showcase?view=charts&story=v2-perf-line-5x1000-tooltip | pending | None |
167-
| `ios-svg-scrollable-line-one-finger-pan` | iOS / svg | Scrollable line one-finger pan | 10,000 total points, 2,000 shown | one-finger pan | `v2-perf-line-10000-pan` | chartkitshowcase://showcase?view=charts&story=v2-perf-line-10000-pan | pending | None |
168-
| `ios-svg-range-selector-drag-and-thumb-resize` | iOS / svg | Range selector drag and thumb resize | 2 series x 10,000 points | drag and thumb resize | `v2-perf-range-2x10000` | chartkitshowcase://showcase?view=charts&story=v2-perf-range-2x10000 | pending | None |
169-
| `ios-svg-scrollable-bar-horizontal-scroll-and-selection` | iOS / svg | Scrollable bar horizontal scroll and selection | 500 bars | horizontal scroll and selection | `v2-perf-bar-500-selection` | chartkitshowcase://showcase?view=charts&story=v2-perf-bar-500-selection | pending | None |
170-
| `ios-svg-combined-chart-shared-tooltip-and-legend` | iOS / svg | Combined chart shared tooltip and legend | bars plus line | shared tooltip and legend | `v2-perf-combined-tooltip` | chartkitshowcase://showcase?view=charts&story=v2-perf-combined-tooltip | pending | None |
171-
| `ios-svg-candlestick-pan-pinch-and-tap-inspection` | iOS / svg | Candlestick pan, pinch, and tap inspection | 1,000 candles | pan, pinch, tap inspection | `v2-perf-candlestick-1000` | chartkitshowcase://showcase?view=charts&story=v2-perf-candlestick-1000 | pending | None |
172-
| `android-svg-small-line-initial-render` | Android / svg | Small line initial render | 100 points | initial render | `v2-perf-line-100` | chartkitshowcase://showcase?view=charts&story=v2-perf-line-100 | pending | None |
173-
| `android-svg-standard-line-scrub` | Android / svg | Standard line scrub | 1,000 points | initial render and scrub | `v2-perf-line-1000-scrub` | chartkitshowcase://showcase?view=charts&story=v2-perf-line-1000-scrub | pending | None |
174-
| `android-svg-dense-line-decimated-overview` | Android / svg | Dense line decimated overview | 10,000 total points | decimated overview | `v2-perf-line-10000-overview` | chartkitshowcase://showcase?view=charts&story=v2-perf-line-10000-overview | pending | None |
175-
| `android-svg-multi-line-shared-tooltip-scrub` | Android / svg | Multi-line shared tooltip scrub | 5 series x 1,000 points | shared tooltip scrub | `v2-perf-line-5x1000-tooltip` | chartkitshowcase://showcase?view=charts&story=v2-perf-line-5x1000-tooltip | pending | None |
176-
| `android-svg-scrollable-line-one-finger-pan` | Android / svg | Scrollable line one-finger pan | 10,000 total points, 2,000 shown | one-finger pan | `v2-perf-line-10000-pan` | chartkitshowcase://showcase?view=charts&story=v2-perf-line-10000-pan | pending | None |
177-
| `android-svg-range-selector-drag-and-thumb-resize` | Android / svg | Range selector drag and thumb resize | 2 series x 10,000 points | drag and thumb resize | `v2-perf-range-2x10000` | chartkitshowcase://showcase?view=charts&story=v2-perf-range-2x10000 | pending | None |
178-
| `android-svg-scrollable-bar-horizontal-scroll-and-selection` | Android / svg | Scrollable bar horizontal scroll and selection | 500 bars | horizontal scroll and selection | `v2-perf-bar-500-selection` | chartkitshowcase://showcase?view=charts&story=v2-perf-bar-500-selection | pending | None |
179-
| `android-svg-combined-chart-shared-tooltip-and-legend` | Android / svg | Combined chart shared tooltip and legend | bars plus line | shared tooltip and legend | `v2-perf-combined-tooltip` | chartkitshowcase://showcase?view=charts&story=v2-perf-combined-tooltip | pending | None |
180-
| `android-svg-candlestick-pan-pinch-and-tap-inspection` | Android / svg | Candlestick pan, pinch, and tap inspection | 1,000 candles | pan, pinch, tap inspection | `v2-perf-candlestick-1000` | chartkitshowcase://showcase?view=charts&story=v2-perf-candlestick-1000 | pending | None |
161+
| Row | Target | Scenario | Data Size | Expected Story Metrics | Interaction | Showcase Story | Deep Link | Status | Evidence |
162+
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
163+
| `ios-svg-small-line-initial-render` | iOS / svg | Small line initial render | 100 points | chart line; 100 total; 100 visible; 1 series | initial render | `v2-perf-line-100` | chartkitshowcase://showcase?view=charts&story=v2-perf-line-100 | pending | None |
164+
| `ios-svg-standard-line-scrub` | iOS / svg | Standard line scrub | 1,000 points | chart line; 1,000 total; 1,000 visible; 1 series | initial render and scrub | `v2-perf-line-1000-scrub` | chartkitshowcase://showcase?view=charts&story=v2-perf-line-1000-scrub | pending | None |
165+
| `ios-svg-dense-line-decimated-overview` | iOS / svg | Dense line decimated overview | 10,000 total points | chart line; 10,000 total; 10,000 visible; 1 series | decimated overview | `v2-perf-line-10000-overview` | chartkitshowcase://showcase?view=charts&story=v2-perf-line-10000-overview | pending | None |
166+
| `ios-svg-multi-line-shared-tooltip-scrub` | iOS / svg | Multi-line shared tooltip scrub | 5 series x 1,000 points | chart line; 1,000 total; 1,000 visible; 5 series | shared tooltip scrub | `v2-perf-line-5x1000-tooltip` | chartkitshowcase://showcase?view=charts&story=v2-perf-line-5x1000-tooltip | pending | None |
167+
| `ios-svg-scrollable-line-one-finger-pan` | iOS / svg | Scrollable line one-finger pan | 10,000 total points, 2,000 shown | chart line; 10,000 total; 2,000 visible; 1 series | one-finger pan | `v2-perf-line-10000-pan` | chartkitshowcase://showcase?view=charts&story=v2-perf-line-10000-pan | pending | None |
168+
| `ios-svg-range-selector-drag-and-thumb-resize` | iOS / svg | Range selector drag and thumb resize | 2 series x 10,000 points | chart line; 10,000 total; 1,500 visible; 2 series | drag and thumb resize | `v2-perf-range-2x10000` | chartkitshowcase://showcase?view=charts&story=v2-perf-range-2x10000 | pending | None |
169+
| `ios-svg-scrollable-bar-horizontal-scroll-and-selection` | iOS / svg | Scrollable bar horizontal scroll and selection | 500 bars | chart bar; 500 total; 24 visible; 1 series | horizontal scroll and selection | `v2-perf-bar-500-selection` | chartkitshowcase://showcase?view=charts&story=v2-perf-bar-500-selection | pending | None |
170+
| `ios-svg-combined-chart-shared-tooltip-and-legend` | iOS / svg | Combined chart shared tooltip and legend | bars plus line | chart combined; 36 total; 36 visible; 2 series | shared tooltip and legend | `v2-perf-combined-tooltip` | chartkitshowcase://showcase?view=charts&story=v2-perf-combined-tooltip | pending | None |
171+
| `ios-svg-candlestick-pan-pinch-and-tap-inspection` | iOS / svg | Candlestick pan, pinch, and tap inspection | 1,000 candles | chart candlestick; 1,000 total; 80 visible; 1 series | pan, pinch, tap inspection | `v2-perf-candlestick-1000` | chartkitshowcase://showcase?view=charts&story=v2-perf-candlestick-1000 | pending | None |
172+
| `android-svg-small-line-initial-render` | Android / svg | Small line initial render | 100 points | chart line; 100 total; 100 visible; 1 series | initial render | `v2-perf-line-100` | chartkitshowcase://showcase?view=charts&story=v2-perf-line-100 | pending | None |
173+
| `android-svg-standard-line-scrub` | Android / svg | Standard line scrub | 1,000 points | chart line; 1,000 total; 1,000 visible; 1 series | initial render and scrub | `v2-perf-line-1000-scrub` | chartkitshowcase://showcase?view=charts&story=v2-perf-line-1000-scrub | pending | None |
174+
| `android-svg-dense-line-decimated-overview` | Android / svg | Dense line decimated overview | 10,000 total points | chart line; 10,000 total; 10,000 visible; 1 series | decimated overview | `v2-perf-line-10000-overview` | chartkitshowcase://showcase?view=charts&story=v2-perf-line-10000-overview | pending | None |
175+
| `android-svg-multi-line-shared-tooltip-scrub` | Android / svg | Multi-line shared tooltip scrub | 5 series x 1,000 points | chart line; 1,000 total; 1,000 visible; 5 series | shared tooltip scrub | `v2-perf-line-5x1000-tooltip` | chartkitshowcase://showcase?view=charts&story=v2-perf-line-5x1000-tooltip | pending | None |
176+
| `android-svg-scrollable-line-one-finger-pan` | Android / svg | Scrollable line one-finger pan | 10,000 total points, 2,000 shown | chart line; 10,000 total; 2,000 visible; 1 series | one-finger pan | `v2-perf-line-10000-pan` | chartkitshowcase://showcase?view=charts&story=v2-perf-line-10000-pan | pending | None |
177+
| `android-svg-range-selector-drag-and-thumb-resize` | Android / svg | Range selector drag and thumb resize | 2 series x 10,000 points | chart line; 10,000 total; 1,500 visible; 2 series | drag and thumb resize | `v2-perf-range-2x10000` | chartkitshowcase://showcase?view=charts&story=v2-perf-range-2x10000 | pending | None |
178+
| `android-svg-scrollable-bar-horizontal-scroll-and-selection` | Android / svg | Scrollable bar horizontal scroll and selection | 500 bars | chart bar; 500 total; 24 visible; 1 series | horizontal scroll and selection | `v2-perf-bar-500-selection` | chartkitshowcase://showcase?view=charts&story=v2-perf-bar-500-selection | pending | None |
179+
| `android-svg-combined-chart-shared-tooltip-and-legend` | Android / svg | Combined chart shared tooltip and legend | bars plus line | chart combined; 36 total; 36 visible; 2 series | shared tooltip and legend | `v2-perf-combined-tooltip` | chartkitshowcase://showcase?view=charts&story=v2-perf-combined-tooltip | pending | None |
180+
| `android-svg-candlestick-pan-pinch-and-tap-inspection` | Android / svg | Candlestick pan, pinch, and tap inspection | 1,000 candles | chart candlestick; 1,000 total; 80 visible; 1 series | pan, pinch, tap inspection | `v2-perf-candlestick-1000` | chartkitshowcase://showcase?view=charts&story=v2-perf-candlestick-1000 | pending | None |
181181

182182
### Deferred Rows
183183

‎scripts/generate-native-qa-checklists.mjs‎

Lines changed: 21 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,24 @@ const formatEvidence = (evidence = []) =>
3636
? evidence.map((item) => `\`${item}\``).join(", ")
3737
: "None";
3838

39+
const formatExpectedStoryMetrics = (metrics) => {
40+
if (!metrics) {
41+
return "";
42+
}
43+
44+
return [
45+
metrics.chartType && `chart ${metrics.chartType}`,
46+
Number.isFinite(metrics.totalPoints) &&
47+
`${metrics.totalPoints.toLocaleString("en-US")} total`,
48+
Number.isFinite(metrics.visiblePoints) &&
49+
`${metrics.visiblePoints.toLocaleString("en-US")} visible`,
50+
Number.isFinite(metrics.seriesCount) &&
51+
`${metrics.seriesCount.toLocaleString("en-US")} series`
52+
]
53+
.filter(Boolean)
54+
.join("; ");
55+
};
56+
3957
const countRowsByStatus = (rows = []) => {
4058
const counts = Object.fromEntries(statusOrder.map((status) => [status, 0]));
4159

@@ -161,8 +179,8 @@ const formatPerformanceRows = (matrix) => {
161179
const platforms = toIdMap(matrix.platforms);
162180
const scenarios = toIdMap(matrix.scenarios);
163181
const lines = [
164-
"| Row | Target | Scenario | Data Size | Interaction | Showcase Story | Deep Link | Status | Evidence |",
165-
"| --- | --- | --- | --- | --- | --- | --- | --- | --- |"
182+
"| Row | Target | Scenario | Data Size | Expected Story Metrics | Interaction | Showcase Story | Deep Link | Status | Evidence |",
183+
"| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |"
166184
];
167185

168186
for (const row of matrix.rows) {
@@ -174,6 +192,7 @@ const formatPerformanceRows = (matrix) => {
174192
`${platform?.label ?? row.platform} / ${row.renderer}`,
175193
scenario?.label ?? row.scenarioId,
176194
scenario?.requiredDataSize ?? "",
195+
formatExpectedStoryMetrics(scenario?.expectedStoryMetrics),
177196
scenario?.interaction ?? "",
178197
scenario?.showcaseStoryId ? `\`${scenario.showcaseStoryId}\`` : "",
179198
getStoryLaunchUrl(scenario?.showcaseStoryId),

‎scripts/generate-native-qa-checklists.test.mjs‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,8 @@ describe("native QA checklist generator", () => {
2626
"chartkitshowcase://showcase?view=charts&page=compat"
2727
);
2828
expect(markdown).toContain("`v2-perf-bar-500-selection`");
29+
expect(markdown).toContain("Expected Story Metrics");
30+
expect(markdown).toContain("bar; 500 total; 24 visible; 1 series");
2931
expect(markdown).toContain(
3032
"chartkitshowcase://showcase?view=charts&story=v2-perf-bar-500-selection"
3133
);

‎scripts/record-native-qa-evidence.mjs‎

Lines changed: 41 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -238,12 +238,30 @@ const getRowLaunchTarget = (matrix, row) => {
238238
};
239239
};
240240

241-
const getRowRequiredCheckGroups = (matrix, row) => {
242-
const page = matrix.pages?.find((item) => item.id === row.pageId);
241+
const getRowRequiredCheckGroups = (matrix, row) =>
242+
matrix.pages?.find((item) => item.id === row.pageId)?.requiredCheckGroups ??
243+
[];
243244

244-
return Array.isArray(page?.requiredCheckGroups)
245-
? page.requiredCheckGroups
246-
: [];
245+
const getRowExpectedStoryMetrics = (matrix, row) =>
246+
matrix.scenarios?.find((item) => item.id === row.scenarioId)
247+
?.expectedStoryMetrics;
248+
249+
const formatExpectedStoryMetrics = (metrics) => {
250+
if (!metrics) {
251+
return "";
252+
}
253+
254+
return [
255+
metrics.chartType && `chart ${metrics.chartType}`,
256+
Number.isFinite(metrics.totalPoints) &&
257+
`${metrics.totalPoints.toLocaleString("en-US")} total`,
258+
Number.isFinite(metrics.visiblePoints) &&
259+
`${metrics.visiblePoints.toLocaleString("en-US")} visible`,
260+
Number.isFinite(metrics.seriesCount) &&
261+
`${metrics.seriesCount.toLocaleString("en-US")} series`
262+
]
263+
.filter(Boolean)
264+
.join("; ");
247265
};
248266

249267
const getRowRequiredChecks = (matrix, row) => {
@@ -258,6 +276,11 @@ const getRowRequiredChecks = (matrix, row) => {
258276
scenario?.requiredDataSize
259277
? `scenario: data size ${scenario.requiredDataSize}`
260278
: "",
279+
scenario?.expectedStoryMetrics
280+
? `scenario: expected story metrics ${formatExpectedStoryMetrics(
281+
scenario.expectedStoryMetrics
282+
)}`
283+
: "",
261284
scenario?.interaction
262285
? `scenario: interaction ${scenario.interaction}`
263286
: "",
@@ -283,6 +306,9 @@ export const listNativeQaRows = async ({
283306
return matrix.rows.map((row) => ({
284307
checks: includeDetails ? getRowRequiredChecks(matrix, row) : [],
285308
evidence: row.evidence ?? [],
309+
expectedStoryMetrics: includeDetails
310+
? getRowExpectedStoryMetrics(matrix, row)
311+
: undefined,
286312
id: row.id,
287313
...getRowLaunchTarget(matrix, row),
288314
requiredCheckGroups: includeDetails
@@ -353,11 +379,8 @@ export const recordNativeQaEvidence = async ({
353379
status
354380
};
355381

356-
if (notes) {
357-
nextRow.notes = notes;
358-
} else {
359-
delete nextRow.notes;
360-
}
382+
if (notes) nextRow.notes = notes;
383+
else delete nextRow.notes;
361384

362385
const nextRows = [...matrix.rows];
363386
nextRows[rowIndex] = nextRow;
@@ -429,6 +452,14 @@ const main = async () => {
429452
console.log(` showcase story: ${row.showcaseStoryId}`);
430453
}
431454

455+
if (row.expectedStoryMetrics) {
456+
console.log(
457+
` expected story metrics: ${formatExpectedStoryMetrics(
458+
row.expectedStoryMetrics
459+
)}`
460+
);
461+
}
462+
432463
for (const check of row.checks) {
433464
console.log(` - ${check}`);
434465
}

‎scripts/record-native-qa-evidence.test.mjs‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -93,10 +93,19 @@ describe("native QA evidence recorder", () => {
9393
);
9494
expect(performanceRow.checks).toContain("metric: initial render time");
9595
expect(performanceRow).toMatchObject({
96+
expectedStoryMetrics: {
97+
chartType: "line",
98+
seriesCount: 1,
99+
totalPoints: 100,
100+
visiblePoints: 100
101+
},
96102
launchUrl:
97103
"chartkitshowcase://showcase?view=charts&story=v2-perf-line-100",
98104
showcaseStoryId: "v2-perf-line-100"
99105
});
106+
expect(performanceRow.checks).toContain(
107+
"scenario: expected story metrics chart line; 100 total; 100 visible; 1 series"
108+
);
100109
expect(skiaRow.checks).toContain(
101110
"scenario: evidence Install optional Skia renderer dependencies, run native release build, and verify the SVG default path still works without static Skia imports."
102111
);

0 commit comments

Comments
 (0)