E2E Coverage Matrix #450
e2e-matrix.yml
on: workflow_dispatch
Matrix: Preset matrix (smoke, self-hosted macOS)
Specialised + canvas smoke
1m 58s
Mini-train matrix (macOS, nightly)
18m 39s
Matrix report
2m 35s
Annotations
19 errors, 4 warnings, and 1 notice
|
Specialised + canvas smoke
Process completed with exit code 1.
|
|
[chromium] › e2e/scenarios/01_canvas_smoke.spec.ts:19:5 › canvas smoke (proof of life) › preset gemma4 runs smoke pipeline through GUI:
vbgui/e2e/fixtures.ts#L76
3) [chromium] › e2e/scenarios/01_canvas_smoke.spec.ts:19:5 › canvas smoke (proof of life) › preset gemma4 runs smoke pipeline through GUI
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(locator).toContainText(expected) failed
Locator: getByTestId('run-result-modal').getByTestId('run-result-overall')
Expected substring: "ok"
Received string: "· fail · 0.1 ms"
Timeout: 5000ms
Call log:
- Expect "toContainText" with timeout 5000ms
- waiting for getByTestId('run-result-modal').getByTestId('run-result-overall')
14 × locator resolved to <span data-testid="run-result-overall">· fail · 0.1 ms</span>
- unexpected value "· fail · 0.1 ms"
at ../fixtures.ts:76
74 | ): Promise<void> {
75 | const overall = modal.getByTestId("run-result-overall");
> 76 | await expect(overall).toContainText(expected);
| ^
77 | }
78 |
79 | export async function closeModal(page: Page): Promise<void> {
at assertOverallStatus (/Users/dave/actions-runner-cppmega-mlx/_work/cppmega_mlx/cppmega_mlx/vbgui/e2e/fixtures.ts:76:25)
at /Users/dave/actions-runner-cppmega-mlx/_work/cppmega_mlx/cppmega_mlx/vbgui/e2e/scenarios/01_canvas_smoke.spec.ts:29:15
|
|
[chromium] › e2e/scenarios/01_canvas_smoke.spec.ts:19:5 › canvas smoke (proof of life) › preset kimi_linear runs smoke pipeline through GUI:
vbgui/e2e/fixtures.ts#L76
2) [chromium] › e2e/scenarios/01_canvas_smoke.spec.ts:19:5 › canvas smoke (proof of life) › preset kimi_linear runs smoke pipeline through GUI
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(locator).toContainText(expected) failed
Locator: getByTestId('run-result-modal').getByTestId('run-result-overall')
Expected substring: "ok"
Received string: "· fail · 0.1 ms"
Timeout: 5000ms
Call log:
- Expect "toContainText" with timeout 5000ms
- waiting for getByTestId('run-result-modal').getByTestId('run-result-overall')
14 × locator resolved to <span data-testid="run-result-overall">· fail · 0.1 ms</span>
- unexpected value "· fail · 0.1 ms"
at ../fixtures.ts:76
74 | ): Promise<void> {
75 | const overall = modal.getByTestId("run-result-overall");
> 76 | await expect(overall).toContainText(expected);
| ^
77 | }
78 |
79 | export async function closeModal(page: Page): Promise<void> {
at assertOverallStatus (/Users/dave/actions-runner-cppmega-mlx/_work/cppmega_mlx/cppmega_mlx/vbgui/e2e/fixtures.ts:76:25)
at /Users/dave/actions-runner-cppmega-mlx/_work/cppmega_mlx/cppmega_mlx/vbgui/e2e/scenarios/01_canvas_smoke.spec.ts:29:15
|
|
[chromium] › e2e/scenarios/01_canvas_smoke.spec.ts:19:5 › canvas smoke (proof of life) › preset qwen3_next runs smoke pipeline through GUI:
vbgui/e2e/fixtures.ts#L76
1) [chromium] › e2e/scenarios/01_canvas_smoke.spec.ts:19:5 › canvas smoke (proof of life) › preset qwen3_next runs smoke pipeline through GUI
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(locator).toContainText(expected) failed
Locator: getByTestId('run-result-modal').getByTestId('run-result-overall')
Expected substring: "ok"
Received string: "· fail · 0.1 ms"
Timeout: 5000ms
Call log:
- Expect "toContainText" with timeout 5000ms
- waiting for getByTestId('run-result-modal').getByTestId('run-result-overall')
14 × locator resolved to <span data-testid="run-result-overall">· fail · 0.1 ms</span>
- unexpected value "· fail · 0.1 ms"
at ../fixtures.ts:76
74 | ): Promise<void> {
75 | const overall = modal.getByTestId("run-result-overall");
> 76 | await expect(overall).toContainText(expected);
| ^
77 | }
78 |
79 | export async function closeModal(page: Page): Promise<void> {
at assertOverallStatus (/Users/dave/actions-runner-cppmega-mlx/_work/cppmega_mlx/cppmega_mlx/vbgui/e2e/fixtures.ts:76:25)
at /Users/dave/actions-runner-cppmega-mlx/_work/cppmega_mlx/cppmega_mlx/vbgui/e2e/scenarios/01_canvas_smoke.spec.ts:29:15
|
|
Preset matrix (smoke, self-hosted macOS) (1)
The run was canceled by @apstenku123.
|
|
Preset matrix (smoke, self-hosted macOS) (2)
The run was canceled by @apstenku123.
|
|
Preset matrix (smoke, self-hosted macOS) (3)
The run was canceled by @apstenku123.
|
|
Mini-train matrix (macOS, nightly)
The run was canceled by @apstenku123.
|
|
[chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T3_minimal_no_fim__P2_doc:
vbgui/e2e/scenarios/03_train_matrix.spec.ts#L66
10) [chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T3_minimal_no_fim__P2_doc
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(locator).toContainText(expected) failed
Locator: getByTestId('run-result-modal').getByTestId('run-result-stage-train')
Expected substring: "fail"
Received string: "·trainskipped0.0"
Timeout: 5000ms
Call log:
- Expect "toContainText" with timeout 5000ms
- waiting for getByTestId('run-result-modal').getByTestId('run-result-stage-train')
14 × locator resolved to <tr data-testid="run-result-stage-train">…</tr>
- unexpected value "·trainskipped0.0"
64 | // Train must FAIL with a typed error (vjp / Metal path). Earlier
65 | // stages still must be green to prove the GUI walk works.
> 66 | await expect(trainRow).toContainText("fail");
| ^
67 | await snapshot(page, "03_train_matrix/xfail", key);
68 | } else {
69 | await assertOverallStatus(modal, "ok");
at /Users/dave/actions-runner-cppmega-mlx/_work/cppmega_mlx/cppmega_mlx/vbgui/e2e/scenarios/03_train_matrix.spec.ts:66:32
|
|
[chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T3_minimal_no_fim__P1_minimal:
vbgui/e2e/scenarios/03_train_matrix.spec.ts#L66
9) [chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T3_minimal_no_fim__P1_minimal
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(locator).toContainText(expected) failed
Locator: getByTestId('run-result-modal').getByTestId('run-result-stage-train')
Expected substring: "fail"
Received string: "·trainskipped0.0"
Timeout: 5000ms
Call log:
- Expect "toContainText" with timeout 5000ms
- waiting for getByTestId('run-result-modal').getByTestId('run-result-stage-train')
14 × locator resolved to <tr data-testid="run-result-stage-train">…</tr>
- unexpected value "·trainskipped0.0"
64 | // Train must FAIL with a typed error (vjp / Metal path). Earlier
65 | // stages still must be green to prove the GUI walk works.
> 66 | await expect(trainRow).toContainText("fail");
| ^
67 | await snapshot(page, "03_train_matrix/xfail", key);
68 | } else {
69 | await assertOverallStatus(modal, "ok");
at /Users/dave/actions-runner-cppmega-mlx/_work/cppmega_mlx/cppmega_mlx/vbgui/e2e/scenarios/03_train_matrix.spec.ts:66:32
|
|
[chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T2_gpt2_small__P4_full:
vbgui/e2e/scenarios/03_train_matrix.spec.ts#L66
8) [chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T2_gpt2_small__P4_full
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(locator).toContainText(expected) failed
Locator: getByTestId('run-result-modal').getByTestId('run-result-stage-train')
Expected substring: "fail"
Received string: "·trainskipped0.0"
Timeout: 5000ms
Call log:
- Expect "toContainText" with timeout 5000ms
- waiting for getByTestId('run-result-modal').getByTestId('run-result-stage-train')
14 × locator resolved to <tr data-testid="run-result-stage-train">…</tr>
- unexpected value "·trainskipped0.0"
64 | // Train must FAIL with a typed error (vjp / Metal path). Earlier
65 | // stages still must be green to prove the GUI walk works.
> 66 | await expect(trainRow).toContainText("fail");
| ^
67 | await snapshot(page, "03_train_matrix/xfail", key);
68 | } else {
69 | await assertOverallStatus(modal, "ok");
at /Users/dave/actions-runner-cppmega-mlx/_work/cppmega_mlx/cppmega_mlx/vbgui/e2e/scenarios/03_train_matrix.spec.ts:66:32
|
|
[chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T2_gpt2_small__P3_engram:
vbgui/e2e/scenarios/03_train_matrix.spec.ts#L66
7) [chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T2_gpt2_small__P3_engram
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(locator).toContainText(expected) failed
Locator: getByTestId('run-result-modal').getByTestId('run-result-stage-train')
Expected substring: "fail"
Received string: "·trainskipped0.0"
Timeout: 5000ms
Call log:
- Expect "toContainText" with timeout 5000ms
- waiting for getByTestId('run-result-modal').getByTestId('run-result-stage-train')
14 × locator resolved to <tr data-testid="run-result-stage-train">…</tr>
- unexpected value "·trainskipped0.0"
64 | // Train must FAIL with a typed error (vjp / Metal path). Earlier
65 | // stages still must be green to prove the GUI walk works.
> 66 | await expect(trainRow).toContainText("fail");
| ^
67 | await snapshot(page, "03_train_matrix/xfail", key);
68 | } else {
69 | await assertOverallStatus(modal, "ok");
at /Users/dave/actions-runner-cppmega-mlx/_work/cppmega_mlx/cppmega_mlx/vbgui/e2e/scenarios/03_train_matrix.spec.ts:66:32
|
|
[chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T2_gpt2_small__P2_doc:
vbgui/e2e/scenarios/03_train_matrix.spec.ts#L66
6) [chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T2_gpt2_small__P2_doc
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(locator).toContainText(expected) failed
Locator: getByTestId('run-result-modal').getByTestId('run-result-stage-train')
Expected substring: "fail"
Received string: "·trainskipped0.0"
Timeout: 5000ms
Call log:
- Expect "toContainText" with timeout 5000ms
- waiting for getByTestId('run-result-modal').getByTestId('run-result-stage-train')
14 × locator resolved to <tr data-testid="run-result-stage-train">…</tr>
- unexpected value "·trainskipped0.0"
64 | // Train must FAIL with a typed error (vjp / Metal path). Earlier
65 | // stages still must be green to prove the GUI walk works.
> 66 | await expect(trainRow).toContainText("fail");
| ^
67 | await snapshot(page, "03_train_matrix/xfail", key);
68 | } else {
69 | await assertOverallStatus(modal, "ok");
at /Users/dave/actions-runner-cppmega-mlx/_work/cppmega_mlx/cppmega_mlx/vbgui/e2e/scenarios/03_train_matrix.spec.ts:66:32
|
|
[chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T2_gpt2_small__P1_minimal:
vbgui/e2e/scenarios/03_train_matrix.spec.ts#L66
5) [chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T2_gpt2_small__P1_minimal
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(locator).toContainText(expected) failed
Locator: getByTestId('run-result-modal').getByTestId('run-result-stage-train')
Expected substring: "fail"
Received string: "·trainskipped0.0"
Timeout: 5000ms
Call log:
- Expect "toContainText" with timeout 5000ms
- waiting for getByTestId('run-result-modal').getByTestId('run-result-stage-train')
14 × locator resolved to <tr data-testid="run-result-stage-train">…</tr>
- unexpected value "·trainskipped0.0"
64 | // Train must FAIL with a typed error (vjp / Metal path). Earlier
65 | // stages still must be green to prove the GUI walk works.
> 66 | await expect(trainRow).toContainText("fail");
| ^
67 | await snapshot(page, "03_train_matrix/xfail", key);
68 | } else {
69 | await assertOverallStatus(modal, "ok");
at /Users/dave/actions-runner-cppmega-mlx/_work/cppmega_mlx/cppmega_mlx/vbgui/e2e/scenarios/03_train_matrix.spec.ts:66:32
|
|
[chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T1_cppmega_v3__P4_full:
vbgui/e2e/scenarios/03_train_matrix.spec.ts#L66
4) [chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T1_cppmega_v3__P4_full
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(locator).toContainText(expected) failed
Locator: getByTestId('run-result-modal').getByTestId('run-result-stage-train')
Expected substring: "fail"
Received string: "·trainskipped0.0"
Timeout: 5000ms
Call log:
- Expect "toContainText" with timeout 5000ms
- waiting for getByTestId('run-result-modal').getByTestId('run-result-stage-train')
14 × locator resolved to <tr data-testid="run-result-stage-train">…</tr>
- unexpected value "·trainskipped0.0"
64 | // Train must FAIL with a typed error (vjp / Metal path). Earlier
65 | // stages still must be green to prove the GUI walk works.
> 66 | await expect(trainRow).toContainText("fail");
| ^
67 | await snapshot(page, "03_train_matrix/xfail", key);
68 | } else {
69 | await assertOverallStatus(modal, "ok");
at /Users/dave/actions-runner-cppmega-mlx/_work/cppmega_mlx/cppmega_mlx/vbgui/e2e/scenarios/03_train_matrix.spec.ts:66:32
|
|
[chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T1_cppmega_v3__P3_engram:
vbgui/e2e/scenarios/03_train_matrix.spec.ts#L66
3) [chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T1_cppmega_v3__P3_engram
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(locator).toContainText(expected) failed
Locator: getByTestId('run-result-modal').getByTestId('run-result-stage-train')
Expected substring: "fail"
Received string: "·trainskipped0.0"
Timeout: 5000ms
Call log:
- Expect "toContainText" with timeout 5000ms
- waiting for getByTestId('run-result-modal').getByTestId('run-result-stage-train')
14 × locator resolved to <tr data-testid="run-result-stage-train">…</tr>
- unexpected value "·trainskipped0.0"
64 | // Train must FAIL with a typed error (vjp / Metal path). Earlier
65 | // stages still must be green to prove the GUI walk works.
> 66 | await expect(trainRow).toContainText("fail");
| ^
67 | await snapshot(page, "03_train_matrix/xfail", key);
68 | } else {
69 | await assertOverallStatus(modal, "ok");
at /Users/dave/actions-runner-cppmega-mlx/_work/cppmega_mlx/cppmega_mlx/vbgui/e2e/scenarios/03_train_matrix.spec.ts:66:32
|
|
[chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T1_cppmega_v3__P2_doc:
vbgui/e2e/scenarios/03_train_matrix.spec.ts#L66
2) [chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T1_cppmega_v3__P2_doc
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(locator).toContainText(expected) failed
Locator: getByTestId('run-result-modal').getByTestId('run-result-stage-train')
Expected substring: "fail"
Received string: "·trainskipped0.0"
Timeout: 5000ms
Call log:
- Expect "toContainText" with timeout 5000ms
- waiting for getByTestId('run-result-modal').getByTestId('run-result-stage-train')
14 × locator resolved to <tr data-testid="run-result-stage-train">…</tr>
- unexpected value "·trainskipped0.0"
64 | // Train must FAIL with a typed error (vjp / Metal path). Earlier
65 | // stages still must be green to prove the GUI walk works.
> 66 | await expect(trainRow).toContainText("fail");
| ^
67 | await snapshot(page, "03_train_matrix/xfail", key);
68 | } else {
69 | await assertOverallStatus(modal, "ok");
at /Users/dave/actions-runner-cppmega-mlx/_work/cppmega_mlx/cppmega_mlx/vbgui/e2e/scenarios/03_train_matrix.spec.ts:66:32
|
|
[chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T1_cppmega_v3__P1_minimal:
vbgui/e2e/scenarios/03_train_matrix.spec.ts#L66
1) [chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T1_cppmega_v3__P1_minimal
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(locator).toContainText(expected) failed
Locator: getByTestId('run-result-modal').getByTestId('run-result-stage-train')
Expected substring: "fail"
Received string: "·trainskipped0.0"
Timeout: 5000ms
Call log:
- Expect "toContainText" with timeout 5000ms
- waiting for getByTestId('run-result-modal').getByTestId('run-result-stage-train')
14 × locator resolved to <tr data-testid="run-result-stage-train">…</tr>
- unexpected value "·trainskipped0.0"
64 | // Train must FAIL with a typed error (vjp / Metal path). Earlier
65 | // stages still must be green to prove the GUI walk works.
> 66 | await expect(trainRow).toContainText("fail");
| ^
67 | await snapshot(page, "03_train_matrix/xfail", key);
68 | } else {
69 | await assertOverallStatus(modal, "ok");
at /Users/dave/actions-runner-cppmega-mlx/_work/cppmega_mlx/cppmega_mlx/vbgui/e2e/scenarios/03_train_matrix.spec.ts:66:32
|
|
E2E Coverage Matrix
The run was canceled by @apstenku123.
|
|
Preset matrix (smoke, self-hosted macOS) (4)
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5, actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020, actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
|
|
Specialised + canvas smoke
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5, actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020, actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
|
|
Mini-train matrix (macOS, nightly)
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5, actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020, actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
|
|
Matrix report
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5, actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093, actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065, actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
|
|
🎭 Playwright Run Summary
3 failed
[chromium] › e2e/scenarios/01_canvas_smoke.spec.ts:19:5 › canvas smoke (proof of life) › preset qwen3_next runs smoke pipeline through GUI
[chromium] › e2e/scenarios/01_canvas_smoke.spec.ts:19:5 › canvas smoke (proof of life) › preset kimi_linear runs smoke pipeline through GUI
[chromium] › e2e/scenarios/01_canvas_smoke.spec.ts:19:5 › canvas smoke (proof of life) › preset gemma4 runs smoke pipeline through GUI
28 passed (47.2s)
|
Artifacts
Produced during runtime
| Name | Size | Digest | |
|---|---|---|---|
|
e2e_matrix_report
|
1008 Bytes |
sha256:d8347c6db7234d8abd684da14d714ceb7b370839c8718bacc7856ecb52a4fb1b
|
|
|
mini-train-results
Expired
|
228 MB |
sha256:a48dca951352056dcdb36858d858f722f628534f6140604473946022ca3f5352
|
|
|
preset-matrix-shard-4
Expired
|
933 Bytes |
sha256:9e0447e0996f73e2fb7386b39348db12cd83f739588351bdf4debf06d9906315
|
|
|
specialised-results
Expired
|
12.6 MB |
sha256:d098903ba518d0c60f861005932e3f3c8835c2c720d2774b519f929134ff128b
|
|