Skip to content

E2E Coverage Matrix #449

E2E Coverage Matrix

E2E Coverage Matrix #449

Manually triggered July 13, 2026 22:56
Status Cancelled
Total duration 12m 57s
Artifacts 5

e2e-matrix.yml

on: workflow_dispatch
Matrix: Preset matrix (smoke, self-hosted macOS)
Specialised + canvas smoke
2m 5s
Specialised + canvas smoke
Mini-train matrix (macOS, nightly)
5m 5s
Mini-train matrix (macOS, nightly)
Matrix report
20s
Matrix report
Fit to window
Zoom out
Zoom in

Annotations

18 errors, 5 warnings, and 1 notice
Specialised + canvas smoke
Process completed with exit code 1.
[chromium] › e2e/scenarios/01_canvas_smoke.spec.ts:19:5 › canvas smoke (proof of life) › preset gemma4 runs smoke pipeline through GUI: vbgui/e2e/fixtures.ts#L76
3) [chromium] › e2e/scenarios/01_canvas_smoke.spec.ts:19:5 › canvas smoke (proof of life) › preset gemma4 runs smoke pipeline through GUI Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: expect(locator).toContainText(expected) failed Locator: getByTestId('run-result-modal').getByTestId('run-result-overall') Expected substring: "ok" Received string: "· fail · 0.1 ms" Timeout: 5000ms Call log: - Expect "toContainText" with timeout 5000ms - waiting for getByTestId('run-result-modal').getByTestId('run-result-overall') 14 × locator resolved to <span data-testid="run-result-overall">· fail · 0.1 ms</span> - unexpected value "· fail · 0.1 ms" at ../fixtures.ts:76 74 | ): Promise<void> { 75 | const overall = modal.getByTestId("run-result-overall"); > 76 | await expect(overall).toContainText(expected); | ^ 77 | } 78 | 79 | export async function closeModal(page: Page): Promise<void> { at assertOverallStatus (/Users/dave/actions-runner-cppmega-mlx/_work/cppmega_mlx/cppmega_mlx/vbgui/e2e/fixtures.ts:76:25) at /Users/dave/actions-runner-cppmega-mlx/_work/cppmega_mlx/cppmega_mlx/vbgui/e2e/scenarios/01_canvas_smoke.spec.ts:29:15
[chromium] › e2e/scenarios/01_canvas_smoke.spec.ts:19:5 › canvas smoke (proof of life) › preset kimi_linear runs smoke pipeline through GUI: vbgui/e2e/fixtures.ts#L76
2) [chromium] › e2e/scenarios/01_canvas_smoke.spec.ts:19:5 › canvas smoke (proof of life) › preset kimi_linear runs smoke pipeline through GUI Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: expect(locator).toContainText(expected) failed Locator: getByTestId('run-result-modal').getByTestId('run-result-overall') Expected substring: "ok" Received string: "· fail · 0.1 ms" Timeout: 5000ms Call log: - Expect "toContainText" with timeout 5000ms - waiting for getByTestId('run-result-modal').getByTestId('run-result-overall') 14 × locator resolved to <span data-testid="run-result-overall">· fail · 0.1 ms</span> - unexpected value "· fail · 0.1 ms" at ../fixtures.ts:76 74 | ): Promise<void> { 75 | const overall = modal.getByTestId("run-result-overall"); > 76 | await expect(overall).toContainText(expected); | ^ 77 | } 78 | 79 | export async function closeModal(page: Page): Promise<void> { at assertOverallStatus (/Users/dave/actions-runner-cppmega-mlx/_work/cppmega_mlx/cppmega_mlx/vbgui/e2e/fixtures.ts:76:25) at /Users/dave/actions-runner-cppmega-mlx/_work/cppmega_mlx/cppmega_mlx/vbgui/e2e/scenarios/01_canvas_smoke.spec.ts:29:15
[chromium] › e2e/scenarios/01_canvas_smoke.spec.ts:19:5 › canvas smoke (proof of life) › preset qwen3_next runs smoke pipeline through GUI: vbgui/e2e/fixtures.ts#L76
1) [chromium] › e2e/scenarios/01_canvas_smoke.spec.ts:19:5 › canvas smoke (proof of life) › preset qwen3_next runs smoke pipeline through GUI Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: expect(locator).toContainText(expected) failed Locator: getByTestId('run-result-modal').getByTestId('run-result-overall') Expected substring: "ok" Received string: "· fail · 0.1 ms" Timeout: 5000ms Call log: - Expect "toContainText" with timeout 5000ms - waiting for getByTestId('run-result-modal').getByTestId('run-result-overall') 14 × locator resolved to <span data-testid="run-result-overall">· fail · 0.1 ms</span> - unexpected value "· fail · 0.1 ms" at ../fixtures.ts:76 74 | ): Promise<void> { 75 | const overall = modal.getByTestId("run-result-overall"); > 76 | await expect(overall).toContainText(expected); | ^ 77 | } 78 | 79 | export async function closeModal(page: Page): Promise<void> { at assertOverallStatus (/Users/dave/actions-runner-cppmega-mlx/_work/cppmega_mlx/cppmega_mlx/vbgui/e2e/fixtures.ts:76:25) at /Users/dave/actions-runner-cppmega-mlx/_work/cppmega_mlx/cppmega_mlx/vbgui/e2e/scenarios/01_canvas_smoke.spec.ts:29:15
Preset matrix (smoke, self-hosted macOS) (3)
Canceling since a higher priority waiting request for e2e-matrix-refs/heads/codex/self-hosted-workflows exists
Preset matrix (smoke, self-hosted macOS) (1)
Canceling since a higher priority waiting request for e2e-matrix-refs/heads/codex/self-hosted-workflows exists
Mini-train matrix (macOS, nightly)
Canceling since a higher priority waiting request for e2e-matrix-refs/heads/codex/self-hosted-workflows exists
[chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T3_minimal_no_fim__P2_doc: vbgui/e2e/scenarios/03_train_matrix.spec.ts#L66
10) [chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T3_minimal_no_fim__P2_doc Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: expect(locator).toContainText(expected) failed Locator: getByTestId('run-result-modal').getByTestId('run-result-stage-train') Expected substring: "fail" Received string: "·trainskipped0.0" Timeout: 5000ms Call log: - Expect "toContainText" with timeout 5000ms - waiting for getByTestId('run-result-modal').getByTestId('run-result-stage-train') 14 × locator resolved to <tr data-testid="run-result-stage-train">…</tr> - unexpected value "·trainskipped0.0" 64 | // Train must FAIL with a typed error (vjp / Metal path). Earlier 65 | // stages still must be green to prove the GUI walk works. > 66 | await expect(trainRow).toContainText("fail"); | ^ 67 | await snapshot(page, "03_train_matrix/xfail", key); 68 | } else { 69 | await assertOverallStatus(modal, "ok"); at /Users/dave/actions-runner-cppmega-mlx/_work/cppmega_mlx/cppmega_mlx/vbgui/e2e/scenarios/03_train_matrix.spec.ts:66:32
[chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T3_minimal_no_fim__P1_minimal: vbgui/e2e/scenarios/03_train_matrix.spec.ts#L66
9) [chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T3_minimal_no_fim__P1_minimal Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: expect(locator).toContainText(expected) failed Locator: getByTestId('run-result-modal').getByTestId('run-result-stage-train') Expected substring: "fail" Received string: "·trainskipped0.0" Timeout: 5000ms Call log: - Expect "toContainText" with timeout 5000ms - waiting for getByTestId('run-result-modal').getByTestId('run-result-stage-train') 14 × locator resolved to <tr data-testid="run-result-stage-train">…</tr> - unexpected value "·trainskipped0.0" 64 | // Train must FAIL with a typed error (vjp / Metal path). Earlier 65 | // stages still must be green to prove the GUI walk works. > 66 | await expect(trainRow).toContainText("fail"); | ^ 67 | await snapshot(page, "03_train_matrix/xfail", key); 68 | } else { 69 | await assertOverallStatus(modal, "ok"); at /Users/dave/actions-runner-cppmega-mlx/_work/cppmega_mlx/cppmega_mlx/vbgui/e2e/scenarios/03_train_matrix.spec.ts:66:32
[chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T2_gpt2_small__P4_full: vbgui/e2e/scenarios/03_train_matrix.spec.ts#L66
8) [chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T2_gpt2_small__P4_full Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: expect(locator).toContainText(expected) failed Locator: getByTestId('run-result-modal').getByTestId('run-result-stage-train') Expected substring: "fail" Received string: "·trainskipped0.0" Timeout: 5000ms Call log: - Expect "toContainText" with timeout 5000ms - waiting for getByTestId('run-result-modal').getByTestId('run-result-stage-train') 14 × locator resolved to <tr data-testid="run-result-stage-train">…</tr> - unexpected value "·trainskipped0.0" 64 | // Train must FAIL with a typed error (vjp / Metal path). Earlier 65 | // stages still must be green to prove the GUI walk works. > 66 | await expect(trainRow).toContainText("fail"); | ^ 67 | await snapshot(page, "03_train_matrix/xfail", key); 68 | } else { 69 | await assertOverallStatus(modal, "ok"); at /Users/dave/actions-runner-cppmega-mlx/_work/cppmega_mlx/cppmega_mlx/vbgui/e2e/scenarios/03_train_matrix.spec.ts:66:32
[chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T2_gpt2_small__P3_engram: vbgui/e2e/scenarios/03_train_matrix.spec.ts#L66
7) [chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T2_gpt2_small__P3_engram Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: expect(locator).toContainText(expected) failed Locator: getByTestId('run-result-modal').getByTestId('run-result-stage-train') Expected substring: "fail" Received string: "·trainskipped0.0" Timeout: 5000ms Call log: - Expect "toContainText" with timeout 5000ms - waiting for getByTestId('run-result-modal').getByTestId('run-result-stage-train') 14 × locator resolved to <tr data-testid="run-result-stage-train">…</tr> - unexpected value "·trainskipped0.0" 64 | // Train must FAIL with a typed error (vjp / Metal path). Earlier 65 | // stages still must be green to prove the GUI walk works. > 66 | await expect(trainRow).toContainText("fail"); | ^ 67 | await snapshot(page, "03_train_matrix/xfail", key); 68 | } else { 69 | await assertOverallStatus(modal, "ok"); at /Users/dave/actions-runner-cppmega-mlx/_work/cppmega_mlx/cppmega_mlx/vbgui/e2e/scenarios/03_train_matrix.spec.ts:66:32
[chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T2_gpt2_small__P2_doc: vbgui/e2e/scenarios/03_train_matrix.spec.ts#L66
6) [chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T2_gpt2_small__P2_doc Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: expect(locator).toContainText(expected) failed Locator: getByTestId('run-result-modal').getByTestId('run-result-stage-train') Expected substring: "fail" Received string: "·trainskipped0.0" Timeout: 5000ms Call log: - Expect "toContainText" with timeout 5000ms - waiting for getByTestId('run-result-modal').getByTestId('run-result-stage-train') 14 × locator resolved to <tr data-testid="run-result-stage-train">…</tr> - unexpected value "·trainskipped0.0" 64 | // Train must FAIL with a typed error (vjp / Metal path). Earlier 65 | // stages still must be green to prove the GUI walk works. > 66 | await expect(trainRow).toContainText("fail"); | ^ 67 | await snapshot(page, "03_train_matrix/xfail", key); 68 | } else { 69 | await assertOverallStatus(modal, "ok"); at /Users/dave/actions-runner-cppmega-mlx/_work/cppmega_mlx/cppmega_mlx/vbgui/e2e/scenarios/03_train_matrix.spec.ts:66:32
[chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T2_gpt2_small__P1_minimal: vbgui/e2e/scenarios/03_train_matrix.spec.ts#L66
5) [chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T2_gpt2_small__P1_minimal Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: expect(locator).toContainText(expected) failed Locator: getByTestId('run-result-modal').getByTestId('run-result-stage-train') Expected substring: "fail" Received string: "·trainskipped0.0" Timeout: 5000ms Call log: - Expect "toContainText" with timeout 5000ms - waiting for getByTestId('run-result-modal').getByTestId('run-result-stage-train') 14 × locator resolved to <tr data-testid="run-result-stage-train">…</tr> - unexpected value "·trainskipped0.0" 64 | // Train must FAIL with a typed error (vjp / Metal path). Earlier 65 | // stages still must be green to prove the GUI walk works. > 66 | await expect(trainRow).toContainText("fail"); | ^ 67 | await snapshot(page, "03_train_matrix/xfail", key); 68 | } else { 69 | await assertOverallStatus(modal, "ok"); at /Users/dave/actions-runner-cppmega-mlx/_work/cppmega_mlx/cppmega_mlx/vbgui/e2e/scenarios/03_train_matrix.spec.ts:66:32
[chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T1_cppmega_v3__P4_full: vbgui/e2e/scenarios/03_train_matrix.spec.ts#L66
4) [chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T1_cppmega_v3__P4_full Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: expect(locator).toContainText(expected) failed Locator: getByTestId('run-result-modal').getByTestId('run-result-stage-train') Expected substring: "fail" Received string: "·trainskipped0.0" Timeout: 5000ms Call log: - Expect "toContainText" with timeout 5000ms - waiting for getByTestId('run-result-modal').getByTestId('run-result-stage-train') 14 × locator resolved to <tr data-testid="run-result-stage-train">…</tr> - unexpected value "·trainskipped0.0" 64 | // Train must FAIL with a typed error (vjp / Metal path). Earlier 65 | // stages still must be green to prove the GUI walk works. > 66 | await expect(trainRow).toContainText("fail"); | ^ 67 | await snapshot(page, "03_train_matrix/xfail", key); 68 | } else { 69 | await assertOverallStatus(modal, "ok"); at /Users/dave/actions-runner-cppmega-mlx/_work/cppmega_mlx/cppmega_mlx/vbgui/e2e/scenarios/03_train_matrix.spec.ts:66:32
[chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T1_cppmega_v3__P3_engram: vbgui/e2e/scenarios/03_train_matrix.spec.ts#L66
3) [chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T1_cppmega_v3__P3_engram Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: expect(locator).toContainText(expected) failed Locator: getByTestId('run-result-modal').getByTestId('run-result-stage-train') Expected substring: "fail" Received string: "·trainskipped0.0" Timeout: 5000ms Call log: - Expect "toContainText" with timeout 5000ms - waiting for getByTestId('run-result-modal').getByTestId('run-result-stage-train') 14 × locator resolved to <tr data-testid="run-result-stage-train">…</tr> - unexpected value "·trainskipped0.0" 64 | // Train must FAIL with a typed error (vjp / Metal path). Earlier 65 | // stages still must be green to prove the GUI walk works. > 66 | await expect(trainRow).toContainText("fail"); | ^ 67 | await snapshot(page, "03_train_matrix/xfail", key); 68 | } else { 69 | await assertOverallStatus(modal, "ok"); at /Users/dave/actions-runner-cppmega-mlx/_work/cppmega_mlx/cppmega_mlx/vbgui/e2e/scenarios/03_train_matrix.spec.ts:66:32
[chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T1_cppmega_v3__P2_doc: vbgui/e2e/scenarios/03_train_matrix.spec.ts#L66
2) [chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T1_cppmega_v3__P2_doc Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: expect(locator).toContainText(expected) failed Locator: getByTestId('run-result-modal').getByTestId('run-result-stage-train') Expected substring: "fail" Received string: "·trainskipped0.0" Timeout: 5000ms Call log: - Expect "toContainText" with timeout 5000ms - waiting for getByTestId('run-result-modal').getByTestId('run-result-stage-train') 14 × locator resolved to <tr data-testid="run-result-stage-train">…</tr> - unexpected value "·trainskipped0.0" 64 | // Train must FAIL with a typed error (vjp / Metal path). Earlier 65 | // stages still must be green to prove the GUI walk works. > 66 | await expect(trainRow).toContainText("fail"); | ^ 67 | await snapshot(page, "03_train_matrix/xfail", key); 68 | } else { 69 | await assertOverallStatus(modal, "ok"); at /Users/dave/actions-runner-cppmega-mlx/_work/cppmega_mlx/cppmega_mlx/vbgui/e2e/scenarios/03_train_matrix.spec.ts:66:32
[chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T1_cppmega_v3__P1_minimal: vbgui/e2e/scenarios/03_train_matrix.spec.ts#L66
1) [chromium] › e2e/scenarios/03_train_matrix.spec.ts:48:5 › mini-train matrix (192 cells) › qwen3_next__T1_cppmega_v3__P1_minimal Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: expect(locator).toContainText(expected) failed Locator: getByTestId('run-result-modal').getByTestId('run-result-stage-train') Expected substring: "fail" Received string: "·trainskipped0.0" Timeout: 5000ms Call log: - Expect "toContainText" with timeout 5000ms - waiting for getByTestId('run-result-modal').getByTestId('run-result-stage-train') 14 × locator resolved to <tr data-testid="run-result-stage-train">…</tr> - unexpected value "·trainskipped0.0" 64 | // Train must FAIL with a typed error (vjp / Metal path). Earlier 65 | // stages still must be green to prove the GUI walk works. > 66 | await expect(trainRow).toContainText("fail"); | ^ 67 | await snapshot(page, "03_train_matrix/xfail", key); 68 | } else { 69 | await assertOverallStatus(modal, "ok"); at /Users/dave/actions-runner-cppmega-mlx/_work/cppmega_mlx/cppmega_mlx/vbgui/e2e/scenarios/03_train_matrix.spec.ts:66:32
E2E Coverage Matrix
Canceling since a higher priority waiting request for e2e-matrix-refs/heads/codex/self-hosted-workflows exists
Preset matrix (smoke, self-hosted macOS) (4)
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5, actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020, actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
Preset matrix (smoke, self-hosted macOS) (2)
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5, actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020, actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
Specialised + canvas smoke
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5, actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020, actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
Mini-train matrix (macOS, nightly)
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5, actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020, actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
Matrix report
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5, actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093, actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065, actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
🎭 Playwright Run Summary
3 failed [chromium] › e2e/scenarios/01_canvas_smoke.spec.ts:19:5 › canvas smoke (proof of life) › preset qwen3_next runs smoke pipeline through GUI [chromium] › e2e/scenarios/01_canvas_smoke.spec.ts:19:5 › canvas smoke (proof of life) › preset kimi_linear runs smoke pipeline through GUI [chromium] › e2e/scenarios/01_canvas_smoke.spec.ts:19:5 › canvas smoke (proof of life) › preset gemma4 runs smoke pipeline through GUI 28 passed (48.8s)

Artifacts

Produced during runtime
Name Size Digest
e2e_matrix_report
1001 Bytes
sha256:3a154630a77383a04b889f20d3d89b2b1c15256ccdff4251e631351371d26c00
mini-train-results Expired
55.9 MB
sha256:72aaebb65140652bed31e4b97dbc15ddd96390519e8da0f36786f32e2a3d7171
preset-matrix-shard-2 Expired
933 Bytes
sha256:77c8ab6adc69bdf0069a014d931f3737de339b23c5993b87cda3d32847af8fd5
preset-matrix-shard-4 Expired
931 Bytes
sha256:01af52eb73cacca09d5a0e90639773b7790d0d1133c1cdaf35195c8d2cd0aaca
specialised-results Expired
13.1 MB
sha256:cae55045b11953fc30ff3ae819117c0b5441e0c719ece9e625b711e29cb3a87f