test: validate E2E coverage infrastructure#2550
Conversation
This PR validates the E2E coverage collection pipeline added in redhat-developer#2383: - Instrument job creates coverage-tagged OCI images - E2E tests collect browser coverage via Istanbul - Coverage uploads to Codecov with upstream repo attribution Will be closed without merge after validation.
|
Skipping CI for Draft Pull Request. |
|
/publish |
|
Publish workflow has completed with success. Publishing process✅ Finished successfully.
Backstage Compatibility Check✅ All workspaces are compatible with the target Backstage version ( No action required. Metadata Validation✅ All metadata files validated successfully. Running e2e tests |
✅ Passed E2E Tests -
|
…2551) fix: provide dummy command to podman create for static plugin images Plugin OCI images are static file bundles with no CMD or ENTRYPOINT. Running 'podman create' without a command argument fails with: Error: no command or entrypoint provided, and no CMD or ENTRYPOINT from image Fix: Pass /bin/true as a dummy command. The container is never run, just created so we can extract files with podman cp. Fixes instrumentation job failure on PR #2550. Ref: https://github.com/redhat-developer/rhdh-plugin-export-overlays/actions/runs/26902663501
|
/publish |
|
Publish workflow has completed with success. Publishing process✅ Finished successfully.
Backstage Compatibility Check✅ All workspaces are compatible with the target Backstage version ( No action required. Metadata Validation✅ All metadata files validated successfully. Running e2e tests |
|
✅ Smoke tests workflow passed. All plugins loaded successfully. |
✅ Passed E2E Tests -
|
github-actions
Bot
added
mandatory-workspace
|
/publish |
|
Publish workflow has completed with success. Publishing process✅ Finished successfully.
Backstage Compatibility Check✅ All workspaces are compatible with the target Backstage version ( No action required. Metadata Validation✅ All metadata files validated successfully. Running e2e tests |
|
✅ Smoke tests workflow passed. All plugins loaded successfully. |
✅ Passed E2E Tests -
|
|
/publish |
|
Publish workflow has completed with success. Publishing process✅ Finished successfully.
Backstage Compatibility Check✅ All workspaces are compatible with the target Backstage version ( No action required. Metadata Validation✅ All metadata files validated successfully. Running e2e tests |
|
✅ Smoke tests workflow passed. All plugins loaded successfully. |
|
/test e2e-ocp-helm |
✅ Passed E2E Tests -
|
Version 2.1.0 includes the automatic __coverage image swap logic from PR redhat-developer/rhdh-e2e-test-utils#95 (merged 2026-06-04). Changes: - Auto-fixture _coverageCollector that collects window.__coverage__ - Automatic swap to __coverage images when E2E_COLLECT_COVERAGE=true and plugin role is frontend-plugin This enables E2E coverage collection to work end-to-end.
gustavolira
force-pushed
the
test/validate-coverage-infrastructure
branch
from
cec1468 to
c94b35f
Compare
✅ Passed E2E Tests -
|
|
/publish |
|
Publish workflow has completed with success. Publishing process✅ Finished successfully.
Backstage Compatibility Check✅ All workspaces are compatible with the target Backstage version ( No action required. Metadata Validation✅ All metadata files validated successfully. Running e2e tests |
|
✅ Smoke tests workflow passed. All plugins loaded successfully. |
✅ Passed E2E Tests -
|
|
/publish |
|
Publish workflow has completed with success. Publishing process✅ Finished successfully.
Backstage Compatibility Check✅ All workspaces are compatible with the target Backstage version ( No action required. Metadata Validation✅ All metadata files validated successfully. Running e2e tests |
|
✅ Smoke tests workflow passed. All plugins loaded successfully. |
✅ Passed E2E Tests -
|
|
/publish |
|
Publish workflow has completed with success. Publishing process✅ Finished successfully.
Backstage Compatibility Check✅ All workspaces are compatible with the target Backstage version ( No action required. Metadata Validation✅ All metadata files validated successfully. Running e2e tests |
|
✅ Smoke tests workflow passed. All plugins loaded successfully. |
✅ Passed E2E Tests -
|
The __coverage image only instrumented the Module Federation bundle (dist/),
but RHDH can serve/execute the Scalprum bundle (dist-scalprum/) at runtime,
which was left uninstrumented — so window.__coverage__ was never created even
though the image looked correctly instrumented.
Verified by pulling pr_2550__1.17.0__coverage: dist/ had 208 instrumented files
with the globalThis fix and a top-level cov_() call in remoteEntry.js, while
dist-scalprum/ had zero. The MF container global was present on the page's
globalThis but __coverage__ was not, confirming the executed bundle was not the
instrumented one.
Changes:
- instrument-plugin.sh: loop over dist/ and dist-scalprum/, instrument every
bundle that exists, overlay all via generated COPY lines. Warn loudly if the
globalThis sed fix leaves any unfixed `new Function("return this")`.
- instrument-plugin.sh: fix OCI dynamic-packages label parse — the dir path is
the object key ([{"<dir>": {"name": ...}}]), not a top-level `name` field. The
old `.[0].name` returned empty and only worked via the metadata fallback.
- tech-radar.spec.ts: add a coverage-mode-only diagnostic probe that fetches the
served MF remoteEntry.js and the Scalprum bundle and logs whether each carries
Istanbul markers, so the served-vs-instrumented question is answered directly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
/publish |
|
Publish workflow has completed with success. Publishing process✅ Finished successfully.
Backstage Compatibility Check✅ All workspaces are compatible with the target Backstage version ( No action required. Metadata Validation✅ All metadata files validated successfully. Running e2e tests |
|
✅ Smoke tests workflow passed. All plugins loaded successfully. |
✅ Passed E2E Tests -
|
|
/test e2e-ocp-helm |
✅ Passed E2E Tests -
|
…ted code RHDH's install-dynamic-plugins only extracts manifest.layers[0] — it assumes dynamic-plugin OCI images are single-layer (image-cache.ts: "keeps each OCI image's single-layer tarball"). Building the coverage image as `FROM prod + COPY` produced a multi-layer image whose FIRST layer is the original uninstrumented base, so RHDH served the original code and ignored our instrumented overlay layers — window.__coverage__ was never created. Fix: build with `podman build --squash-all` so the result is a single layer containing the merged instrumented filesystem; assert single-layer afterward. Verified locally by replicating RHDH's exact extraction (skopeo copy → extract layers[0] only): the main scalprum bundle in layers[0] goes from 24,028 B (original) without squash to 290,210 B (instrumented) with --squash-all. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
/publish |
|
|
Publish workflow has completed with success. Publishing process✅ Finished successfully.
Backstage Compatibility Check✅ All workspaces are compatible with the target Backstage version ( No action required. Metadata Validation✅ All metadata files validated successfully. Running e2e tests |
|
✅ Smoke tests workflow passed. All plugins loaded successfully. |
✅ Passed E2E Tests -
|
|
PR needs rebase. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
3 participants
Purpose
This PR validates the E2E coverage collection pipeline implemented in #2383.
What This Tests
instrumentjob creates__coveragetagged OCI imagesTest Plan
/publishto trigger plugin buildinstrumentjob completes successfullybackstage/community-pluginsrepoExpected Artifacts
ghcr.io/.../backstage-community-plugin-tech-radar:pr_XXXX__1.1.0ghcr.io/.../backstage-community-plugin-tech-radar:pr_XXXX__1.1.0__coveragee2e-tech-radarStatus
⏳ Awaiting
/publishcommandNote: This PR will be closed without merge after validation completes.