chore(ci): e2e report improve#2355
Merged
Merged
Conversation
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Capture Ginkgo output as a fallback report source so BeforeSuite failures are surfaced even when the JSON report is missing. Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
nevermarine
requested changes
May 15, 2026
- Merge Ginkgo JSON and output parsers behind a single parseGinkgoFile helper with declarative source descriptors. - Build a single suiteNodePattern and extract small predicates for the Ginkgo stdout reason block. - Show the suite node name (e.g. SynchronizedAfterSuite) as the failed group in the messenger thread instead of "Unknown". Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
The previous suite node regex was too permissive: it matched a plain "[SynchronizedBeforeSuite]" section header that appears in Ginkgo stdout before the actual "[FAILED]" line. As a result, the log-only fallback could not extract suite setup failures from real CI output. Tighten the pattern to require either "[X] [FAILED]" (body) or "[FAIL] [X]" (summary) and add a regression test based on a real BeforeSuite timeout from CI. Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
- Describe the test-results table as a single columns array with header, alignment, and value functions so the "Errors" column is just one entry that is appended conditionally, instead of duplicating the row template for the with- and without-errors variants. - Move getFailedTestEntries inside summarizeFailedTestGroups so the caller passes a single report argument instead of pre-extracting entries on every call site. Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Document the new helpers introduced by recent refactors: - cluster-report.js: findGinkgoOutput, emptyParsedReport, parseGinkgoFile plus a GinkgoSourceDescriptor typedef used by ginkgoJsonSource and ginkgoOutputSource. - shared/ginkgo-report-utils.js: findFailedSuiteNode, extractFailureReasonFromOutput, parseGinkgoOutput. - messenger/markdown.js: buildTestResultsColumns, buildMarkdownRow, summarizeFailedTestGroups plus a TestResultsColumn typedef. Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Three test files (cluster-report.test.js, messenger-report.test.js, fs-utils.test.js) duplicated the same temp-directory and core-mock helpers. Move them to a single shared/test-utils.js module and let each test wrap it with its own prefix via a thin inTempDir helper. Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
archivedReportPattern and ginkgoOutputPattern both inlined the same regex-escape expression. Move it to a tiny escapeRegExp helper so the two callers share one implementation. Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
The six-branch if-chain was a pure status → template mapping. Replace it with a statusMessageTemplates object that makes the symmetry between statuses obvious and keeps the function body to a single line. Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
normalizeJobResult guarantees one of success/cancelled/skipped/failure, so once the loop has filtered out success, stageResult already matches the status name we want to emit. The nested ternary was an identity mapping; use stageResult directly. Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
findGinkgoReport and findGinkgoOutput were two thin wrappers around findSingleMatchingFile that only differed in the file-name regex and the human-readable label. The same duplication leaked into the call site, which picked the source descriptor twice (once via the if/else on rawReportPath, once when calling parseGinkgoFile). Move the file-name pattern into the source descriptor itself, expose a single findGinkgoSource(config, source) lookup, and call parseGinkgoFile once with the descriptor selected up front. Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Replace the three near-identical if blocks with a loop over a list of required keys. The error messages stay in the same "buildClusterReport requires <key>" format, so existing tests and CI behaviour are unchanged. Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
splitReportsBySection re-checked getReportClusterKey for every filter. Filter once up front and chain the three category predicates against the pre-filtered list. renderClusterFailuresSection and renderMissingReportsSection only differed in the heading and the per-report message lookup. Extract a single renderBulletSection helper and pass per-section getMessage functions for cluster failures and missing reports. The emitted markdown is byte-for-byte identical to the previous output, which the existing messenger-report tests verify. Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
The previous signature unpacked apiUrl/channelId/token out of the loop object only to be reassembled at every call site. Take the loop credentials object as-is and accept (loop, message, rootId, core) so the two call sites in makeThreadedReportInLoop become one-line calls. Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
nevermarine
approved these changes
May 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Improve the E2E messenger report generated by CI and reorganize the
report scripts so the same logic is easier to follow.
User-facing changes:
Reasoncolumn to the failed-tests thread table and renameTest grouptoTests.workflow/job URL.
Errorscolumn in the main test-results table when thereare no Ginkgo errors.
Success Ratefrom executed specs only:passed / (passed + failed + errors).Failure.Messagetext as the failed-test reason.*exec.ExitErrorwith%wintest/e2e/internal/framework/ssh.goso Ginkgo no longer dumps Gointernals into the report when an SSH command fails.
a fallback report source so suite-level failures such as
SynchronizedBeforeSuiteare still surfaced when the JSON report ismissing or contains only setup failure data.
Internal refactors (no behavior change, covered by existing tests):
withTempDir/createCorejest helpers intoshared/test-utils.jsto remove duplication across three test files.escapeRegExpinshared/report-model.js; consolidatebuildStatusMessageinto a status → template lookup table; remove aredundant ternary in
buildClusterStatus.findGinkgoReport/findGinkgoOutput→ onefindGinkgoSource) and callparseGinkgoFileonce with the descriptor picked up front.requireClusterReportConfiginstead of three identicalifblocks.renderClusterFailuresSection/renderMissingReportsSectionintoa single
renderBulletSectionhelper. Output stays byte-identical;verified by existing
messenger-reporttests.loopcredentials object as-is topostToLoopApiso thetwo call sites collapse to one-liners.
Why do we need it, and what problem does it solve?
The current E2E notification thread only lists failed test groups and
does not show why a test failed. This forces engineers to open CI
artifacts and logs even for simple failures.
The main report also counted skipped specs in the
Success Ratedenominator, which made Ginkgo runs look worse than the executed-spec
result. SSH command failures contained noisy Go internals in
Failure.Messagebecause*exec.ExitErrorwas wrapped with%w.When E2E fails in
SynchronizedBeforeSuite, Ginkgo may skip all specsand the JSON report may be absent or contain no regular test failures.
Previously the messenger report could only show a generic storage
failure (for example
nfswithE2E TEST FAILED), without the actualsetup failure reason. The pipeline now captures Ginkgo stdout into
e2e_output_<storage>_<date>.logand the report parses suite-levelfailures out of it as a fallback.
The follow-up refactors remove duplicated helpers, collapse parallel
branches, and replace inline regex/
if-chain logic with declarativedata, making the report scripts easier to read and extend without
changing the rendered output.
What is the expected result?
failed specs.
Errorscolumn is hidden when all error counts are zero;Success Rateis calculated without skipped specs;TestsandReasoncolumns;Reasoncontains the GinkgoFailure.Messagetext.*exec.ExitErrorinternals.SynchronizedBeforeSuiteand no JSON report iscreated, the report is still generated from the captured Ginkgo
output and the failed-tests thread shows the suite-level failure
reason. Suite-level failures do not change spec metrics such as
passed, failed, errors, or total.
.github/scripts/js/e2e/report/still passes(37 tests) and
tmp/test-ci/report/run.shproduces the samemarkdown for the bundled fixtures.
Checklist
Changelog entries