Skip to content

E2E: scorecard GitHub open PRs aggregated test fails due to pod restart timing #2602

Description

@subhashkhileri

Classification

fix_category: infra_flake

Failed Tests

Test Error
[scorecard] Aggregated scorecard (GitHub): info tooltips, drill-down, table UI expect(locator).toBeVisible() failed - getByTestId('scorecard-homepage-card-github.open_prs').getByText('Ideal', { exact: true }) not found; card shows "0 entities" and "No data found"
[scorecard] (retry1) Same test page.waitForEvent: Timeout 10000ms exceeded while waiting for event "popup" - RHDH pod down ("Application is not available")

Root Cause

Infrastructure timing issue caused by a pod restart:

  1. The RHDH pod started and the github.open_prs scheduled task was registered with a 5s initial delay
  2. At 04:55:08, the first pull completed with 0 entities because the catalog hadn't finished processing entities yet
  3. At 04:56:14, the liveness probe failed with context deadline exceeded and the pod was killed
  4. The new pod started at 04:56:58 and registered the task again, but the scheduled pull never completed before the test ran at ~04:59:51
  5. The test saw "0 entities" / "No data found" for the GitHub open PRs card because metrics were never populated
  6. On retry, the pod was again restarting (due to another liveness failure), showing "Application is not available"

The Jira metrics (jira.open_issues) processed 4 entities successfully at 05:00:11, and OpenSSF metrics processed 2 entities each. The github.open_prs task simply didn't execute a pull cycle in the new pod before the test ran.

Remediation

This is a transient infrastructure issue. The test already has retries: 1 configured, but the retry failed because the pod restart cascaded into a longer outage. No code fix needed — the 2-minute stabilization wait in beforeAll is already in place. The pod restart was caused by a transient liveness probe timeout, not a systematic issue.

Artifacts

https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-redhat-developer-rhdh-plugin-export-overlays-main-e2e-ocp-helm-nightly/2064921035702013952

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions