Skip to content

[CI] [GHA] Add validation schemas for artefacts created by CI Doctor (investigation and pattern files)#36640

Merged
akashchi merged 14 commits into
openvinotoolkit:masterfrom
akashchi:ci/gha/json-schema-for-artefacts-ci-doctor
Jul 1, 2026
Merged

[CI] [GHA] Add validation schemas for artefacts created by CI Doctor (investigation and pattern files)#36640
akashchi merged 14 commits into
openvinotoolkit:masterfrom
akashchi:ci/gha/json-schema-for-artefacts-ci-doctor

Conversation

@akashchi

Copy link
Copy Markdown
Contributor

Tickets:

  • 189802

AI Assistance:

  • AI assistance used: yes
    • Backbones of the .json schemas

@akashchi akashchi added this to the 2026.3 milestone Jun 30, 2026
@akashchi akashchi requested review from a team as code owners June 30, 2026 11:44
@akashchi akashchi requested review from michalzx and removed request for a team June 30, 2026 11:44
@github-actions github-actions Bot added category: CI OpenVINO public CI category: docs OpenVINO documentation github_actions Pull requests that update GitHub Actions code labels Jun 30, 2026
@akashchi akashchi requested a review from as-suvorov June 30, 2026 11:50
@akashchi akashchi requested a review from Copilot June 30, 2026 13:48

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the CI Doctor GitHub Actions workflows to support schema-driven validation for CI Doctor MQ artefacts (investigation + pattern JSON files), and refreshes the generated lock workflows/actions to newer gh-aw/firewall tooling versions.

Changes:

  • Add JSON Schemas for CI Doctor MQ investigation and pattern artefacts under .github/ci-doctor-mq/schemas/.
  • Update CI Doctor MQ instructions to require schema conformance and validation when writing artefacts.
  • Regenerate/update the locked CI Doctor workflows and the gh-aw actions lockfile (version bumps + new env/policy wiring).

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
.github/workflows/ci-doctor.lock.yml Regenerated locked workflow (tooling version bumps, runtime features/policy wiring, and safe-outputs schema updates).
.github/workflows/ci-doctor-mq.md Updates CI Doctor MQ runbook to enforce schema-based artefact validation and job-agnostic signatures.
.github/workflows/ci-doctor-mq.lock.yml Regenerated locked MQ workflow (tooling version bumps, runtime features/policy wiring, safe-outputs description updates).
.github/ci-doctor-mq/schemas/pattern.schema.json New JSON Schema defining the canonical aggregated pattern record format.
.github/ci-doctor-mq/schemas/investigation.schema.json New JSON Schema defining the canonical investigation record format.
.github/aw/actions-lock.json Updates pinned gh-aw action versions/SHAs to match regenerated workflows.

Comment thread .github/workflows/ci-doctor-mq.md
Comment thread .github/workflows/ci-doctor.lock.yml
Comment thread .github/workflows/ci-doctor.lock.yml
@akashchi akashchi enabled auto-merge July 1, 2026 07:54
@akashchi akashchi added this pull request to the merge queue Jul 1, 2026
@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

CI Doctor — Merge Queue failure on this PR

Pipeline: Windows (VS 2022, Python 3.11, Debug)
Failure: Windows Build fails - FinalizeArtifact ENOTFOUND on self-hosted runner

Possible remedy

  1. Re-run the failed job — the build itself succeeded (all ~126MB of artifact bytes were uploaded to blob storage). Only the finalization API call failed. A simple re-run should pass.
  2. If re-runs keep failing, check that the self-hosted Windows runner has stable DNS resolution to all GitHub endpoints (see GitHub docs).
  3. This is not caused by your PR changes — the PR modifies CI/GHA validation schemas, not the build or runner configuration.

What happened

The Build / Build job failed at the Upload OpenVINO package step with ##[error]Failed to FinalizeArtifact: Unable to make request: ENOTFOUND. The artifact data was fully uploaded to blob storage, but the self-hosted Windows runner lost DNS resolution when making the final registration call to GitHub's artifact API. A similar transient network failure (ECONNREFUSED) was seen on Windows self-hosted runners on 2026-06-29.

Generated by CI Failure Doctor — Merge Queue · 112.5 AIC · ⌖ 10.2 AIC · ⊞ 15.6K ·

@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

CI Doctor — Merge Queue failure on this PR

Pipeline: Windows (VS 2022, Python 3.11, Release)
Failure: test_keras_lstm_flags_float32 fails with LSTM accuracy mismatch on CPU

Possible remedy

  1. Re-queue the PR — this is a pre-existing flaky test unrelated to your changes (CI Doctor has seen this failure 3 times; first on 2026-06-26). Your PR only adds JSON validation schemas for CI Doctor artifacts.
  2. If the failure persists, a team member can add the specific test parameterization to an xfail skip-list on Windows CPU with a tracking ticket.
  3. Long-term fix: disable flags=True (training-mode dropout) in the failing test parameterization or fix the random seed to ensure deterministic LSTM outputs.
  4. If widening tolerance is preferred, rtol=0.2 would accommodate the observed max diff of ~0.117.

What happened

The TensorFlow 2 Layer Tests - TF FE step failed because test_keras_lstm_flags_float32 produced a max numerical diff of 0.117 between OpenVINO CPU and TensorFlow reference. This is caused by training-mode dropout (flags=True, dropout rates 0.1/0.3) compounding across 8 LSTM time steps, which produces non-deterministic results that exceed the comparison threshold on Windows.

This same flaky test has been seen 3 times total (runs 28219131186, 28161192347, and the current run), all on PRs unrelated to LSTM or TF frontend.

Generated by CI Failure Doctor — Merge Queue · 121.7 AIC · ⌖ 12.9 AIC · ⊞ 15.6K ·

@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jul 1, 2026
@akashchi akashchi added this pull request to the merge queue Jul 1, 2026
Merged via the queue into openvinotoolkit:master with commit 2958752 Jul 1, 2026
188 of 190 checks passed
@akashchi akashchi deleted the ci/gha/json-schema-for-artefacts-ci-doctor branch July 1, 2026 17:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

category: CI OpenVINO public CI category: docs OpenVINO documentation github_actions Pull requests that update GitHub Actions code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants