Skip to content

[NOT FOR MERGE][DEBUG] CVS-183350: Add debug instrumentation for CI reproduction#1694

Closed
yuxu42 wants to merge 6 commits intohuggingface:mainfrom
yuxu42:support_ov_2026_1
Closed

[NOT FOR MERGE][DEBUG] CVS-183350: Add debug instrumentation for CI reproduction#1694
yuxu42 wants to merge 6 commits intohuggingface:mainfrom
yuxu42:support_ov_2026_1

Conversation

@yuxu42
Copy link
Copy Markdown

@yuxu42 yuxu42 commented Apr 21, 2026

Purpose

This PR adds debug instrumentation to investigate CVS-183350 (OpenVINO 2026.1.0 inference precision regression) that only reproduces on CI hardware but not locally.

Changes

  • Add setUpClass to print CI CPU info (model name, flags)
  • Add per-test debug output for opt/pegasus/xglm showing:
    • Actual max diff values
    • Tolerance threshold
    • Logits samples

Expected CI Behavior

Tests for opt and pegasus will fail, but will print diagnostic information showing:

  • CI CPU model (likely Intel Xeon Cascade Lake/Ice Lake)
  • Actual numerical diff (expected > 1e-4)
  • Logits comparison

This data will help root cause the OpenVINO CPU plugin regression.

Note

This is a debug PR - not intended to merge. Will be reverted after collecting CI diagnostics.

rkazants and others added 6 commits April 16, 2026 21:24
The latest OpenVINO is required for MoE optimizations

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Add diagnostic output to capture:
- CPU model and flags from lscpu
- Actual max diff values for opt/pegasus/xglm
- Logits samples for comparison

This will help identify the exact hardware and numerical differences
on CI runners (Cascade Lake/Ice Lake) vs local (Sapphire Rapids).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@rkazants rkazants closed this Apr 21, 2026
@rkazants rkazants reopened this Apr 21, 2026
@rkazants rkazants changed the title [DEBUG] CVS-183350: Add debug instrumentation for CI reproduction [NOT FOR MERGE][DEBUG] CVS-183350: Add debug instrumentation for CI reproduction Apr 21, 2026
@HuggingFaceDocBuilderDev
Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@rkazants
Copy link
Copy Markdown
Collaborator

Looks no longer needed PR

@rkazants rkazants closed this Apr 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants