doc: validation proposal#177
Conversation
|
|
||
| | Trigger | Scope | Gates? | | ||
| | ---------------------------- | ------------------------------------ | ------------ | | ||
| | OV bump PR / PAI PR | smoke (CPU) | yes | |
There was a problem hiding this comment.
I think I'd be useful also figure out model storage approach here. We can have models pre-converted to IRs and stored somewhere (HF?), or obtain them on-the-fly. For some policies export takes significant time (~10min), which would increase CI latency, since we'd like to run smoke tests on every PAI PR
| ## Next | ||
|
|
||
| - Build the `ov_smoke` test (three contract stages, model discovery, version logging). | ||
| - Lock `openvino` + `openvino_tokenizers` as a matched pair in |
There was a problem hiding this comment.
We may need to create Renovate rule that will group openvino + openvino_tokenizers version bump in one PR
| - Add a small tokenizer-bearing model the gate can run — the tokenizer break | ||
| only shows on a model with an OV tokenizer; pi05 is 6.25 GB (too large for | ||
| hosted CI) and ACT has none, so the gate needs a small tokenizer model. | ||
| - Wire `ov_smoke` into [`library.yml`](../../.github/workflows/library.yml) as a |
There was a problem hiding this comment.
Are we going to run ov_smoke on every PR or only for PRs that bump OpenVINO version?
If we need to achieve the second option, it can be implemented via Renovate rule that will open PR with openvino + openvino_tokenizers bump from pre-defined branch name and then execute ov_smoke if branch_name == predefined.
| code. The OV team does not author test logic in-house. | ||
| - **Orchestration and reporting live in OpenVINO** — the workflow YAMLs, runners, | ||
| matrix, and dashboards are OV-side tooling. OV consumes the physicalai test by | ||
| pinned tag (`git clone physicalai@<tag>` → `pytest -m ov_smoke`). |
There was a problem hiding this comment.
Currently, our tagging strategy covers only releases.
Do we need to maintain PhAI release candidate tags as well?
For weekly tests - can we use git clone physicalai@main? In this case we would not need to implement automation on the OpenVINO repo side to periodically bump the version (though it will be mutable).
3 participants
Summary
Why
Validation
Breaking changes
Related issues