Test plan design

[mgumowsk]

Test Type, Location, Purpose

Model API

Test type	Location	Purpose	Devices	Speed
Unit tests	model_api/tests/unit/	Fast, mocked — validates pre/postprocessing, result types, adapters, etc.	CPU	<10s
Integration tests	model_api/tests/integration/	Tests that exported model integrates with model_api. Download from HF → model(image) → crash = fail	CPU	~minutes
Functional tests	model_api/tests/functional/	Reference-based output comparison per model type, tolerances per device. Refactor current accuracy tests. Trigger on specific model class change.	CPU	~minutes
Nightly tests	model_api/tests/nigthly/	Run integration tests across various hardware.	CPU, NPU, iGPU	~hours
Performance tests	model_api/tests/performance/	Tests the performance of hf models across various hardware. Test preprocessing, postprocessing, bare ov model. Need design.	CPU, NPU, GPU	~days

Model Converter

Test type	Location	Purpose	Devices	Speed
Unit tests	model_converter/tests/unit/	Fast, mocked — validates converters code	CPU	<10s
Integration tests	model_converter/tests/integration	verify IR → accuracy drop check → HF structure validation . Needs design.	CPU	~hours
Release tests	model_converter/tests/release-tests	Runs integration tests on across various hardware. Run after OV release. Needs design.	CPU, NPU, GPU	~hours

To be planned

• How to handle user experience, how to handle user models?
• Integrate Anomalib tests
• integrate OTX tests