tests: rename video_test_framework_codec.py to vvs_test_runner.py

Shorter, clearer name for the unified test entry point.
Update all references in CI, READMEs, and unit tests.

Move decode/encode frameworks to libs/ as library modules

Author: dabrain34

.github/workflows/test.yml

@@ -31,11 +31,11 @@ jobs:
 
       - name: Run flake8
         run: |
-          python -m flake8 tests/*/*.py --max-line-length=100 --show-source --statistics
+          python -m flake8 tests/*.py tests/*/*.py --max-line-length=100 --show-source --statistics
 
       - name: Run pylint
         run: |
-          python -m pylint tests/*/*.py --output-format=text
+          python -m pylint tests/*.py tests/*/*.py --output-format=text
 
   unit-tests:
     name: Unit Tests
@@ -82,4 +82,4 @@ jobs:
 
       - name: Run codec test framework
         run: |
-          python3 ./tests/video_test_framework_codec.py
+          python3 ./tests/vvs_test_runner.py

README.md

@@ -120,16 +120,16 @@ The project includes a comprehensive test framework for validating both encoder
 
 ```bash
 # Run all tests with default paths
-python3 tests/video_test_framework_codec.py
+python3 tests/vvs_test_runner.py
 
 # List all available test samples
-python3 tests/video_test_framework_codec.py --list-samples
+python3 tests/vvs_test_runner.py --list-samples
 
 # Test only H.264 decoder tests
-python3 tests/video_test_framework_decode.py --test "decode_h264_*"
+python3 tests/vvs_test_runner.py --decoder-only --test "decode_h264_*"
 
 # Test only H.264 encoder tests
-python3 tests/video_test_framework_encode.py --test "encode_h264_*"
+python3 tests/vvs_test_runner.py --encoder-only --test "encode_h264_*"
 ```
 
 For complete documentation, command-line options, and advanced usage examples, see **[tests/README.md](tests/README.md)**.

tests/README.md

@@ -10,19 +10,19 @@ cmake -B BUILD -DCMAKE_BUILD_TYPE=Release
 cmake --build BUILD --parallel
 
 # Run all tests (auto-downloads test samples)
-python3 tests/video_test_framework_codec.py
+python3 tests/vvs_test_runner.py
 
 # Run only H.264 decoder tests
-python3 tests/video_test_framework_codec.py --decoder-only --codec h264
+python3 tests/vvs_test_runner.py --decoder-only --codec h264
 
 # Run only encoder tests with verbose output
-python3 tests/video_test_framework_codec.py --encoder-only --verbose
+python3 tests/vvs_test_runner.py --encoder-only --verbose
 
 # List all available test samples
-python3 tests/video_test_framework_codec.py --list-samples
+python3 tests/vvs_test_runner.py --list-samples
 
 # Run a specific test by name
-python3 tests/video_test_framework_codec.py --test "h264_4k_main"
+python3 tests/vvs_test_runner.py --test "h264_4k_main"
 ```
 
 ## Table of Contents
@@ -47,9 +47,9 @@ python3 tests/video_test_framework_codec.py --test "h264_4k_main"
 
 ### Core Scripts
 
-- **`video_test_framework_codec.py`** - Unified test orchestrator that runs both encoder and decoder tests
-- **`video_test_framework_encode.py`** - Encoder-specific test runner
-- **`video_test_framework_decode.py`** - Decoder-specific test runner
+- **`vvs_test_runner.py`** - Unified test entry point that runs encoder, decoder, or both tests
+- **`libs/video_test_framework_encode.py`** - Encoder test framework classes (library module)
+- **`libs/video_test_framework_decode.py`** - Decoder test framework classes (library module)
 
 
 ### Configuration Files
@@ -175,10 +175,10 @@ To add a new decode test sample to `decode_samples.json`:
 6. **Test the new entry**:
    ```bash
    # Run only your new test
-   python3 tests/video_test_framework_codec.py --decoder-only --test "your_test_name"
+   python3 tests/vvs_test_runner.py --decoder-only --test "your_test_name"
 
    # Verify MD5 validation works
-   python3 tests/video_test_framework_decode.py --test "your_test_name" --verbose
+   python3 tests/vvs_test_runner.py --decoder-only --test "your_test_name" --verbose
    ```
 
 **Notes:**
@@ -297,10 +297,10 @@ To add a new encode test sample to `encode_samples.json`:
 6. **Test the new entry**:
    ```bash
    # Run only your new test
-   python3 tests/video_test_framework_codec.py --encoder-only --test "your_test_name"
+   python3 tests/vvs_test_runner.py --encoder-only --test "your_test_name"
 
    # With verbose output to see encoder command
-   python3 tests/video_test_framework_encode.py --test "your_test_name" --verbose
+   python3 tests/vvs_test_runner.py --encoder-only --test "your_test_name" --verbose
    ```
 
 **Notes:**
@@ -367,36 +367,36 @@ The framework uses a skip list system to skip tests that are known to fail on sp
 
 ```bash
 # Run tests ignoring the skip list (run all tests)
-python3 video_test_framework_codec.py --ignore-skip-list
+python3 vvs_test_runner.py --ignore-skip-list
 
 # Run only skipped tests (useful for testing fixes)
-python3 video_test_framework_codec.py --only-skipped
+python3 vvs_test_runner.py --only-skipped
 
 # Use a custom skip list
-python3 video_test_framework_codec.py --skip-list my_skip_list.json
+python3 vvs_test_runner.py --skip-list my_skip_list.json
 ```
 
 
 ### Usage Examples
 
 #### Run All Tests
 ```bash
-python3 video_test_framework_codec.py
+python3 vvs_test_runner.py
 ```
 
 #### Run Encoder Tests Only
 ```bash
-python3 video_test_framework_codec.py --encoder-only --codec h264
+python3 vvs_test_runner.py --encoder-only --codec h264
 ```
 
 #### Run Specific Test Pattern
 ```bash
-python3 video_test_framework_codec.py --test "*baseline*" --verbose
+python3 vvs_test_runner.py --test "*baseline*" --verbose
 ```
 
 #### Export Results to JSON
 ```bash
-python3 video_test_framework_codec.py --export-json results.json
+python3 vvs_test_runner.py --export-json results.json
 ```
 
 ### Command Line Options
@@ -422,17 +422,17 @@ python3 video_test_framework_codec.py --export-json results.json
 - `--decode-test-suite FILE` - Path to custom decode test suite JSON file
 - `--encode-test-suite FILE` - Path to custom encode test suite JSON file
 
-#### Codec Framework Only (video_test_framework_codec.py)
+#### Framework Selection
 
 - `--encoder-only` - Run only encoder tests
 - `--decoder-only` - Run only decoder tests
 
-#### Decoder Only (video_test_framework_decode.py)
+#### Decode-Specific Options
 
-- `--display` - Enable display output (removes --noPresent flag)
+- `--decode-display` - Enable display output for decode tests (removes --noPresent flag)
 - `--no-verify-md5` - Disable MD5 verification of decoded output
 
-#### Encoder Only (video_test_framework_encode.py)
+#### Encode-Specific Options
 
 - `--no-validate-with-decoder` - Disable validation of encoder output with decoder
 - `--decoder-args ARGS` - Additional arguments to pass to decoder during validation
@@ -457,17 +457,17 @@ Tests are automatically prefixed with their type for display:
 When filtering tests with `--test`, you can use either the base name or the prefixed name:
 ```bash
 # These are equivalent - run a specific test by base name or full name
-python3 video_test_framework_codec.py --test "h264_4k_main"
-python3 video_test_framework_codec.py --test "decode_h264_4k_main"
+python3 vvs_test_runner.py --test "h264_4k_main"
+python3 vvs_test_runner.py --test "decode_h264_4k_main"
 
 # Run only decoder tests (using prefix)
-python3 video_test_framework_codec.py --test "decode_*"
+python3 vvs_test_runner.py --test "decode_*"
 
 # Run only H.264 encoder tests
-python3 video_test_framework_codec.py --test "encode_h264_*"
+python3 vvs_test_runner.py --test "encode_h264_*"
 
 # Run all AV1 tests (both encode and decode)
-python3 video_test_framework_codec.py --test "av1_*"
+python3 vvs_test_runner.py --test "av1_*"
 ```
 
 ---
@@ -491,10 +491,10 @@ The framework supports [Fluster](https://github.com/fluendo/fluster) test suite
 Example usage:
 ```bash
 # Use Fluster JVT-AVC_V1 test suite
-python3 video_test_framework_decode.py --decode-test-suite path/to/JVT-AVC_V1.json
+python3 vvs_test_runner.py --decoder-only --decode-test-suite path/to/JVT-AVC_V1.json
 
 # Filter specific tests from Fluster suite
-python3 video_test_framework_decode.py --decode-test-suite JVT-AVC_V1.json --test "*baseline*"
+python3 vvs_test_runner.py --decoder-only --decode-test-suite JVT-AVC_V1.json --test "*baseline*"
 ```
 
 **Note**: Fluster format is only supported for decode tests, not encode tests.
@@ -515,19 +515,17 @@ JSON export includes:
 - Individual test results with timing and status information
 - Support for both encoder and decoder test results in unified format
 
-### Individual Framework Usage
+### Running Specific Test Types
 
-Each component can be run independently:
+Use `--encoder-only` or `--decoder-only` to run a single test type:
 
 ```bash
 # List available samples
-python3 video_test_framework_codec.py --list-samples
-python3 video_test_framework_encode.py --list-samples
-python3 video_test_framework_decode.py --list-samples
+python3 vvs_test_runner.py --list-samples
 
 # Encoder tests only
-python3 video_test_framework_encode.py
+python3 vvs_test_runner.py --encoder-only
 
-# Decoder tests only
-python3 video_test_framework_decode.py --display
-```
+# Decoder tests only with display
+python3 vvs_test_runner.py --decoder-only --decode-display
+```

tests/libs/__init__.py

@@ -6,6 +6,8 @@
 - video_test_config_base: Base configuration classes and platform utilities
 - video_test_fetch_sample: Asset download and verification system
 - video_test_framework_base: Base test framework class and common utilities
+- video_test_framework_decode: Decoder test sample and framework classes
+- video_test_framework_encode: Encoder test sample and framework classes
 
 Copyright 2025 Igalia S.L.
 Licensed under the Apache License, Version 2.0

tests/libs/video_test_config_base.py

@@ -244,6 +244,21 @@ class BaseTestConfig:  # pylint: disable=too-many-instance-attributes
     source_checksum: str = ""
     source_filepath: str = ""
 
+    @staticmethod
+    def _parse_base_fields(data: dict) -> dict:
+        """Extract base config fields from a dictionary."""
+        return {
+            "name": data["name"],
+            "codec": CodecType(data["codec"]),
+            "expect_success": data.get("expect_success", True),
+            "extra_args": data.get("extra_args"),
+            "description": data.get("description", ""),
+            "timeout": data.get("timeout"),
+            "source_url": data["source_url"],
+            "source_checksum": data["source_checksum"],
+            "source_filepath": data["source_filepath"],
+        }
+
 
 @dataclass(init=False)
 class TestResult:
@@ -486,7 +501,9 @@ def load_and_download_samples(sample_class, json_file: str,
                   f"{data.get('name', 'unknown')}: {e}")
 
     if not samples:
-        return True
+        print(f"  Warning: all {len(samples_data)} {test_type} sample(s) "
+              f"failed to parse")
+        return False
     return download_sample_assets(samples, f"{test_type} test")
 
 

tests/libs/video_test_framework_base.py

@@ -884,11 +884,4 @@ def _print_single_result(self, result: TestResult) -> None:
             print("   --- End of decoder validation output ---")
 
 
-# Re-export main functions for backwards compatibility
-# These are now defined in video_test_framework_main.py
-# pylint: disable=wrong-import-position
-from tests.libs.video_test_framework_main import (  # noqa: E402
-    run_framework_main, run_complete_framework_main)
-
-__all__ = ['VulkanVideoTestFrameworkBase', 'run_framework_main',
-           'run_complete_framework_main']
+__all__ = ['VulkanVideoTestFrameworkBase']