[optimize]Optimized LLMPerf Test Cases (#634)

Description:
This PR introduces several improvements to the LLMPerf test suite based
on real-world usage feedback and aims to simplify maintenance and
configuration:
Optimized LLMPerf Test Cases
Updated and refined existing test cases according to actual usage
patterns and performance testing scenarios to better reflect real-world
conditions.
Simplified Parameter Capture
Refactored the parameter capture logic to reduce code duplication and
overall complexity, improving readability and maintainability.
Added extra_info Configuration Option
Introduced a new extra_info field in the configuration to allow users to
annotate or tag service launch parameters (e.g., model name, hardware
specs, or custom flags), facilitating better traceability and debugging.
Updated Documentation for Pytest Integration
Revised and expanded the documentation to properly reflect how to run
LLMPerf tests under the pytest framework, including examples and best
practices.
These changes collectively enhance test robustness, reduce boilerplate,
and improve the developer experience when integrating LLMPerf into
automated testing pipelines.

Author: Potterluo

test/README.md

@@ -1,179 +1,236 @@
-# Pytest
-[简体中文](README_zh.md)
-A comprehensive Pytest testing framework featuring configuration management, database integration, performance testing, and HTML report generation.
+# Pytest Automation Testing Framework Guide
+[English](README.md) | [简体中文](README_zh.md)
 
-## 📋 Features
+This guide introduces an automation testing framework built on **pytest 7.0+**, integrating core capabilities such as **configuration management**, **database integration**, **performance testing**, and **HTML report generation**. It is suitable for unit testing, functional testing, and end-to-end (E2E) testing scenarios.
 
-- **Modern Testing Framework**: Complete test solution built on Pytest 7.0+
-- **Configuration Management**: YAML-based config with thread-safe singleton pattern
-- **Database Integration**: Built-in MySQL support with automatic result storage
-- **HTML Reports**: Auto-generated pytest HTML test reports
-- **Tagging System**: Multi-dimensional test tags (stage, feature, platform, etc.)
+---
 
-## 🗂️ Project Structure
+## 📋 Core Framework Features
 
-```
+- **Modern Testing Architecture**: Built on pytest 7.0+, compatible with Python 3.11+, and supports a rich plugin ecosystem.
+- **Centralized Configuration Management**: Thread-safe singleton-pattern configuration loading via YAML files.
+- **Database Integration**: Built-in PostgreSQL support for automatically persisting test results to a database; if no database is configured, results are saved locally.
+- **Visual Test Reporting**: Integrated with the pytest-html plugin to auto-generate clear and comprehensive HTML test reports.
+- **Multi-dimensional Test Tagging**: Supports categorizing and filtering test cases by dimensions such as test stage, feature module, and execution platform.
+
+---
+
+## 🗂️ Project Directory Structure
+
+```text
 pytest_demo/
-├── common/                          # Common modules
+├── common/                          # Shared utility modules
 │   ├── __init__.py
-│   ├── config_utils.py              # Configuration utilities
-│   ├── db_utils.py                  # Database utilities
-│   └── capture_utils                # Return-value capture utilities
-├── results/                         # Result storage folder
-├── suites/                          # Test suites
-│   ├── UnitTest                     # Unit tests
-│   ├── Feature                      # Feature tests
+│   ├── config_utils.py              # Configuration loading and management
+│   ├── db_utils.py                  # Database operation utilities
+│   └── capture_utils.py             # Test data capture and export utilities
+├── results/                         # Output directory for test results and logs
+├── suites/                          # Test suite directory
+│   ├── UnitTest/                    # Unit tests
+│   ├── Feature/                     # Functional tests
 │   └── E2E/                         # End-to-end tests
-│       └── test_demo_performance.py # Sample test file
-├── config.yaml                      # Main config file
-├── conftest.py                      # Pytest config
-├── pytest.ini                       # Pytest settings
-├── requirements.txt                 # Dependencies
-└── README.md                        # This doc (CN)
+├── config.yaml                      # Main configuration file (YAML format)
+├── conftest.py                      # Shared pytest configuration and fixture definitions
+├── pytest.ini                       # pytest runtime parameter configuration
+├── requirements.txt                 # Project dependencies
+└── README.md                        # Project documentation (this document)
 ```
 
+---
+
 ## 🚀 Quick Start
 
-### Prerequisites
+### Environment Requirements
+
+- Python 3.11 or higher
 
-- Python 3.8+
-- MySQL 5.7+ (optional, for DB features)
-- Git
+### Installation and Setup
 
-### Installation
+1. **Install Dependencies**
 
-1. **Install dependencies**
    ```bash
    pip install -r requirements.txt
    ```
 
-2. **Configure database** (optional)
+2. **(Optional) Configure Database**
+
+   Edit the database section in `config.yaml`:
 
-   Edit `config.yaml`:
    ```yaml
-    database:
-      backup: "results/"
-      host: "127.0.0.1"
-      port: 3306
-      name: "ucm_pytest"
-      user: "root"
-      password: "123456"
-      charset: "utf8mb4"
+   database:
+     enabled: true                     # Enable database result storage (set to false to save to local directory)
+     backup: "results/"
+     host: "127.0.0.1"
+     port: 5432
+     name: "ucm_test"
+     user: "postgres"
+     password: "123456"
    ```
 
-3. **Run tests**
+3. **Run Tests**
+
    ```bash
+   # Navigate to the project root directory
+   cd /test
+
    # Run all tests
    pytest
 
-   # Run tests by tag
-   pytest --stage=1
-   pytest --feature=performance
+   # Run selected tests by markers (examples)
+   pytest --stage=0                      # Run unit tests only
+   pytest --feature=test_uc_performance  # Run a specific feature module
    ```
 
-## ⚙️ Configuration
+---
+
+## ⚙️ Development Guidelines and Best Practices
+
+### 1. Test Case Organization Conventions
+
+All test cases must reside under the `suites/` directory and follow these naming conventions:
 
-### config.yaml
+- **File Names**: Must start with `test_` (e.g., `test_login.py`)
+- **Class Names**: Must start with `Test` (e.g., `TestClassA`)
+- **Function Names**: Must start with `test_` (e.g., `test_valid_user_login`)
 
-Full YAML-based config. Key sections:
+> pytest configuration (`pytest.ini`) has pre-set these discovery rules:
+>
+> ```ini
+> testpaths = suites
+> python_files = test_*.py
+> python_classes = Test*
+> python_functions = test_*
+> ```
 
-- **reports**: Report settings (HTML, timestamp, etc.)
-- **database**: MySQL connection details
+---
 
-## 🧪 Test Examples
+### 2. Multi-dimensional Marker System
 
-### Basic functional test
+The framework supports the following marker types:
+
+| Marker Type | Value Description |
+|------------|-------------------|
+| `stage`    | `0`=Unit Test, `1`=Smoke Test, `2`=Regression Test, `3`=Release Test |
+| `feature`  | Feature module identifier (e.g., `"uc_performance"`) |
+| `platform` | Execution platform (e.g., `"Ascend"`, `"CUDA"`) |
+
+**Usage Example:**
 
 ```python
-# suites/E2E/test_demo_performance.py
 import pytest
 
-@pytest.fixture(scope="module", name="calc")
-def calculator():
-    return Calculator()
+@pytest.mark.stage(0)
+@pytest.mark.feature("uc_unit_test")
+@pytest.mark.platform("Ascend")
+def test_unit_case():
+    assert True
+```
 
-@pytest.mark.feature("mark")
-class TestCalculator:
-    def test_add(self, calc):
-        assert calc.add(1, 2) == 3
+**Run tests with specific markers:**
 
-    def test_divide_by_zero(self, calc):
-        with pytest.raises(ZeroDivisionError):
-            calc.divide(6, 0)
+```bash
+pytest --stage=0 --feature=capture_demo
 ```
 
-## 🏷️ Tagging System
+---
 
-Multi-dimensional tags supported:
+### 3. Configuration and Parameter Management
 
-### Stage tags
-- `stage(0)`: Unit tests
-- `stage(1)`: Smoke tests
-- `stage(2)`: Regression tests
-- `stage(3)`: Release tests
+- **Static configurations** (e.g., database connections, API endpoints) should be maintained in `config.yaml` and loaded via `config_utils`:
 
-### Functional tags
-- `feature`: Module tag
-- `platform`: Platform tag (GPU/NPU)
+  ```python
+  from common.config_utils import config_utils
 
-### Usage
+  db_config = config_utils.get_config("database")
+  api_url = config_utils.get_nested_config("easyPerf.api.url")
+  ```
 
-```bash
-# Run smoke tests and above
-pytest --stage=1+
+- **Dynamic test parameters** (e.g., input length, concurrency count) should use `@pytest.mark.parametrize`:
 
-# Run by feature
-pytest --feature=performance
-pytest --feature=performance,reliability
+  ```python
+  perf_scenarios = [
+      (4000, 1024, 80),
+      (2000, 512, 40)
+  ]
+  scenario_ids = [f"in_{s[0]}-out_{s[1]}-con_{s[2]}" for s in perf_scenarios]
 
-# Run by platform
-pytest --platform=gpu
-```
+  @pytest.mark.feature("uc_performance_test")
+  @pytest.mark.parametrize("in_tokens, out_tokens, concurrent", perf_scenarios, ids=scenario_ids)
+  def test_performance(in_tokens, out_tokens, concurrent):
+      res = run_case(in_tokens, out_tokens, concurrent)
+      assert res is not None
+  ```
+
+---
 
-### HTML Reports
+### 4. Automatic Test Data Collection and Export
 
-Auto-generated timestamped HTML reports:
-- Location: `reports/pytest_YYYYMMDD_HHMMSS/report.html`
-- Detailed results, errors, timing
-- Customizable title & style
+The framework supports automatic capture and persistence of test result data using the `@export_vars` decorator.
 
-### Database Storage
+**Usage Requirements:**
+- The decorator must be the innermost decorator (closest to the function definition).
+- The function’s return value must be a dictionary containing at least one of the following fields:
+  - `_name`: Target database table name (**required**)
+  - `_data`: A dictionary or list of dictionaries (key-value pairs become table columns)
+  - `_proj`: A list of dictionaries (for structured projection data)
 
-If enabled, results are auto-saved to MySQL.  
-To add new record types, ask DB admin to create tables; otherwise only local files are used.
+**Examples:**
 
-Example:
 ```python
-@pytest.mark.feature("capture")  # Must be top decorator
+from common.capture_utils import export_vars
+import pytest
+
+@pytest.mark.feature("capture_demo")
+@export_vars
+def test_capture_scalar():
+    return {"_name": "demo_case", "_data": {"accuracy": 0.95, "loss": 0.05}}
+
+@pytest.mark.feature("capture_demo")
 @export_vars
-def test_capture_mix():
-    assert 1 == 1
+def test_capture_list():
+    return {"_name": "demo_case", "_data": {"accuracy": [0.9, 0.95], "loss": [0.1, 0.05]}}
+
+@pytest.mark.feature("demo")
+@export_vars
+def test_proj_data():
     return {
-        '_name': 'demo',
-        '_data': {
-            'length': 10086,            # single value
-            'accuracy': [0.1, 0.2, 0.3], # list
-            'loss': [0.1, 0.2, 0.3],     # list
-        }
+        "_name": "demo_case",
+        "_proj": [
+            {"accuracy": 0.88, "loss": 0.12},
+            {"accuracy": 0.92, "loss": 0.08}
+        ]
     }
 ```
 
-### Config Access
+> Data will be automatically written to the database or a local file based on the `database.enabled` setting in `config.yaml`.
+
+---
+
+### 5. Fixture Usage Guidelines
+
+`@pytest.fixture` is used to provide reusable test dependencies (e.g., database connections, service instances).
+
+**Example:**
 
-Read settings easily:
 ```python
-from common.config_utils import config_utils
-# Get config
-db_config = config_utils.get_config("database")
-api_config = config_utils.get_nested_config("easyPerf.api")
-```
+import pytest
+
+class Calculator:
+    def add(self, a, b): return a + b
+    def divide(self, a, b): return a / b
 
-## 🛠️ Development Guide
+@pytest.fixture(scope="module")
+def calc():
+    return Calculator()
+
+@pytest.mark.feature("calculator")
+class TestCalculator:
+    def test_add(self, calc):
+        assert calc.add(1, 2) == 3
 
-### Adding New Tests
+    def test_divide_by_zero(self, calc):
+        with pytest.raises(ZeroDivisionError):
+            calc.divide(6, 0)
+```
 
-1. Create test files under `suites/` categories
-2. Apply appropriate tags
-3. Naming: `test_*.py`
-4. Use fixtures & marks for data management
-5. Keep custom marks concise and aligned with overall goals
+> **Tip**: Fixture scope (`scope`) can be set to `function` (default), `class`, `module`, or `session` to optimize resource reuse efficiency.