|
| 1 | +# Development Guide |
| 2 | + |
| 3 | +This guide covers the development setup, tools, and practices for OVMobileBench. |
| 4 | + |
| 5 | +## Development Setup |
| 6 | + |
| 7 | +### Prerequisites |
| 8 | + |
| 9 | +- Python 3.11+ |
| 10 | +- Git |
| 11 | +- Android SDK/NDK (for Android development) |
| 12 | +- Docker (optional, for containerized testing) |
| 13 | + |
| 14 | +### Installation |
| 15 | + |
| 16 | +1. **Clone the repository** |
| 17 | + |
| 18 | + ```bash |
| 19 | + git clone https://github.com/embedded-dev-research/OVMobileBench |
| 20 | + cd OVMobileBench |
| 21 | + ``` |
| 22 | + |
| 23 | +2. **Create virtual environment** |
| 24 | + |
| 25 | + ```bash |
| 26 | + python -m venv .venv |
| 27 | + source .venv/bin/activate # On Windows: .venv\Scripts\activate |
| 28 | + ``` |
| 29 | + |
| 30 | +3. **Install dependencies** |
| 31 | + |
| 32 | + ```bash |
| 33 | + pip install -r requirements.txt |
| 34 | + pip install -e . # Install in editable mode |
| 35 | + ``` |
| 36 | + |
| 37 | +4. **Install pre-commit hooks** |
| 38 | + |
| 39 | + ```bash |
| 40 | + pre-commit install |
| 41 | + ``` |
| 42 | + |
| 43 | +## Code Quality Tools |
| 44 | + |
| 45 | +### Pre-commit Hooks |
| 46 | + |
| 47 | +We use pre-commit to ensure code quality. It runs automatically on `git commit`. |
| 48 | + |
| 49 | +**Included checks:** |
| 50 | + |
| 51 | +- **pyupgrade** - Modernizes Python syntax to 3.11+ |
| 52 | +- **black** - Code formatting (100 char line length) |
| 53 | +- **ruff** - Fast Python linter (replaces flake8, isort, and more) |
| 54 | +- **mypy** - Static type checking |
| 55 | +- **isort** - Import sorting |
| 56 | +- **yamllint** - YAML file linting |
| 57 | +- **markdownlint** - Markdown file linting |
| 58 | +- **codespell** - Spell checking |
| 59 | +- **commitizen** - Commit message validation |
| 60 | + |
| 61 | +### Running Checks Manually |
| 62 | + |
| 63 | +```bash |
| 64 | +# Run all checks |
| 65 | +pre-commit run --all-files |
| 66 | + |
| 67 | +# Run specific check |
| 68 | +pre-commit run black --all-files |
| 69 | +pre-commit run ruff --all-files |
| 70 | + |
| 71 | +# Auto-fix issues |
| 72 | +pre-commit run --all-files --hook-stage manual |
| 73 | + |
| 74 | +# Run without pre-commit |
| 75 | +black ovmobilebench tests |
| 76 | +ruff check ovmobilebench tests |
| 77 | +mypy ovmobilebench --ignore-missing-imports |
| 78 | +pytest tests/ |
| 79 | +``` |
| 80 | + |
| 81 | +### Bypassing Checks (Emergency Only) |
| 82 | + |
| 83 | +```bash |
| 84 | +git commit --no-verify -m "Emergency fix" |
| 85 | +``` |
| 86 | + |
| 87 | +## Testing |
| 88 | + |
| 89 | +### Running Tests |
| 90 | + |
| 91 | +```bash |
| 92 | +# Run all tests |
| 93 | +pytest tests/ |
| 94 | + |
| 95 | +# Run with coverage |
| 96 | +pytest tests/ --cov=ovmobilebench --cov-report=html |
| 97 | + |
| 98 | +# Run specific test file |
| 99 | +pytest tests/test_config.py |
| 100 | + |
| 101 | +# Run specific test |
| 102 | +pytest tests/test_config.py::TestConfigLoader::test_load_experiment |
| 103 | + |
| 104 | +# Run with verbose output |
| 105 | +pytest tests/ -v |
| 106 | + |
| 107 | +# Run in parallel |
| 108 | +pytest tests/ -n auto |
| 109 | +``` |
| 110 | + |
| 111 | +### Test Structure |
| 112 | + |
| 113 | +``` |
| 114 | +tests/ |
| 115 | +├── android/ |
| 116 | +│ └── installer/ # Android installer tests |
| 117 | +├── test_*.py # Unit tests for each module |
| 118 | +├── conftest.py # Shared fixtures |
| 119 | +└── data/ # Test data files |
| 120 | +``` |
| 121 | + |
| 122 | +### Writing Tests |
| 123 | + |
| 124 | +1. Use pytest fixtures for reusable test data |
| 125 | +2. Mock external dependencies (ADB, SSH, file system) |
| 126 | +3. Aim for >70% test coverage |
| 127 | +4. Test both success and failure cases |
| 128 | + |
| 129 | +Example test: |
| 130 | + |
| 131 | +```python |
| 132 | +import pytest |
| 133 | +from unittest.mock import Mock, patch |
| 134 | + |
| 135 | +def test_feature(): |
| 136 | + # Arrange |
| 137 | + mock_device = Mock() |
| 138 | + mock_device.shell.return_value = "output" |
| 139 | + |
| 140 | + # Act |
| 141 | + result = process_device(mock_device) |
| 142 | + |
| 143 | + # Assert |
| 144 | + assert result == "expected" |
| 145 | + mock_device.shell.assert_called_once_with("command") |
| 146 | +``` |
| 147 | + |
| 148 | +## CI/CD Pipeline |
| 149 | + |
| 150 | +### GitHub Actions Workflow |
| 151 | + |
| 152 | +The CI pipeline runs on every push and pull request: |
| 153 | + |
| 154 | +1. **Pre-commit Stage** (Ubuntu only) |
| 155 | + - Runs all linting and formatting checks |
| 156 | + - Fastest stage, fails early on style issues |
| 157 | + |
| 158 | +2. **Test Stage** (Ubuntu, macOS, Windows) |
| 159 | + - Runs unit tests on all platforms |
| 160 | + - Generates coverage reports |
| 161 | + |
| 162 | +3. **Build Stage** (All platforms) |
| 163 | + - Verifies package building |
| 164 | + - Tests installation process |
| 165 | + |
| 166 | +4. **Validation Stage** (All platforms) |
| 167 | + - Validates configuration schemas |
| 168 | + - Tests CLI commands |
| 169 | + |
| 170 | +5. **Device Tests** (When available) |
| 171 | + - Integration tests with real/emulated devices |
| 172 | + - Android emulator tests |
| 173 | + |
| 174 | +### Local CI Testing |
| 175 | + |
| 176 | +Test CI locally using act: |
| 177 | + |
| 178 | +```bash |
| 179 | +# Install act |
| 180 | +brew install act # macOS |
| 181 | +# or download from https://github.com/nektos/act |
| 182 | + |
| 183 | +# Run CI locally |
| 184 | +act push |
| 185 | +``` |
| 186 | + |
| 187 | +## Code Style Guidelines |
| 188 | + |
| 189 | +### Python |
| 190 | + |
| 191 | +- Use type hints for all function parameters and returns |
| 192 | +- Write docstrings for all public functions/classes |
| 193 | +- Follow PEP 8 with exceptions defined in `.ruff.toml` |
| 194 | +- Maximum line length: 100 characters |
| 195 | +- Use f-strings for string formatting |
| 196 | +- Prefer pathlib over os.path |
| 197 | + |
| 198 | +### Documentation |
| 199 | + |
| 200 | +- Use Markdown for all documentation |
| 201 | +- Include code examples in docstrings |
| 202 | +- Keep README.md updated with user-facing changes |
| 203 | +- Update ARCHITECTURE.md for design changes |
| 204 | + |
| 205 | +### Commit Messages |
| 206 | + |
| 207 | +Follow [Conventional Commits](https://www.conventionalcommits.org/): |
| 208 | + |
| 209 | +``` |
| 210 | +<type>(<scope>): <subject> |
| 211 | +
|
| 212 | +<body> |
| 213 | +
|
| 214 | +<footer> |
| 215 | +``` |
| 216 | + |
| 217 | +Types: |
| 218 | + |
| 219 | +- `feat`: New feature |
| 220 | +- `fix`: Bug fix |
| 221 | +- `docs`: Documentation changes |
| 222 | +- `style`: Code style changes (formatting, etc.) |
| 223 | +- `refactor`: Code refactoring |
| 224 | +- `perf`: Performance improvements |
| 225 | +- `test`: Test changes |
| 226 | +- `build`: Build system changes |
| 227 | +- `ci`: CI/CD changes |
| 228 | +- `chore`: Other changes |
| 229 | + |
| 230 | +Examples: |
| 231 | + |
| 232 | +```bash |
| 233 | +git commit -m "feat(android): add AVD creation support" |
| 234 | +git commit -m "fix(runner): handle timeout correctly" |
| 235 | +git commit -m "docs: update installation guide" |
| 236 | +``` |
| 237 | + |
| 238 | +## Debugging |
| 239 | + |
| 240 | +### VS Code Configuration |
| 241 | + |
| 242 | +`.vscode/launch.json`: |
| 243 | + |
| 244 | +```json |
| 245 | +{ |
| 246 | + "version": "0.2.0", |
| 247 | + "configurations": [ |
| 248 | + { |
| 249 | + "name": "Debug CLI", |
| 250 | + "type": "python", |
| 251 | + "request": "launch", |
| 252 | + "module": "ovmobilebench", |
| 253 | + "args": ["build", "-c", "experiments/android_example.yaml"], |
| 254 | + "console": "integratedTerminal" |
| 255 | + } |
| 256 | + ] |
| 257 | +} |
| 258 | +``` |
| 259 | + |
| 260 | +### PyCharm Configuration |
| 261 | + |
| 262 | +1. Create new Python configuration |
| 263 | +2. Set module: `ovmobilebench` |
| 264 | +3. Set parameters: `build -c experiments/android_example.yaml` |
| 265 | +4. Set working directory: project root |
| 266 | + |
| 267 | +### Common Issues |
| 268 | + |
| 269 | +**Import Errors** |
| 270 | + |
| 271 | +```bash |
| 272 | +# Reinstall in editable mode |
| 273 | +pip install -e . |
| 274 | +``` |
| 275 | + |
| 276 | +**Type Checking Errors** |
| 277 | + |
| 278 | +```bash |
| 279 | +# Install type stubs |
| 280 | +pip install types-PyYAML types-paramiko |
| 281 | +``` |
| 282 | + |
| 283 | +**Pre-commit Failures** |
| 284 | + |
| 285 | +```bash |
| 286 | +# Update pre-commit hooks |
| 287 | +pre-commit autoupdate |
| 288 | + |
| 289 | +# Clean and reinstall |
| 290 | +pre-commit clean |
| 291 | +pre-commit install |
| 292 | +``` |
| 293 | + |
| 294 | +## Release Process |
| 295 | + |
| 296 | +1. Update version in `pyproject.toml` |
| 297 | +2. Update CHANGELOG.md |
| 298 | +3. Create release branch: `git checkout -b release/v1.2.3` |
| 299 | +4. Run full test suite: `pytest tests/` |
| 300 | +5. Create PR to main branch |
| 301 | +6. After merge, tag release: `git tag v1.2.3` |
| 302 | +7. Push tag: `git push origin v1.2.3` |
| 303 | +8. GitHub Actions will automatically publish to PyPI |
| 304 | + |
| 305 | +## Additional Resources |
| 306 | + |
| 307 | +- [Python Type Hints](https://docs.python.org/3/library/typing.html) |
| 308 | +- [Pytest Documentation](https://docs.pytest.org/) |
| 309 | +- [Black Code Style](https://black.readthedocs.io/) |
| 310 | +- [Ruff Rules](https://docs.astral.sh/ruff/rules/) |
| 311 | +- [Pre-commit Hooks](https://pre-commit.com/) |
0 commit comments