|
| 1 | +# PyOsmo Roadmap |
| 2 | + |
| 3 | +## Overview |
| 4 | + |
| 5 | +This document tracks PyOsmo's development roadmap and feature comparison with the mature Java OSMO implementation (v3.7.1). It consolidates the project's priorities, current status, and planned improvements. |
| 6 | + |
| 7 | +**Current Version**: v0.2.2 |
| 8 | +**Python Support**: 3.11–3.14 |
| 9 | + |
| 10 | +--- |
| 11 | + |
| 12 | +## What's Been Done (v0.1.3 → v0.2.2) |
| 13 | + |
| 14 | +These items from the original roadmap are complete: |
| 15 | + |
| 16 | +- **Decorator-based model API**: `@step`, `@guard`, `@weight`, `@pre`, `@post`, `@requires` decorators |
| 17 | +- **Fluent configuration API**: Method chaining for Osmo setup (see `fluent_api_guide.md`) |
| 18 | +- **Inspect-based model introspection**: Modern model discovery replacing `dir()` introspection |
| 19 | +- **Plugin registry**: Extensible architecture for algorithms, end conditions, error strategies |
| 20 | +- **pytest integration**: `pytest_pyosmo` plugin for native pytest discovery |
| 21 | +- **Build modernization**: Consolidated to `pyproject.toml`, removed `setup.py` |
| 22 | +- **Development tooling**: Migrated to ruff (linting/formatting) and ty (type checking) |
| 23 | +- **CI/CD**: GitHub Actions on Python 3.11–3.14 with uv |
| 24 | +- **README**: Complete and accurate |
| 25 | +- **Publishing workflow**: Fixed for Python 3.11+ |
| 26 | + |
| 27 | +--- |
| 28 | + |
| 29 | +## Priority Classification |
| 30 | + |
| 31 | +- **P1 (High)**: Important for production readiness and adoption |
| 32 | +- **P2 (Medium)**: Quality improvements, nice-to-have features |
| 33 | +- **P3 (Low)**: Future enhancements, optimization opportunities |
| 34 | + |
| 35 | +--- |
| 36 | + |
| 37 | +## P1 — High Priority |
| 38 | + |
| 39 | +### Type Hints & Docstrings |
| 40 | + |
| 41 | +Improve type hint coverage on the public API and add Google-style docstrings to all public classes and methods. Run ty in CI for validation. |
| 42 | + |
| 43 | +### Requirements Traceability |
| 44 | + |
| 45 | +Port Java OSMO's requirement tracking system: |
| 46 | +- `@requires()` decorator for associating steps with requirements |
| 47 | +- `@requires_all()` / `@requires_any()` for AND/OR logic |
| 48 | +- Requirement coverage tracking in history |
| 49 | +- Requirement coverage end condition |
| 50 | + |
| 51 | +```python |
| 52 | +from pyosmo.decorators import requires |
| 53 | + |
| 54 | +class LoginModel: |
| 55 | + @requires("REQ-001") |
| 56 | + def step_login(self): |
| 57 | + pass |
| 58 | +``` |
| 59 | + |
| 60 | +### State & Step-Pair Coverage |
| 61 | + |
| 62 | +- `@state` decorator for marking state-defining methods |
| 63 | +- Track unique states visited during execution |
| 64 | +- Track step sequences (pairs) for transition coverage |
| 65 | +- End conditions based on state/step-pair coverage |
| 66 | + |
| 67 | +### Structured Reporting |
| 68 | + |
| 69 | +Replace basic console output with professional reports: |
| 70 | +- JSON export for data interchange |
| 71 | +- HTML reports (Jinja2-based) |
| 72 | +- JUnit XML for CI/CD integration |
| 73 | +- Markdown reports for documentation |
| 74 | + |
| 75 | +```python |
| 76 | +from pyosmo.reporting import HTMLReporter, JSONReporter |
| 77 | + |
| 78 | +HTMLReporter(osmo.history).save("report.html") |
| 79 | +JSONReporter(osmo.history).save("data.json") |
| 80 | +``` |
| 81 | + |
| 82 | +### Model Validation & Analysis |
| 83 | + |
| 84 | +Static model analysis before execution: |
| 85 | +- Validate model structure |
| 86 | +- Detect unreachable steps (always-false guards) |
| 87 | +- Detect always-enabled steps (missing guards) |
| 88 | +- CLI command: `pyosmo validate mymodel.py` |
| 89 | + |
| 90 | +### Tutorials & Documentation |
| 91 | + |
| 92 | +- Tutorial series: Getting Started, Data-Driven Testing, Scenario-Based Testing |
| 93 | +- Generated API reference documentation (Sphinx or pdoc) |
| 94 | +- Comprehensive user guide |
| 95 | + |
| 96 | +--- |
| 97 | + |
| 98 | +## P2 — Medium Priority |
| 99 | + |
| 100 | +### Variable Coverage |
| 101 | + |
| 102 | +Track data variation across test executions: |
| 103 | +- `@variable()` decorator with category-partition support |
| 104 | +- Variable coverage metrics and end conditions |
| 105 | + |
| 106 | +### Optimizer Algorithm |
| 107 | + |
| 108 | +Greedy step selection to accelerate coverage: |
| 109 | +- Select steps that cover uncovered requirements |
| 110 | +- Select steps that visit unvisited states |
| 111 | +- Reduce execution time to achieve coverage goals |
| 112 | + |
| 113 | +### Test Case Persistence |
| 114 | + |
| 115 | +Save and replay generated test sequences: |
| 116 | +- Save sequences to JSON |
| 117 | +- Replay saved sequences for regression |
| 118 | +- Test case minimization for failure reproduction |
| 119 | + |
| 120 | +### Performance Profiling |
| 121 | + |
| 122 | +Built-in execution profiling: |
| 123 | +- Step execution time tracking |
| 124 | +- Guard evaluation time tracking |
| 125 | +- Performance reports and CLI profiling mode |
| 126 | + |
| 127 | +### Enhanced Error Reporting |
| 128 | + |
| 129 | +Richer error context: |
| 130 | +- Step execution traceback capture |
| 131 | +- Model state snapshot on error |
| 132 | +- Error reproduction scripts |
| 133 | + |
| 134 | +--- |
| 135 | + |
| 136 | +## P3 — Future Enhancements |
| 137 | + |
| 138 | +- **Parallel execution**: Multi-process test generation |
| 139 | +- **Adaptive algorithms**: Runtime-adapting step selection |
| 140 | +- **Model visualization**: State machine graphs (GraphViz/Mermaid) |
| 141 | +- **Distributed testing**: Multi-machine coordination |
| 142 | +- **Constraint solving**: Z3/OR-Tools integration |
| 143 | +- **Probability-based end condition**: Random stopping with configurable probability |
| 144 | + |
| 145 | +--- |
| 146 | + |
| 147 | +## Feature Comparison: PyOsmo vs Java OSMO |
| 148 | + |
| 149 | +### Legend |
| 150 | + |
| 151 | +- ✅ Fully implemented |
| 152 | +- ⚠️ Partially implemented |
| 153 | +- ❌ Missing |
| 154 | +- 🔄 Different approach (language idioms) |
| 155 | + |
| 156 | +### Core Features |
| 157 | + |
| 158 | +| Feature | Java OSMO | PyOsmo | Status | |
| 159 | +|---------|-----------|--------|--------| |
| 160 | +| Step declaration | `@TestStep` annotation | `step_*` naming + `@step` decorator | ✅ | |
| 161 | +| Guard declaration | `@Guard` annotation | `guard_*` naming + `@guard` decorator | ✅ | |
| 162 | +| Weight declaration | `@Weight` annotation | `weight_*` naming + `@weight` decorator | ✅ | |
| 163 | +| Pre/post step hooks | `@Before`/`@After` | `@pre`/`@post` decorators | ✅ | |
| 164 | +| Suite/test lifecycle | `@BeforeSuite`, etc. | `before_suite()`, etc. | ✅ | |
| 165 | +| Multiple models | Model composition | `Osmo(model1, model2)` | ✅ | |
| 166 | +| Fluent config API | Builder pattern | Method chaining | ✅ | |
| 167 | + |
| 168 | +### Algorithms |
| 169 | + |
| 170 | +| Algorithm | Java OSMO | PyOsmo | Status | |
| 171 | +|-----------|-----------|--------|--------| |
| 172 | +| Random | ✅ | ✅ RandomAlgorithm | ✅ | |
| 173 | +| Balancing | ✅ | ✅ BalancingAlgorithm | ✅ | |
| 174 | +| Weighted Random | ✅ | ✅ WeightedAlgorithm | ✅ | |
| 175 | +| Weighted Balancing | ✅ | ✅ WeightedBalancingAlgorithm | ✅ | |
| 176 | +| Optimizer | ✅ | ❌ | P2 | |
| 177 | +| Custom algorithms | ✅ Extensible | ✅ Extensible | ✅ | |
| 178 | + |
| 179 | +### End Conditions |
| 180 | + |
| 181 | +| Condition | Java OSMO | PyOsmo | Status | |
| 182 | +|-----------|-----------|--------|--------| |
| 183 | +| Length-based | ✅ | ✅ Length | ✅ | |
| 184 | +| Time-based | ✅ | ✅ Time | ✅ | |
| 185 | +| Step coverage | ✅ | ✅ StepCoverage | ✅ | |
| 186 | +| Logical combinators | ✅ And/Or | ✅ And/Or | ✅ | |
| 187 | +| Endless | ✅ | ✅ Endless | ✅ | |
| 188 | +| Requirement coverage | ✅ | ❌ | P1 | |
| 189 | +| State coverage | ✅ | ❌ | P1 | |
| 190 | +| Probability-based | ✅ | ❌ | P3 | |
| 191 | + |
| 192 | +### Error Handling |
| 193 | + |
| 194 | +| Strategy | Java OSMO | PyOsmo | Status | |
| 195 | +|----------|-----------|--------|--------| |
| 196 | +| Always raise | ✅ | ✅ AlwaysRaise | ✅ | |
| 197 | +| Always ignore | ✅ | ✅ AlwaysIgnore | ✅ | |
| 198 | +| Allow count | ✅ | ✅ AllowCount | ✅ | |
| 199 | +| Ignore asserts | ⚠️ | ✅ IgnoreAsserts | 🔄 | |
| 200 | +| Custom strategies | ✅ | ✅ Extensible | ✅ | |
| 201 | + |
| 202 | +### Coverage & Traceability |
| 203 | + |
| 204 | +| Feature | Java OSMO | PyOsmo | Status | |
| 205 | +|---------|-----------|--------|--------| |
| 206 | +| Step coverage | ✅ | ✅ | ✅ | |
| 207 | +| Requirements tracking | ✅ | ❌ | P1 | |
| 208 | +| State coverage | ✅ | ❌ | P1 | |
| 209 | +| Step-pair coverage | ✅ | ❌ | P1 | |
| 210 | +| Variable coverage | ✅ | ❌ | P2 | |
| 211 | + |
| 212 | +### Reporting & Output |
| 213 | + |
| 214 | +| Feature | Java OSMO | PyOsmo | Status | |
| 215 | +|---------|-----------|--------|--------| |
| 216 | +| Console output | ✅ | ✅ | ✅ | |
| 217 | +| Step statistics | ✅ | ✅ | ✅ | |
| 218 | +| HTML reports | ✅ | ❌ | P1 | |
| 219 | +| JSON export | ⚠️ | ❌ | P1 | |
| 220 | +| JUnit XML | ✅ | ❌ | P1 | |
| 221 | +| Test persistence | ✅ | ❌ | P2 | |
| 222 | + |
| 223 | +### Integration |
| 224 | + |
| 225 | +| Feature | Java OSMO | PyOsmo | Status | |
| 226 | +|---------|-----------|--------|--------| |
| 227 | +| Test framework | JUnit | pytest | ✅ (language-specific) | |
| 228 | +| Package manager | Maven | pip/PyPI | ✅ (language-specific) | |
| 229 | +| CI/CD templates | ⚠️ | ✅ GitHub Actions | ✅ | |
| 230 | + |
| 231 | +### Summary by Status |
| 232 | + |
| 233 | +| Status | Count | |
| 234 | +|--------|-------| |
| 235 | +| ✅ Fully implemented | ~28 | |
| 236 | +| ⚠️ Partial / 🔄 Different | ~5 | |
| 237 | +| ❌ Missing (planned) | ~15 | |
| 238 | + |
| 239 | +--- |
| 240 | + |
| 241 | +## PyOsmo's Competitive Advantages |
| 242 | + |
| 243 | +1. **Language simplicity**: Python is easier to learn than Java |
| 244 | +2. **Rich ecosystem**: Integrates naturally with pytest, Selenium, Playwright, requests |
| 245 | +3. **Interactive development**: Works in Jupyter notebooks and IPython |
| 246 | +4. **Modern tooling**: pip/uv, ruff, ty — fast and ergonomic |
| 247 | +5. **Flexible modeling**: Duck typing + decorators for model definition |
| 248 | +6. **Pure Python**: No graphical tools or DSLs — just code |
| 249 | + |
| 250 | +--- |
| 251 | + |
| 252 | +*Last updated: 2026-02-14* |
0 commit comments