|
| 1 | +# Lifecycle Feature Integration Tests |
| 2 | + |
| 3 | +This document describes the lifecycle feature integration tests implemented in this repository, including the approach taken and the rationale behind design decisions. |
| 4 | + |
| 5 | +## Overview |
| 6 | + |
| 7 | +Lifecycle feature integration tests verify that applications can properly integrate with the S-CORE lifecycle management framework. These tests validate three key capabilities that the Launch Manager provides: |
| 8 | + |
| 9 | +1. **Process Launching Support** — Applications can report execution state to the Launch Manager |
| 10 | +2. **Dependency Ordering** — Applications can use sequential health monitoring for ordered initialization |
| 11 | +3. **Parallel Launching** — Multiple independent health monitors can run concurrently |
| 12 | + |
| 13 | +## Requirements Tested |
| 14 | + |
| 15 | +The lifecycle FIT tests partially verify the following requirements: |
| 16 | + |
| 17 | +- **feat_req__lifecycle__launch_support** — The Launch Manager shall provide support for launching Processes |
| 18 | +- **feat_req__lifecycle__dependency_ordering** — The Launch Manager shall provide support for ordering the launching of Processes based on the dependencies |
| 19 | +- **feat_req__lifecycle__parallel_launching** — The Launch Manager shall provide support for launching Processes in parallel |
| 20 | + |
| 21 | +## Implementation Approach |
| 22 | + |
| 23 | +### 1. Real API Integration vs. Simulation |
| 24 | + |
| 25 | +**Decision**: Use real lifecycle and health monitoring APIs from `@score_lifecycle_health` |
| 26 | + |
| 27 | +**Rationale**: |
| 28 | +- Provides realistic demonstration of how applications integrate with lifecycle services |
| 29 | +- Tests actual API surface that application developers will use |
| 30 | +- Validates API availability and correctness |
| 31 | +- Enables detection of breaking API changes early |
| 32 | + |
| 33 | +**Graceful Degradation**: |
| 34 | +The tests are designed to run in environments without the Launch Manager daemon. When the daemon is not available: |
| 35 | +- Lifecycle client calls return `false` (C++) or `false` (Rust) instead of panicking |
| 36 | +- Tests log informational messages explaining the demonstration nature |
| 37 | +- Tests still validate API signatures and integration patterns |
| 38 | +- Tests pass successfully, confirming proper integration code |
| 39 | + |
| 40 | +### 2. Dual Language Implementation (Rust + C++) |
| 41 | + |
| 42 | +**Decision**: Implement all scenarios in both Rust and C++ |
| 43 | + |
| 44 | +**Rationale**: |
| 45 | +- **Language Parity**: Ensures both language ecosystems have equal support |
| 46 | +- **API Verification**: Confirms lifecycle APIs work correctly in both languages |
| 47 | +- **Documentation by Example**: Provides working examples for developers in both languages |
| 48 | +- **Comprehensive Coverage**: Detects language-specific integration issues |
| 49 | + |
| 50 | +**Implementation Details**: |
| 51 | + |
| 52 | +| Aspect | Rust Implementation | C++ Implementation | |
| 53 | +|--------|--------------------|--------------------| |
| 54 | +| **Lifecycle Client** | `lifecycle_client_rs::report_execution_state_running()` | `score::lcm::LifecycleClient{}.ReportExecutionState()` | |
| 55 | +| **Health Monitoring** | `health_monitoring_lib::HealthMonitorBuilder` | Demonstrates API patterns with logging | |
| 56 | +| **Logging** | Structured JSON logs via `tracing::info!()` | Plain text via `std::cout` | |
| 57 | +| **Dependencies** | `@score_lifecycle_health//...rust_bindings:lifecycle_client_rs`<br/>`@score_lifecycle_health//src/health_monitoring_lib` | `@score_lifecycle_health//...lifecycle_client_lib:lifecycle_client` | |
| 58 | + |
| 59 | +### 3. Test Architecture |
| 60 | + |
| 61 | +**Three-Layer Design**: |
| 62 | + |
| 63 | +``` |
| 64 | +┌─────────────────────────────────────────────────────────┐ |
| 65 | +│ Python Test Layer (test_cases/tests/lifecycle/) │ |
| 66 | +│ - Orchestrates test execution │ |
| 67 | +│ - Validates test results │ |
| 68 | +│ - Provides parametrization (rust/cpp) │ |
| 69 | +└─────────────────────────────────────────────────────────┘ |
| 70 | + │ |
| 71 | + ▼ |
| 72 | +┌─────────────────────────────────────────────────────────┐ |
| 73 | +│ Scenario Base (test_cases/lifecycle_scenario.py) │ |
| 74 | +│ - Provides shared fixtures (temp_dir) │ |
| 75 | +│ - Utility functions for config management │ |
| 76 | +└─────────────────────────────────────────────────────────┘ |
| 77 | + │ |
| 78 | + ▼ |
| 79 | +┌──────────────────────────┬──────────────────────────────┐ |
| 80 | +│ Rust Scenarios │ C++ Scenarios │ |
| 81 | +│ (test_scenarios/rust/) │ (test_scenarios/cpp/) │ |
| 82 | +│ - Real API calls │ - Real API calls │ |
| 83 | +│ - Structured logging │ - Plain text logging │ |
| 84 | +└──────────────────────────┴──────────────────────────────┘ |
| 85 | +``` |
| 86 | + |
| 87 | +**Rationale**: |
| 88 | +- **Separation of Concerns**: Python for orchestration, Rust/C++ for implementation |
| 89 | +- **Language-Appropriate Testing**: Leverages pytest for test management |
| 90 | +- **Reusability**: Base scenario class reduces code duplication |
| 91 | +- **Flexibility**: Easy to add new test scenarios or languages |
| 92 | + |
| 93 | +### 4. Test Validation Strategy |
| 94 | + |
| 95 | +**Decision**: Use different validation approaches for Rust vs C++ |
| 96 | + |
| 97 | +**Rust Tests**: |
| 98 | +- Parse structured JSON logs from `tracing` framework |
| 99 | +- Use `LogContainer` to query specific log messages |
| 100 | +- Validate presence and content of structured log fields |
| 101 | + |
| 102 | +**C++ Tests**: |
| 103 | +- Parse plain text stdout output |
| 104 | +- Use string matching to validate key messages |
| 105 | +- Simpler logging reduces dependencies |
| 106 | + |
| 107 | +**Rationale**: |
| 108 | +- **Rust**: Tracing infrastructure already available, provides rich structured logging |
| 109 | +- **C++**: Minimizes dependencies, uses `std::cout` for simplicity |
| 110 | +- **Pragmatic**: Both approaches validate the same behavior effectively |
| 111 | + |
| 112 | +## Test Scenarios |
| 113 | + |
| 114 | +### 1. Process Launching Support |
| 115 | + |
| 116 | +**Purpose**: Verify that applications can report execution state to Launch Manager |
| 117 | + |
| 118 | +**Implementation**: |
| 119 | +- Calls `report_execution_state_running()` or `ReportExecutionState(kRunning)` |
| 120 | +- Logs success or graceful degradation message |
| 121 | +- Simulates application work with configurable duration |
| 122 | + |
| 123 | +**Validation**: |
| 124 | +- Confirms API was called successfully |
| 125 | +- Verifies application completed without errors |
| 126 | +- Checks for appropriate log messages |
| 127 | + |
| 128 | +**File Locations**: |
| 129 | +- Rust: `test_scenarios/rust/src/scenarios/lifecycle/launch_manager_support.rs` |
| 130 | +- C++: `test_scenarios/cpp/src/scenarios/lifecycle/launch_manager_support.cpp` |
| 131 | +- Test: `test_cases/tests/lifecycle/test_process_launching.py` |
| 132 | + |
| 133 | +### 2. Dependency Ordering |
| 134 | + |
| 135 | +**Purpose**: Demonstrate sequential initialization patterns for dependency-ordered supervision |
| 136 | + |
| 137 | +**Implementation**: |
| 138 | +- Rust: Creates health monitor with sequential deadline monitors |
| 139 | +- C++: Demonstrates sequential checkpoint reporting pattern |
| 140 | +- Reports checkpoints in sequence: `init_step_0`, `init_step_1`, etc. |
| 141 | + |
| 142 | +**Validation**: |
| 143 | +- Confirms all checkpoints reported in correct order |
| 144 | +- Verifies sequential execution pattern |
| 145 | +- Checks completion message |
| 146 | + |
| 147 | +**File Locations**: |
| 148 | +- Rust: `test_scenarios/rust/src/scenarios/lifecycle/launch_manager_support.rs` |
| 149 | +- C++: `test_scenarios/cpp/src/scenarios/lifecycle/launch_manager_support.cpp` |
| 150 | +- Test: `test_cases/tests/lifecycle/test_dependency_ordering.py` |
| 151 | + |
| 152 | +### 3. Parallel Launching |
| 153 | + |
| 154 | +**Purpose**: Demonstrate that multiple independent health monitors can run concurrently |
| 155 | + |
| 156 | +**Implementation**: |
| 157 | +- Creates multiple parallel monitors (configurable count, default 4) |
| 158 | +- Each monitor runs in a separate thread |
| 159 | +- Demonstrates concurrent monitoring capability |
| 160 | +- C++: Uses mutex for thread-safe stdout logging |
| 161 | + |
| 162 | +**Validation**: |
| 163 | +- Confirms all parallel monitors started |
| 164 | +- Verifies all monitors completed successfully |
| 165 | +- Checks for completion confirmation message |
| 166 | + |
| 167 | +**File Locations**: |
| 168 | +- Rust: `test_scenarios/rust/src/scenarios/lifecycle/launch_manager_support.rs` |
| 169 | +- C++: `test_scenarios/cpp/src/scenarios/lifecycle/launch_manager_support.cpp` |
| 170 | +- Test: `test_cases/tests/lifecycle/test_parallel_launching.py` |
| 171 | + |
| 172 | +## Running the Tests |
| 173 | + |
| 174 | +### Run All Lifecycle Tests (Both Languages) |
| 175 | + |
| 176 | +```bash |
| 177 | +bazel test --config=linux-x86_64 \ |
| 178 | + //feature_integration_tests/test_cases:fit_rust \ |
| 179 | + //feature_integration_tests/test_cases:fit_cpp \ |
| 180 | + --test_filter="*lifecycle*" |
| 181 | +``` |
| 182 | + |
| 183 | +### Run Specific Language |
| 184 | + |
| 185 | +```bash |
| 186 | +# Rust only |
| 187 | +bazel test //feature_integration_tests/test_cases:fit_rust --test_filter="*lifecycle*" |
| 188 | + |
| 189 | +# C++ only |
| 190 | +bazel test --config=linux-x86_64 //feature_integration_tests/test_cases:fit_cpp --test_filter="*lifecycle*" |
| 191 | +``` |
| 192 | + |
| 193 | +### Run Specific Scenario |
| 194 | + |
| 195 | +```bash |
| 196 | +# Process launching support |
| 197 | +bazel test --config=linux-x86_64 //feature_integration_tests/test_cases:fit_cpp \ |
| 198 | + --test_filter="*process_launching*" |
| 199 | + |
| 200 | +# Dependency ordering |
| 201 | +bazel test --config=linux-x86_64 //feature_integration_tests/test_cases:fit_rust \ |
| 202 | + --test_filter="*dependency_ordering*" |
| 203 | + |
| 204 | +# Parallel launching |
| 205 | +bazel test --config=linux-x86_64 //feature_integration_tests/test_cases:fit_cpp \ |
| 206 | + --test_filter="*parallel_launching*" |
| 207 | +``` |
| 208 | + |
| 209 | +### Debug Individual Scenarios |
| 210 | + |
| 211 | +List available scenarios: |
| 212 | +```bash |
| 213 | +bazel run //feature_integration_tests/test_scenarios/rust:rust_test_scenarios -- --list-scenarios |
| 214 | +bazel run --config=linux-x86_64 //feature_integration_tests/test_scenarios/cpp:cpp_test_scenarios -- --list-scenarios |
| 215 | +``` |
| 216 | + |
| 217 | +Run a specific scenario directly: |
| 218 | +```bash |
| 219 | +bazel run //feature_integration_tests/test_scenarios/rust:rust_test_scenarios -- \ |
| 220 | + rust lifecycle.process_launching_support '{"test":{"test_duration_ms":100,"checkpoint_count":3}}' |
| 221 | +``` |
| 222 | + |
| 223 | +## Test Configuration |
| 224 | + |
| 225 | +All lifecycle tests accept a JSON configuration with the following structure: |
| 226 | + |
| 227 | +```json |
| 228 | +{ |
| 229 | + "test": { |
| 230 | + "test_duration_ms": 100, |
| 231 | + "checkpoint_count": 3 |
| 232 | + } |
| 233 | +} |
| 234 | +``` |
| 235 | + |
| 236 | +**Parameters**: |
| 237 | +- `test_duration_ms`: Duration in milliseconds for simulated application work |
| 238 | +- `checkpoint_count`: Number of checkpoints/monitors to create (for ordering and parallel tests) |
| 239 | + |
| 240 | +## Dependencies |
| 241 | + |
| 242 | +### Rust |
| 243 | +- `lifecycle_client_rs` — Lifecycle client bindings for Rust |
| 244 | +- `health_monitoring_lib` — Health monitoring library |
| 245 | +- `test_scenarios_rust` — Test scenario framework |
| 246 | +- `tracing` — Structured logging |
| 247 | + |
| 248 | +### C++ |
| 249 | +- `score::lcm::lifecycle_client` — Lifecycle client library |
| 250 | +- `score::json` — JSON parsing |
| 251 | +- `test_scenarios_cpp` — Test scenario framework |
| 252 | + |
| 253 | +### Python |
| 254 | +- `pytest` — Test framework and orchestration |
| 255 | +- `testing_utils` — Log parsing and scenario execution utilities |
| 256 | + |
| 257 | +## Future Enhancements |
| 258 | + |
| 259 | +Potential areas for expansion: |
| 260 | + |
| 261 | +1. **Integration with Real Launch Manager**: |
| 262 | + - Tests that run against actual Launch Manager daemon |
| 263 | + - Validation of supervision and recovery behavior |
| 264 | + - End-to-end lifecycle state transitions |
| 265 | + |
| 266 | +2. **Additional Scenarios**: |
| 267 | + - Process termination and cleanup |
| 268 | + - Error recovery patterns |
| 269 | + - State persistence across restarts |
| 270 | + |
| 271 | +3. **Performance Testing**: |
| 272 | + - Measure supervision overhead |
| 273 | + - Validate scalability with many parallel processes |
| 274 | + - Benchmark deadline monitoring accuracy |
| 275 | + |
| 276 | +4. **Configuration Testing**: |
| 277 | + - Test various Launch Manager configurations |
| 278 | + - Validate configuration schema compliance |
| 279 | + - Test configuration error handling |
| 280 | + |
| 281 | +## References |
| 282 | + |
| 283 | +- [Launch Manager Design](https://github.com/eclipse-score/lifecycle) |
| 284 | +- [Health Monitoring Documentation](https://github.com/eclipse-score/lifecycle/tree/main/src/health_monitoring_lib) |
| 285 | +- [Lifecycle Client API](https://github.com/eclipse-score/lifecycle/tree/main/src/launch_manager_daemon/lifecycle_client_lib) |
| 286 | +- [Feature Integration Test Framework](./README.md) |
| 287 | + |
| 288 | +--- |
| 289 | + |
| 290 | +**Copyright (c) 2026 Contributors to the Eclipse Foundation** |
| 291 | + |
| 292 | +Licensed under the Apache License, Version 2.0 |
0 commit comments