|
| 1 | +# iOS XCTest Agent MVP Plan |
| 2 | + |
| 3 | +## Goal |
| 4 | + |
| 5 | +Implement an iOS XCTest-based agent that can run against both simulators and physical devices, and use it in the MVP to auto-accept permission prompts on a best-effort basis. |
| 6 | + |
| 7 | +This should be a generic XCTest integration for Harness, not a permission-specific helper, so it can be reused for other iOS system-level automation later. |
| 8 | + |
| 9 | +## MVP Scope |
| 10 | + |
| 11 | +- iOS only |
| 12 | +- Support both simulator and physical device targets |
| 13 | +- Start the XCTest agent once per Harness run |
| 14 | +- Stop the XCTest agent during Harness teardown |
| 15 | +- Best-effort auto-accept of permission prompts |
| 16 | +- Unknown prompts are ignored silently |
| 17 | +- No public testing API changes |
| 18 | +- No deny/override behavior |
| 19 | +- No Android implementation in this phase |
| 20 | +- No `simctl privacy` optimization in this phase |
| 21 | + |
| 22 | +## Architecture Direction |
| 23 | + |
| 24 | +- Add a generic run-level lifecycle hook so platform runners can prepare and dispose auxiliary tooling needed for the run. |
| 25 | +- Implement the iOS side using a generic `XCTest agent` concept owned by `platform-ios`. |
| 26 | +- Package the XCTest agent as a small Xcode project generated with `xcodegen`. |
| 27 | +- Use the same XCTest agent concept for both iOS simulators and physical devices. |
| 28 | +- Keep permission prompt handling as the first XCTest agent capability, not the only one. |
| 29 | + |
| 30 | +## Phase 1: Lifecycle Integration |
| 31 | + |
| 32 | +Status: Completed |
| 33 | + |
| 34 | +Objective: create the Harness and platform lifecycle seam needed to run auxiliary tooling once per run. |
| 35 | + |
| 36 | +Deliverables: |
| 37 | + |
| 38 | +- Run-level prepare/dispose hooks available on platform runners |
| 39 | +- Harness wired to invoke those hooks once per run |
| 40 | +- Coverage for success, error, and teardown paths |
| 41 | + |
| 42 | +Notes: |
| 43 | + |
| 44 | +- This phase should remain generic and not mention XCTest directly in shared abstractions. |
| 45 | +- The outcome should be reusable by any future platform-owned run helper. |
| 46 | + |
| 47 | +Parallelization: |
| 48 | + |
| 49 | +- Can be done independently from XCTest project creation |
| 50 | +- Must land before full end-to-end iOS wiring is completed |
| 51 | + |
| 52 | +## Phase 2: XCTest Agent Project |
| 53 | + |
| 54 | +Status: Completed |
| 55 | + |
| 56 | +Objective: create the reusable iOS XCTest agent project and prove it can be generated reproducibly. |
| 57 | + |
| 58 | +Deliverables: |
| 59 | + |
| 60 | +- New internal `xctest-agent` project inside `packages/platform-ios` |
| 61 | +- Project generated from `xcodegen` spec rather than manually maintained project internals |
| 62 | +- Minimal shared project structure suitable for both simulator and physical-device builds |
| 63 | +- Documented build assumptions and cache inputs |
| 64 | + |
| 65 | +Notes: |
| 66 | + |
| 67 | +- This phase focuses on project packaging and generation, not Harness integration. |
| 68 | +- The top-level naming should stay generic so additional XCTest-driven capabilities can be added later. |
| 69 | + |
| 70 | +Parallelization: |
| 71 | + |
| 72 | +- Can proceed in parallel with Phase 1 |
| 73 | +- Can also proceed in parallel with the host-side iOS orchestration design work in Phase 3 |
| 74 | + |
| 75 | +## Phase 3: iOS XCTest Agent Orchestration |
| 76 | + |
| 77 | +Status: Completed |
| 78 | + |
| 79 | +Objective: add host-side orchestration in `platform-ios` to build, cache, start, and stop the XCTest agent. |
| 80 | + |
| 81 | +Deliverables: |
| 82 | + |
| 83 | +- Internal `platform-ios` orchestration for the XCTest agent |
| 84 | +- Support for simulator destinations |
| 85 | +- Support for physical-device destinations |
| 86 | +- Artifact reuse strategy for simulator and device builds |
| 87 | +- Clear separation between agent lifecycle management and agent behaviors |
| 88 | + |
| 89 | +Notes: |
| 90 | + |
| 91 | +- Simulator and physical device should share the same orchestration model, even if build artifacts differ. |
| 92 | +- The orchestration should treat the agent as a long-lived run-level helper, not something restarted per test file. |
| 93 | + |
| 94 | +Parallelization: |
| 95 | + |
| 96 | +- Depends on enough output from Phase 2 to know what project is being built and launched |
| 97 | +- Can be developed in parallel with Phase 4 if the behavior contract is kept narrow |
| 98 | + |
| 99 | +## Phase 4: Permission Prompt Capability |
| 100 | + |
| 101 | +Status: Completed |
| 102 | + |
| 103 | +Objective: implement the first XCTest agent capability: best-effort auto-accept of permission prompts. |
| 104 | + |
| 105 | +Deliverables: |
| 106 | + |
| 107 | +- Permission prompt interruption handling inside the XCTest agent |
| 108 | +- Best-effort positive-action tapping behavior |
| 109 | +- Silent ignore behavior for unrecognized prompts |
| 110 | +- Capability scoped so it can later live beside other XCTest agent behaviors |
| 111 | + |
| 112 | +Notes: |
| 113 | + |
| 114 | +- This phase should not introduce any public Harness API. |
| 115 | +- The implementation should be framed as one capability of the generic agent. |
| 116 | + |
| 117 | +Parallelization: |
| 118 | + |
| 119 | +- Can proceed in parallel with most of Phase 3 once the lifecycle between host and agent is understood |
| 120 | +- Final validation depends on Phase 3 integration |
| 121 | + |
| 122 | +## Phase 5: End-to-End iOS Wiring |
| 123 | + |
| 124 | +Status: Completed |
| 125 | + |
| 126 | +Objective: connect the generic lifecycle, iOS orchestration, and permission capability into the actual Harness run flow. |
| 127 | + |
| 128 | +Deliverables: |
| 129 | + |
| 130 | +- iOS simulator runs start the XCTest agent before first app launch |
| 131 | +- iOS physical-device runs start the XCTest agent before first app launch |
| 132 | +- Both stop the agent during teardown |
| 133 | +- Existing app launch and restart behavior remains unchanged |
| 134 | +- No per-file permission synchronization is introduced |
| 135 | + |
| 136 | +Notes: |
| 137 | + |
| 138 | +- The agent should be started lazily before the first app launch, not eagerly at Harness creation time. |
| 139 | +- This phase is where the MVP becomes functionally available. |
| 140 | + |
| 141 | +Parallelization: |
| 142 | + |
| 143 | +- Depends on Phases 1 through 4 |
| 144 | +- Should be kept small by reusing the outputs of earlier phases rather than adding new concepts |
| 145 | + |
| 146 | +## Phase 6: Validation And Hardening |
| 147 | + |
| 148 | +Objective: verify the MVP works on real targets and stabilize the integration. |
| 149 | + |
| 150 | +Deliverables: |
| 151 | + |
| 152 | +- Automated coverage for host-side lifecycle and orchestration behavior |
| 153 | +- Manual validation on at least one iOS simulator |
| 154 | +- Manual validation on at least one physical iOS device |
| 155 | +- Basic operational documentation for future contributors |
| 156 | + |
| 157 | +Validation focus: |
| 158 | + |
| 159 | +- First-run build experience |
| 160 | +- Reuse of cached artifacts on later runs |
| 161 | +- Permission prompt auto-accept for at least one real prompt source such as camera |
| 162 | +- No obvious teardown leaks or stuck background processes |
| 163 | + |
| 164 | +Parallelization: |
| 165 | + |
| 166 | +- Automated coverage can be built alongside Phase 5 |
| 167 | +- Manual validation happens after end-to-end wiring is in place |
| 168 | + |
| 169 | +## Suggested Parallel Workstreams |
| 170 | + |
| 171 | +### Stream A: Shared Lifecycle |
| 172 | + |
| 173 | +- Phase 1 |
| 174 | + |
| 175 | +### Stream B: XCTest Agent Project |
| 176 | + |
| 177 | +- Phase 2 |
| 178 | + |
| 179 | +### Stream C: iOS Agent Runtime Orchestration |
| 180 | + |
| 181 | +- Phase 3 |
| 182 | + |
| 183 | +### Stream D: Permission Capability |
| 184 | + |
| 185 | +- Phase 4 |
| 186 | + |
| 187 | +### Stream E: Final Wiring And Validation |
| 188 | + |
| 189 | +- Phase 5 |
| 190 | +- Phase 6 |
| 191 | + |
| 192 | +## Dependency Summary |
| 193 | + |
| 194 | +- Phase 1 is required before final integration |
| 195 | +- Phase 2 is required before full orchestration can be finalized |
| 196 | +- Phase 3 depends on Phase 2 |
| 197 | +- Phase 4 can begin before Phase 3 is finished, but depends on the agent project shape from Phase 2 |
| 198 | +- Phase 5 depends on Phases 1 through 4 |
| 199 | +- Phase 6 depends on Phase 5 |
| 200 | + |
| 201 | +## Explicit Non-Goals For This Plan |
| 202 | + |
| 203 | +- Public permission configuration API |
| 204 | +- Per-test or per-file permission overrides |
| 205 | +- Deny behavior |
| 206 | +- Android permission automation |
| 207 | +- Simulator fast-path optimization through `simctl privacy` |
| 208 | +- Strict unsupported-permission detection or reporting |
| 209 | + |
| 210 | +## Follow-Up After MVP |
| 211 | + |
| 212 | +- Add Android best-effort pregrant support via `adb` |
| 213 | +- Add `simctl privacy` fast path for the iOS simulator where supported |
| 214 | +- Add more XCTest agent capabilities beyond permission prompts |
| 215 | +- Revisit public API design once internal behavior is proven in practice |
0 commit comments