Refactor AppHarness to use reflex run subprocess instead of threads

[devin-ai-integration]

Refactor AppHarness to use subprocess instead of threads

Summary

This PR refactors the AppHarness class in reflex/testing.py to use reflex run in a subprocess instead of thread-based server management. This aligns better with how Reflex apps normally run (using granian by default) and reduces custom code maintenance burden.

Key Changes:

• Replaced thread-based _start_backend and _start_frontend methods with subprocess-based _start_subprocess
• Added _wait_for_servers method that parses console output to detect server readiness using regex patterns
• Implemented shell=True approach for subprocess to properly capture stdout/stderr from reflex run
• Added proper process cleanup in stop() method to terminate subprocesses
• Fixed all darglint documentation errors for parameter and return type documentation
• Maintained API compatibility where possible (state management methods now raise NotImplementedError)

Current Status:

• All 12 integration-app-harness CI tests are still failing, but this is expected as the refactoring is not yet complete
• Local testing shows the subprocess implementation correctly starts servers and detects readiness
• All pre-commit checks (darglint, ruff, pyright) are now passing

Review & Testing Checklist for Human

🔴 High Risk - 5 items to verify:

• Test subprocess implementation locally: Run integration tests locally to verify servers start correctly and process cleanup works (no hanging reflex run processes after tests)
• Verify console output parsing reliability: Test that the regex patterns in _wait_for_servers correctly detect "App running at:" and "Backend running at:" messages across different environments
• Check process cleanup edge cases: Test that subprocess termination works correctly when tests fail or are interrupted (e.g., with Ctrl+C)
• Test with different state managers: Verify the refactored AppHarness works with both memory and Redis state managers
• Validate API compatibility: Check that existing tests that use app_harness.get_state() still work correctly with the new subprocess approach

Recommended Test Plan:

1. Run uv run pytest tests/integration/test_input.py::test_fully_controlled_input -v -s locally
2. Check ps aux | grep reflex before and after tests to verify no hanging processes
3. Test with REDIS_URL=redis://localhost:6379 to verify Redis state manager compatibility
4. Review CI logs for specific integration test failure patterns to understand remaining issues

---

Diagram

%%{ init : { "theme" : "default" }}%%
graph TD
    Testing["reflex/testing.py<br/>AppHarness class"]:::major-edit
    Processes["reflex/utils/processes.py<br/>new_process helper"]:::context
    IntegrationTests["tests/integration/<br/>12 failing tests"]:::context
    
    Testing -->|"uses subprocess"| Processes
    Testing -->|"starts via"| ReflexRun["reflex run command<br/>with shell=True"]:::minor-edit
    IntegrationTests -->|"instantiate"| Testing
    
    subgraph "AppHarness Methods"
        StartSubprocess["_start_subprocess()<br/>NEW METHOD"]:::major-edit
        WaitServers["_wait_for_servers()<br/>NEW METHOD"]:::major-edit
        Stop["stop()<br/>UPDATED"]:::minor-edit
    end
    
    Testing --> StartSubprocess
    Testing --> WaitServers
    Testing --> Stop
    
    subgraph Legend
        L1[Major Edit]:::major-edit
        L2[Minor Edit]:::minor-edit
        L3[Context/No Edit]:::context
    end

classDef major-edit fill:#90EE90
classDef minor-edit fill:#87CEEB
classDef context fill:#FFFFFF

Loading

Notes

• Link to Devin run: https://app.devin.ai/sessions/bc06be7219144a1fb3ff1795617e9a29
• Requested by: masen@reflex.dev
• Linear ticket: ENG-6317
• The shell=True approach was critical for properly capturing subprocess output - previous attempts without it failed to detect server readiness
• Integration test failures are expected at this stage; the goal was to get the basic subprocess infrastructure working
• All darglint errors have been resolved by adding proper parameter documentation and fixing return type documentation

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment and CI monitoring

[linear]

ENG-6317 AppHarness support for granian

[greptile-apps]

PR Summary

Major architectural change to reflex/testing.py replacing the thread-based AppHarness implementation with a subprocess-based approach using 'reflex run' command.

• Eliminates direct uvicorn server and frontend thread management in favor of unified subprocess control
• Introduces port management for both frontend and backend servers through reflex subprocess
• Removes custom TCP server implementation for static file serving
• Adds process lifecycle management with proper cleanup handling
• Maintains backward compatibility through deprecated thread-related fields

_{1 file reviewed, 2 comments
Edit PR Review Bot Settings | Greptile}

[codspeed-hq]

CodSpeed Performance Report

Merging #5555 will not alter performance

_{Comparing devin/ENG-6317-1752182691 (a709884) with main (60b1baf)}

Summary

✅ 8 untouched benchmarks

[masenf]

unfortunately this seems to add lots of time in the CI, not sure why the port takes so long to be available again after stopping reflex... @adhami3310 ?

[adhami3310]

that's fun, i think we can improve this a tad bit by printing why a port is closed, let me try that in main