feat: add support to cancel exectuions and WS

Author: xavidop

SERVER.md

@@ -1,10 +1,11 @@
 # Voiceflow CLI API Server
 
-The Voiceflow CLI now includes an HTTP API server that exposes test execution functionality as REST endpoints with auto-generated OpenAPI/Swagger documentation.
+The Voiceflow CLI now includes an HTTP API server that exposes test execution functionality as REST endpoints with auto-generated OpenAPI/Swagger documentation and **WebSocket support for real-time log streaming**.
 
 ## Features
 
 - **HTTP API**: Execute test suites via REST endpoints
+- **WebSocket API**: Execute, cancel, and monitor test suites over a persistent WebSocket connection with real-time log streaming
 - **Real-time Logging**: Capture and return test execution logs in API responses
 - **OpenAPI/Swagger**: Auto-generated API documentation at `/swagger/index.html`
 - **Asynchronous Execution**: Non-blocking test execution with status tracking
@@ -140,6 +141,157 @@ Once the server is running, you can access the interactive API documentation at:
 http://localhost:8080/swagger/index.html
 ```
 
+## WebSocket API
+
+Connect to `ws://localhost:8080/ws` for a persistent connection with real-time log streaming. The WebSocket endpoint exposes the same functionality as the REST API but pushes log lines as they happen instead of requiring polling.
+
+### Protocol
+
+All messages are JSON objects.
+
+**Client → Server:**
+
+| Action | Description | Fields |
+|--------|-------------|--------|
+| `execute` | Start a test suite | `{ "action": "execute", "data": <TestExecutionRequest> }` |
+| `cancel` | Cancel a running execution | `{ "action": "cancel", "id": "<execution-id>" }` |
+| `status` | Get execution status | `{ "action": "status", "id": "<execution-id>" }` |
+
+**Server → Client:**
+
+| Type | Description |
+|------|-------------|
+| `log` | Real-time log line: `{ "type": "log", "id": "...", "message": "..." }` |
+| `status` | Status update: `{ "type": "status", "id": "...", "data": <TestStatusResponse> }` |
+| `result` | Final result when execution finishes: `{ "type": "result", "id": "...", "data": <TestStatusResponse> }` |
+| `error` | Error message: `{ "type": "error", "message": "..." }` |
+
+### WebSocket Examples
+
+#### Using websocat (CLI)
+
+```bash
+# Install: brew install websocat
+websocat ws://localhost:8080/ws
+```
+
+Then paste JSON messages:
+
+```json
+{"action":"execute","data":{"suite":{"name":"Example Suite","description":"Test","environment_name":"production","tests":[{"id":"test_1","test":{"name":"Example test","description":"Test","interactions":[{"id":"t1","user":{"type":"text","text":"hi"},"agent":{"validate":[{"type":"contains","value":"hello"}]}}]}}]}}}
+```
+
+Cancel a running execution:
+
+```json
+{"action":"cancel","id":"<execution-id-from-status-message>"}
+```
+
+#### Using JavaScript
+
+```javascript
+const ws = new WebSocket('ws://localhost:8080/ws');
+
+ws.onopen = () => {
+  // Execute a test suite
+  ws.send(JSON.stringify({
+    action: 'execute',
+    data: {
+      api_key: 'your_api_key',
+      suite: {
+        name: 'Example Suite',
+        description: 'Suite used as an example',
+        environment_name: 'production',
+        tests: [
+          {
+            id: 'test_1',
+            test: {
+              name: 'Example test',
+              description: 'These are some tests',
+              interactions: [
+                {
+                  id: 'test_1_1',
+                  user: { type: 'text', text: 'hi' },
+                  agent: { validate: [{ type: 'contains', value: 'hello' }] }
+                }
+              ]
+            }
+          }
+        ]
+      }
+    }
+  }));
+};
+
+ws.onmessage = (event) => {
+  const msg = JSON.parse(event.data);
+  switch (msg.type) {
+    case 'log':
+      console.log(`[LOG] ${msg.message}`);
+      break;
+    case 'status':
+      console.log(`[STATUS] ${msg.id}: ${msg.message || msg.data.status}`);
+      // Cancel example: ws.send(JSON.stringify({ action: 'cancel', id: msg.id }));
+      break;
+    case 'result':
+      console.log('[RESULT]', msg.data);
+      break;
+    case 'error':
+      console.error('[ERROR]', msg.message);
+      break;
+  }
+};
+```
+
+#### Using Python
+
+```python
+import asyncio
+import json
+import websockets
+
+async def run_tests():
+    async with websockets.connect('ws://localhost:8080/ws') as ws:
+        # Execute a test suite
+        await ws.send(json.dumps({
+            'action': 'execute',
+            'data': {
+                'api_key': 'your_api_key',
+                'suite': {
+                    'name': 'Example Suite',
+                    'description': 'Test',
+                    'environment_name': 'production',
+                    'tests': [{
+                        'id': 'test_1',
+                        'test': {
+                            'name': 'Example test',
+                            'description': 'Test',
+                            'interactions': [{
+                                'id': 't1',
+                                'user': {'type': 'text', 'text': 'hi'},
+                                'agent': {'validate': [{'type': 'contains', 'value': 'hello'}]}
+                            }]
+                        }
+                    }]
+                }
+            }
+        }))
+
+        # Read messages until execution completes
+        async for message in ws:
+            msg = json.loads(message)
+            if msg['type'] == 'log':
+                print(f"[LOG] {msg['message']}")
+            elif msg['type'] == 'result':
+                print(f"[RESULT] {msg['data']}")
+                break
+            elif msg['type'] == 'error':
+                print(f"[ERROR] {msg['message']}")
+                break
+
+asyncio.run(run_tests())
+```
+
 ## Usage Examples
 
 ### Using curl

docs/docs/server/api-endpoints.md

@@ -1,13 +1,23 @@
 # API Endpoints
 
-## Health Check
+## REST API
+
+### Health Check
 ```http
 GET /health
 ```
 
 Returns the health status of the server.
 
-## Execute Test Suite
+**Response:**
+```json
+{
+  "status": "healthy",
+  "time": "2026-02-23T14:00:00+01:00"
+}
+```
+
+### Execute Test Suite
 ```http
 POST /api/v1/tests/execute
 Content-Type: application/json
@@ -49,7 +59,7 @@ Content-Type: application/json
 }
 ```
 
-**Response:**
+**Response (202 Accepted):**
 ```json
 {
   "id": "123e4567-e89b-12d3-a456-426614174000",
@@ -59,23 +69,23 @@ Content-Type: application/json
 }
 ```
 
-Executes a test suite asynchronously and returns an execution ID for tracking. The suite configuration and tests are now embedded directly in the request body, making the API more HTTP-friendly and eliminating the need for file system access.
+Executes a test suite asynchronously and returns an execution ID for tracking. The suite configuration and tests are embedded directly in the request body, making the API more HTTP-friendly and eliminating the need for file system access.
 
-### Request Parameters
+#### Request Parameters
 
 - `api_key` (optional): Override the global Voiceflow API key for this specific test execution
 - `voiceflow_subdomain` (optional): Override the global Voiceflow subdomain for this specific test execution. This allows you to test against different Voiceflow environments or custom subdomains without affecting the global configuration
 - `suite`: The test suite configuration containing the test definitions
 
-### Using Custom Subdomains
+#### Using Custom Subdomains
 
 When you specify a `voiceflow_subdomain`, the API will use that subdomain for all interactions in the test suite. For example:
 
 - If you set `"voiceflow_subdomain": "my-custom-env"`, requests will be sent to `https://general-runtime.my-custom-env.voiceflow.com`
 - If you omit this field, the global subdomain configuration will be used
 - This is particularly useful for testing against staging environments or customer-specific deployments
 
-## Get Test Status
+### Get Test Status
 ```http
 GET /api/v1/tests/status/{execution_id}
 ```
@@ -97,7 +107,33 @@ GET /api/v1/tests/status/{execution_id}
 
 Retrieves the current status and logs of a test execution.
 
-## System Information
+### Cancel Test Execution
+```http
+POST /api/v1/tests/cancel/{execution_id}
+```
+
+Cancels a running test execution.
+
+**Response (200 OK):**
+```json
+{
+  "id": "123e4567-e89b-12d3-a456-426614174000",
+  "status": "cancelled",
+  "started_at": "2023-01-01T00:00:00Z",
+  "completed_at": "2023-01-01T00:02:30Z",
+  "logs": [
+    "Starting test suite execution...",
+    "Test suite execution cancelled"
+  ]
+}
+```
+
+**Error Responses:**
+
+- `404 Not Found`: No execution with the given ID
+- `409 Conflict`: Execution is not running (already completed, failed, or cancelled)
+
+### System Information
 ```http
 GET /api/v1/system/info
 ```
@@ -114,6 +150,31 @@ GET /api/v1/system/info
 
 Returns system information about the running server instance.
 
+## WebSocket API
+
+Connect to `ws://localhost:8080/ws` for a persistent connection with real-time log streaming. The WebSocket endpoint exposes the same functionality as the REST API but pushes log lines as they happen instead of requiring polling.
+
+### Protocol
+
+All messages are JSON objects.
+
+**Client → Server:**
+
+| Action | Description | Fields |
+|--------|-------------|--------|
+| `execute` | Start a test suite | `{ "action": "execute", "data": <TestExecutionRequest> }` |
+| `cancel` | Cancel a running execution | `{ "action": "cancel", "id": "<execution-id>" }` |
+| `status` | Get execution status | `{ "action": "status", "id": "<execution-id>" }` |
+
+**Server → Client:**
+
+| Type | Description |
+|------|-------------|
+| `log` | Real-time log line: `{ "type": "log", "id": "...", "message": "..." }` |
+| `status` | Status update: `{ "type": "status", "id": "...", "data": <TestStatusResponse> }` |
+| `result` | Final result when execution finishes: `{ "type": "result", "id": "...", "data": <TestStatusResponse> }` |
+| `error` | Error message: `{ "type": "error", "message": "..." }` |
+
 ## OpenAPI/Swagger Documentation
 
 Once the server is running, you can access the interactive API documentation at:

docs/docs/server/introduction.md

@@ -1,13 +1,15 @@
 # API Server Introduction
 
-The Voiceflow CLI now includes an HTTP API server that exposes test execution functionality as REST endpoints with auto-generated OpenAPI/Swagger documentation.
+The Voiceflow CLI now includes an HTTP API server that exposes test execution functionality as REST endpoints with auto-generated OpenAPI/Swagger documentation and **WebSocket support for real-time log streaming**.
 
 ## Features
 
 - **HTTP API**: Execute test suites via REST endpoints
+- **WebSocket API**: Execute, cancel, and monitor test suites over a persistent WebSocket connection with real-time log streaming
 - **Real-time Logging**: Capture and return test execution logs in API responses
 - **OpenAPI/Swagger**: Auto-generated API documentation at `/swagger/index.html`
 - **Asynchronous Execution**: Non-blocking test execution with status tracking
+- **Cancellation Support**: Cancel running test executions via REST or WebSocket
 - **Custom Subdomains**: Support for per-request Voiceflow subdomain overrides
 - **Multi-Environment Testing**: Concurrent testing against different environments
 - **Thread Safety**: No race conditions when testing multiple subdomains simultaneously

docs/docs/server/public-instance.md

@@ -6,6 +6,8 @@ If you want to try out the Voiceflow CLI server without setting it up locally, y
 
 **Base URL:** https://voiceflow-cli-api.xavidop.me/
 
+**WebSocket URL:** wss://voiceflow-cli-api.xavidop.me/ws
+
 ## Interactive API Documentation
 
 You can explore and test the API endpoints directly in your browser using our Swagger playground:
@@ -23,4 +25,4 @@ The Swagger interface allows you to:
 1. Visit the [Swagger playground](https://docs.voiceflow.com/reference/post_api-v1-tests-execute#/)
 2. Explore the available endpoints
 3. Try out the API calls with your own data
-4. Use the examples from the [usage examples](usage-examples.md) by replacing `http://localhost:8080` with `https://voiceflow-cli-api.xavidop.me`
+4. Use the examples from the [usage examples](usage-examples.md) by replacing `http://localhost:8080` with `https://voiceflow-cli-api.xavidop.me` (and `ws://localhost:8080` with `wss://voiceflow-cli-api.xavidop.me` for WebSocket)

docs/docs/server/security-troubleshooting.md

@@ -10,6 +10,8 @@
 
 - **CORS**: Cross-Origin Resource Sharing (CORS) is enabled by default. You can disable it using `--cors=false` if not needed.
 
+- **WebSocket**: The WebSocket endpoint (`/ws`) accepts connections from all origins by default. In production, consider restricting allowed origins via a reverse proxy.
+
 ## Troubleshooting
 
 ### Server Won't Start
@@ -43,6 +45,31 @@
 2. Verify the server is running and accessible
 3. Check the server logs for any startup errors
 
+### REST Execute Returns Immediately
+
+**Problem**: `POST /api/v1/tests/execute` returns right away with status `"running"` instead of waiting for the tests to finish.
+
+**Explanation**: This is expected behavior. The REST endpoint is **asynchronous** — it starts the test execution in the background and returns an execution ID immediately.
+
+**Solutions**:
+
+1. Poll the status endpoint to track progress:
+   ```bash
+   curl http://localhost:8080/api/v1/tests/status/YOUR_EXECUTION_ID
+   ```
+
+2. If you want real-time streaming of logs as they happen, use the **WebSocket endpoint** (`/ws`) instead — see the [usage examples](usage-examples.md#websocket-api) for details.
+
+### WebSocket Connection Closes Immediately
+
+**Problem**: WebSocket connection closes right after connecting.
+
+**Solutions**:
+
+1. Make sure you're using a proper WebSocket client (e.g., `wscat`, `websocat`), not `curl` — HTTP tools like `curl` do not support WebSocket
+2. Check server logs for upgrade errors
+3. Verify the URL uses the `ws://` scheme: `ws://localhost:8080/ws`
+
 ### Logs Not Appearing
 
 **Problem**: Test execution logs are not visible or incomplete.