Add streaming test results via Server-Sent Events (SSE)#189
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adds real-time streaming of test results via Server-Sent Events (SSE) to TestBox. It introduces a new StreamingService.cfc utility class and a StreamingRunner.cfm runner endpoint, and enables the feature via a streaming=true URL parameter in the existing test runner files.
Changes:
- New
StreamingService.cfc: Core SSE service that emits progress events (testRunStart,bundleStart/End,suiteStart/End,specStart/End,testRunEnd) during test execution viawriteOutput/cfflush. - New
StreamingRunner.cfm: Runner endpoint that initializes SSE headers, runs TestBox with streaming callbacks, and emits a finaltestRunEndevent with the full results memento. - Updated
tests/runner.cfmandcfml/tests/runner.cfm: Addedurl.streamingparameter and conditional routing to eitherStreamingRunner.cfmor the existingHTMLRunner.cfm.
Reviewed changes
Copilot reviewed 5 out of 6 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
system/util/StreamingService.cfc |
New core SSE service: sets response headers, streams events as JSON-encoded SSE, provides TestBox lifecycle callbacks |
system/runners/StreamingRunner.cfm |
New runner that wires StreamingService to TestBox and streams testRunStart/testRunEnd framing events |
tests/runner.cfm |
Adds url.streaming param and conditional include for StreamingRunner vs HTMLRunner |
cfml/tests/runner.cfm |
Same streaming conditional routing for the CFML-specific test runner |
tests/specs/streaming/StreamingServiceTest.cfc |
New unit tests for StreamingService callbacks and event data structures |
server-boxlang-cfml@1.json |
Whitespace/indentation normalization only — no functional change |
Implement real-time test progress streaming for CI tools like CommandBox to display incremental progress instead of waiting for entire test suite completion. - Add StreamingService.cfc with SSE utilities and TestBox callbacks - Add StreamingRunner.cfm endpoint activated via ?streaming=true URL param - Update runner.cfm files to support streaming mode - Add comprehensive unit tests for StreamingService
elpete
force-pushed
the
feature/streaming-test-results
branch
from
6b60e95 to
855cb7b
Compare
…events The bundle stats (including id) are not available when onBundleStart fires, so consumers should use the 'path' field for correlation instead.
Use savecontent to capture actual output from streamEvent instead of testing manually constructed strings.
Wrap writeOutput/cfflush in try/catch to prevent client disconnects from interrupting test execution. Errors are logged to testbox-streaming log file.
When a test suite uses asyncAll=true, spec events are invoked from cfthread and won't be streamed in real-time since writeOutput/cfflush don't affect the parent request's HTTP response buffer.
Instead of assuming the last bundleStats entry corresponds to the current bundle, match by path using the target's metadata. Falls back to last entry if no match is found.
If an exception propagates out of runRaw(), stream an 'error' event before rethrowing so SSE consumers can detect the failure instead of waiting indefinitely for testRunEnd.
Generate id using hash(path) so consumers can correlate bundleStart and bundleEnd events. This id is deterministic and available before internal bundle stats are created.
- Add flushEnabled property (default true) to StreamingService - Skip cfflush when flushEnabled is false to prevent response commit - Update tests to disable flushing and fix bundleEnd id assertion
The SSE format test was failing on Lucee because savecontent doesn't capture writeOutput from nested method calls. Added formatSSEEvent() method that returns the formatted string directly, which streamEvent() now uses internally. Tests now call formatSSEEvent() directly instead of relying on savecontent to capture writeOutput.
Implement thread-safe event queuing using ConcurrentLinkedQueue so SSE streaming works when specs run in parallel cfthread blocks. Events are auto-queued when detected inside a thread, then drained to output after thread joins via onAsyncDrain callback.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds real-time streaming test results output via Server-Sent Events (SSE), allowing test consumers (like testbox-cli) to display test progress as tests execute rather than waiting for the entire test run to complete.
Features
streaming=trueURL parameterEvent Types
The SSE stream emits the following events:
testRunStart- Emitted when the entire test run beginsbundleStart/bundleEnd- Bundle lifecycle events (correlate viapathfield)suiteStart/suiteEnd- Suite lifecycle eventsspecStart/specEnd- Individual spec lifecycle eventstestRunEnd- Final event with complete TestBox resultsCompatibility
Related
Testing
The streaming can be tested by:
/tests/runner.cfm?streaming=truecurl