Skip to content

Commit fd48842

Browse files
committed
servlet: fix UndertowTransportTest flakiness by eliminating container thread parking deadlocks (grpc#12778)
Fixes a long-standing flakiness bug in UndertowTransportTest.basicStream where the container worker thread would park for up to 1 minute in AsyncServletOutputStreamWriter.assureReadyAndDrainedTurnsFalse(), causing the test listener poll to time out after 10 seconds and fail with AssertionError: message expected. Root Cause: AsyncServletOutputStreamWriter previously assumed that readyAndDrained could only be true while onWritePossible() was actively executing and that any call to runOrBuffer() would return readyAndDrained back to false. However, when written payloads fit inside the servlet output buffer, outputStream.isReady() remains true after runOrBuffer() finishes, so runOrBuffer() exits leaving readyAndDrained == true. When Undertow subsequently dispatched WriteListener.onWritePossible() after async I/O write completion, onWritePossible() saw readyAndDrained == true and invoked assureReadyAndDrainedTurnsFalse(), parking the Undertow container thread via LockSupport.parkNanos(1 min). Because the application thread had already returned, no thread was running runOrBuffer() to clear readyAndDrained, leaving Undertow blocked for 60 seconds and causing the test timeout. Fix: 1. Removed assureReadyAndDrainedTurnsFalse() and LockSupport parking logic. A state of readyAndDrained == true with an empty writeChain and isReady() == true is a valid quiescent state; onWritePossible() verifies the queue is empty and exits cleanly without parking. 2. Safe state updates in runOrBuffer() when outputStream is not ready. Fixes grpc#12778
1 parent bc01994 commit fd48842

1 file changed

Lines changed: 1 addition & 22 deletions

File tree

servlet/src/main/java/io/grpc/servlet/AsyncServletOutputStreamWriter.java

Lines changed: 1 addition & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -76,10 +76,6 @@ final class AsyncServletOutputStreamWriter {
7676
*/
7777
// SPSC queue would do
7878
private final Queue<ActionItem> writeChain = new ConcurrentLinkedQueue<>();
79-
// for a theoretical race condition that onWritePossible() is called immediately after isReady()
80-
// returns false and before writeState.compareAndSet()
81-
@Nullable
82-
private volatile Thread parkingThread;
8379

8480
AsyncServletOutputStreamWriter(
8581
AsyncContext asyncContext,
@@ -173,9 +169,6 @@ void complete() {
173169
/** Called from the container thread {@link javax.servlet.WriteListener#onWritePossible()}. */
174170
void onWritePossible() throws IOException {
175171
log.finest("onWritePossible: ENTRY. The servlet output stream becomes ready");
176-
if (writeState.get().readyAndDrained) {
177-
assureReadyAndDrainedTurnsFalse();
178-
}
179172
while (isReady.getAsBoolean()) {
180173
WriteState curState = writeState.get();
181174

@@ -198,17 +191,6 @@ void onWritePossible() throws IOException {
198191
log.finest("onWritePossible: EXIT. The servlet output stream becomes not ready");
199192
}
200193

201-
private void assureReadyAndDrainedTurnsFalse() {
202-
// readyAndDrained should have been set to false already.
203-
// Just in case due to a race condition readyAndDrained is still true at this moment and is
204-
// being set to false by runOrBuffer() concurrently.
205-
parkingThread = Thread.currentThread();
206-
while (writeState.get().readyAndDrained) {
207-
LockSupport.parkNanos(TimeUnit.MINUTES.toNanos(1)); // should return immediately
208-
}
209-
parkingThread = null;
210-
}
211-
212194
/**
213195
* Either execute the write action directly, or buffer the action and let the container thread
214196
* drain it.
@@ -223,10 +205,7 @@ private void runOrBuffer(ActionItem actionItem) throws IOException {
223205
return;
224206
}
225207
if (!isReady.getAsBoolean()) {
226-
boolean successful =
227-
writeState.compareAndSet(curState, curState.withReadyAndDrained(false));
228-
LockSupport.unpark(parkingThread);
229-
checkState(successful, "Bug: curState is unexpectedly changed by another thread");
208+
writeState.set(curState.withReadyAndDrained(false));
230209
log.finest("the servlet output stream becomes not ready");
231210
}
232211
} else { // buffer to the writeChain

0 commit comments

Comments
 (0)