Skip to content

Fall back to longpoll when the websocket keeps dying after opening - #6768

Open
rdeese wants to merge 1 commit into
phoenixframework:mainfrom
rdeese:fix-6766-websocket-flap-fallback
Open

Fall back to longpoll when the websocket keeps dying after opening#6768
rdeese wants to merge 1 commit into
phoenixframework:mainfrom
rdeese:fix-6766-websocket-flap-fallback

Conversation

@rdeese

@rdeese rdeese commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Fixes #6766.

The bug

Socket.connectWithFallback has two paths to the fallback transport, and both are gated on the primary transport never having opened:

  • the fallbackTimer, and
  • the onError handler, guarded by if(primaryTransport && !established).

A websocket that opens and is then killed a few hundred milliseconds later escapes both. established latches on the first open, and the reconnect path re-enters connectWithFallback, whose first statement clears the pending fallback timer — so every cycle starts the clock over. reconnectTimer.reset() in onConnOpen keeps the backoff at its floor, and the client flaps indefinitely without ever trying longpoll.

The timer that gets re-armed on open does cover the neighbouring case, where the connection stays up past longPollFallbackMs without answering the ping. The gap is the connection that dies before that threshold, over and over.

The fix

Treat "opened but never proved useful" as primary-transport failure. onConnClose counts a connection that closed without ever having delivered a message; a connection that delivered one resets the count. Once MAX_UNUSABLE_PRIMARY_CONNECTIONS (3) pile up in a row, the next connectWithFallback falls back immediately.

Why "received a message" rather than time-alive: uptime is not evidence of usefulness — a middlebox can hold a socket open and drop every frame — while an inbound message proves the path works end to end. On a healthy connection that proof arrives within one RTT, because connectWithFallback already pings on open, so a legitimately flaky-but-working network keeps reconnecting as it does today.

The fallback is deliberately not memorized in sessionStorage here. The primary transport did connect, so the existing "only memorize LP if we never connected to primary" invariant stands and the next page load tries websockets again.

Tests

Two tests in assets/test/socket_test.js drive a fake transport under fake timers:

  • opens, then dies before any message, repeatedly → replaceTransport(LongPoll) after 3 connections. On main this test runs 11 connections in the same window and never falls back.
  • opens, delivers a message, then dies, repeatedly → no fallback, and the counter stays at 0.

npm test is green (180 passing, 3 skipped).

Where this came from

We hit this in production with users behind enterprise security appliances that allow the websocket upgrade and then kill the established tunnel. One customer's client logged roughly 100 open/close cycles against 11 message arrivals over a week: the socket kept opening, almost never carried anything, and the fallback never engaged, so those users had no working transport path at all.

We have a reproduction harness (a node middlebox proxy that kills sockets a fixed delay after open, plus a Playwright driver) and are glad to share it or run a patch through it.

Noticed, not fixed here

errorRef is only removed when the fallback engages, so a long-lived socket accumulates one error callback per reconnect. Out of scope for this fix — happy to send it separately if you'd like it.


Submitted on behalf of Rupert Deese by Claude, the AI engineering assistant at Gearflow.

`connectWithFallback` could only fall back before the primary transport
had ever opened: the fallback timer, and the error handler guarded by
`!established`. A websocket that opens and is then killed moments later
escaped both, because `connectWithFallback` runs again on every reconnect
and clears the pending fallback timer, so the client flapped forever and
longpoll was never tried. Some enterprise middleboxes behave exactly this
way: they let the handshake through, then kill the tunnel.

Count primary connections that opened but closed again without ever
delivering a message, and fall back once enough of them happen in a row.
A connection that carried a message resets the count. The fallback is not
memorized in this case, since the primary transport did connect.

Closes phoenixframework#6766

Co-Authored-By: Claude <noreply@anthropic.com>
@SteffenDE

Copy link
Copy Markdown
Member

So I've been thinking about this and I don't think we should go along with it for now. I'm worried that if we start to assume arbitrary middleboxes, we will end up in a rabbit hole of edge cases. Since this is the first time this comes up, I don't think it is a frequent enough problem to warrant a fix in Phoenix. If this comes up more frequently, we'll definitely reconsider.

For now, I'd suggest to implement your fallback in your app.js:

import {Socket, LongPoll} from "phoenix"
import {LiveSocket} from "phoenix_live_view"

const csrfToken = document
  .querySelector("meta[name='csrf-token']")
  .getAttribute("content")

const liveSocket = new LiveSocket("/live", Socket, {
  params: {_csrf_token: csrfToken}
})

const socket = liveSocket.getSocket()

const maxUnusableConnections = 3

let opened = false
let receivedMessage = false
let unusableConnections = 0
let usingFallback = false

socket.onOpen(() => {
  if(usingFallback) return

  opened = true
  receivedMessage = false
})

socket.onMessage(() => {
  if(usingFallback) return

  receivedMessage = true
  unusableConnections = 0
})

socket.onClose(event => {
  if(usingFallback) return

  if(opened && !receivedMessage && event?.code !== 1000){
    unusableConnections++
  }

  opened = false
  receivedMessage = false

  if(unusableConnections >= maxUnusableConnections){
    usingFallback = true
    liveSocket.replaceTransport(LongPoll)
  }
})

liveSocket.connect()
window.liveSocket = liveSocket

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

phoenix.js: connectWithFallback never falls back when the websocket dies shortly after opening

2 participants