feat(connect): survive WS port conflicts + clearer firewall guidance (#61)

Address operator review of the connect one-liner install:

- WS bind failure is no longer fatal: bind the listener eagerly and, on
  failure (e.g. :8081 already in use), log a warning and run without the
  sink, so a taken port can't crash-loop the container and tear down the
  DoubleZero tunnel (src/main.rs, src/sinks/ws.rs — bind/serve split).
- Installer WS port preflight: detect an already-bound WS port and offer
  to pick another port, disable the sink, or continue (scripts/connect*.sh).
- WS_BIND="" now works through the one-liner (forwarded even when empty),
  so the WS sink can be disabled without a hand-written docker run.
- Firewall guidance for default-deny-incoming hosts: admit the decapsulated
  inner multicast on the tunnel interface (ufw allow in on doublezero1),
  since allowing GRE alone isn't enough.

Docs + CHANGELOG updated.

Author: juan-malbeclabs

CHANGELOG.md

@@ -34,6 +34,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   `FEEDS` row depends on it.
 
 ### Changed
+- Installer (`scripts/connect*.sh`) usability fixes after review:
+  - **WebSocket port preflight**: before starting the container the installer checks whether the WS
+    port is already bound on the host and, interactively, offers to pick another port, disable the
+    sink, or continue (non-interactively it warns and continues — the bridge then runs without the
+    sink, tunnel unaffected).
+  - **`WS_BIND=""` now works through the one-liner**: `WS_BIND` is forwarded whenever it is *set*,
+    including set-but-empty, so the WS sink can be disabled straight from the pipe (previously only
+    non-empty values were relayed, forcing a hand-written `docker run`).
+  - **Firewall guidance for default-deny-incoming hosts**: the ufw/firewalld hints now note that
+    allowing GRE + UDP 44880 admits only the *outer* encapsulated packets — the decapsulated inner
+    multicast re-traverses `INPUT` on the tunnel interface (`doublezero1`) and must be allowed too
+    (`sudo ufw allow in on doublezero1`). Mirrored in `README.md` / `scripts/README.md`.
 - Public-feeder transport scaffolding extracted into a venue-generic `ingest::public_feeder`
   (a `PublicVenue` trait + one reconnecting run loop + shared decode helpers); Hyperliquid
   (`ingest::ws_feeder`) is the first implementor (#53). The four `dz_ws_feeder_*` metrics are now
@@ -49,6 +61,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
     Applied in both `src/main.rs` and the image `ENV`.
 
 ### Fixed
+- A taken WebSocket-sink port no longer takes the whole bridge down. A bind failure on `--ws-bind`
+  (e.g. the default `0.0.0.0:8081` colliding with a pre-existing `127.0.0.1:8081` listener) was
+  fatal: the process exited, the container's `--restart unless-stopped` restarted it, doublezerod
+  and the DoubleZero tunnel came down with it, and — since `doublezero connect multicast` runs only
+  once from the installer — the tunnel never re-established (status stuck `disconnected`, the real
+  cause buried in the restart loop). The listener is now bound eagerly (`sinks::ws::bind`, split
+  from `serve`) and a bind failure is logged and skipped: the bridge runs without the sink while
+  the tunnel and shred forwarding keep going (`src/main.rs`, `src/sinks/ws.rs`).
 - Installer pre-flight access-pass check (`scripts/connect*.sh`) hardened after review:
   - A confirmed miss (an identity with no pass for the host IP or `0.0.0.0`) now only hard-aborts
     when the public IP was **explicitly supplied** via `DZ_CLIENT_IP`; when the IP was only

CLAUDE.md

@@ -40,7 +40,10 @@ One WS-server task plus **one receiver task per selected feed** share a single
 instrument snapshot. `main.rs` selects feeds (`--feed`, or all of `ingest::feeds::FEEDS` by
 default), builds the shared `Arbiter` around the broadcast `Sender`, spawns the receivers into a
 `JoinSet` (and the optional public WS input feeders), and exits if the WS server, any receiver, or a
-feeder task returns.
+feeder task returns. **Exception:** the WS listener is bound *eagerly* before its task is spawned,
+and a bind failure (e.g. the port is already taken) is logged as a warning and the process runs
+without the sink — never fatal, so a taken WS port can't crash-loop the container and tear down the
+DoubleZero tunnel.
 
 Ingest has **two source transports** that converge on one shared `arbiter` before the broadcast:
 the always-on DZ Edge **multicast** receivers, and optional **public WebSocket** feeders (off by
@@ -163,7 +166,9 @@ Modules are grouped by role under `src/`:
   `depth` per symbol** (full state), then streams quotes/trades/midpoints/depth. Implements the
   PROTOCOL.md v1 surface: optional per-client subscribe/unsubscribe filtering (empty filter list =
   firehose), app ping/pong + server WS-ping heartbeat with idle-timeout reaping, and the limits
-  (max clients/subs/inbound-rate, broadcast backpressure where a slow client drops oldest).
+  (max clients/subs/inbound-rate, broadcast backpressure where a slow client drops oldest). The
+  listener is bound via `ws::bind()` (separate from `ws::serve()`) so `main.rs` can treat a bind
+  failure as non-fatal — a taken port disables the sink but leaves the tunnel running.
 - **`model.rs`** — wire types (`NormalizedQuote`/`NormalizedTrade`/`NormalizedMidpoint`/
   `NormalizedDepth`/`NormalizedInstrument`, the `FeedMessage` tagged enum) and the `now_ns()` /
   `now_mono_ns()` clocks. The `InstrumentSnapshot` and `DepthSnapshot` are both keyed by

README.md

@@ -62,7 +62,10 @@ What the script does:
 
 Requirements: **Linux/amd64**, GRE connectivity (allow IP protocol 47 at the cloud provider; on
 AWS disable the ENI source/dest check), and a host public IP authorized onchain for the chosen
-environment. See [scripts/README.md](scripts/README.md) for the full requirements and caveats.
+environment. If the host runs a **default-deny-incoming** firewall, also admit the decapsulated
+inner multicast on the tunnel interface (e.g. `sudo ufw allow in on doublezero1`) — allowing GRE
+alone isn't enough, since the inner UDP re-traverses `INPUT` on `doublezero1` after decapsulation.
+See [scripts/README.md](scripts/README.md) for the full requirements and caveats.
 
 ## Configure (override the one-liner)
 
@@ -100,9 +103,14 @@ struct in [`src/main.rs`](src/main.rs); per-feature config lives in the [docs](d
 > `warn`. Set `RUST_LOG=debug` for verbose output. The installer also caps the container log on
 > disk (json-file driver, ~60 MB ceiling) so it can't fill the host.
 
-> **Limitation:** only **non-empty** values are forwarded, so you can't pass an *empty* override
-> (e.g. `WS_BIND=""` to disable the WebSocket sink) through the installer. For that, run a
-> hand-written `docker run` — see [Self-hosting](docs/self-hosting.md).
+> **Note:** only **non-empty** values are forwarded, with one exception: `WS_BIND` is forwarded
+> whenever it is *set* — including set-but-empty — so `WS_BIND="" curl … | bash` disables the
+> WebSocket sink straight from the one-liner. The installer also runs a host-side **port
+> preflight**: if the WS port is already taken it warns and (interactively) offers to pick another
+> port, disable the sink, or continue. Even if a conflict slips through, a WS bind failure is
+> non-fatal — the bridge logs it and keeps running (the tunnel and shred forwarding are
+> unaffected). A hand-written `docker run` is still an option — see
+> [Self-hosting](docs/self-hosting.md).
 
 Examples:
 

docs/output-sinks.md

@@ -45,6 +45,8 @@ quotes/trades/midpoints/depth, with optional per-client subscribe/unsubscribe fi
 heartbeat/limit enforcement.
 
 > **Note:** when running via the installer one-liner, set these as env vars before the pipe (or
-> with `docker run -e`). The `WS_BIND=""` disable case can't go through the installer (it only
-> forwards non-empty values) — run a hand-written `docker run` for that. See
+> with `docker run -e`). `WS_BIND=""` (disable the sink) **does** go through the installer —
+> `WS_BIND` is forwarded whenever it is set, including set-but-empty — and the installer runs a
+> host-side port preflight that flags a taken WS port before starting the container. A taken port
+> is non-fatal regardless: the bridge logs the bind failure and runs without the sink. See
 > [Configure](../README.md#configure-override-the-one-liner).

scripts/README.md

@@ -118,6 +118,16 @@ The bridge serves:
   kernel tunnels.
 - **GRE connectivity.** On a cloud host you must also allow IP protocol 47 at the provider level
   (and, on AWS, disable the ENI source/dest check) — the script warns but can't fix this for you.
+- **Host firewall (default-deny-incoming).** Allowing GRE (protocol 47) + UDP 44880 admits the
+  *outer* encapsulated packets, but the kernel decapsulates them and the *inner* multicast UDP
+  re-traverses the `INPUT` chain on the tunnel interface (`doublezero1`), where a default-deny
+  policy drops it. Also admit that interface, e.g. `sudo ufw allow in on doublezero1` (or place
+  `doublezero1` in a firewalld trusted zone). The installer warns about this when it detects an
+  active ufw/firewalld.
+- **WebSocket port.** The bridge serves the WS sink on `:8081` by default. If that port is already
+  in use the installer's preflight flags it and (interactively) offers to pick another port,
+  disable the sink (`WS_BIND=""`), or continue; a bind failure is non-fatal either way (the tunnel
+  is unaffected).
 - **Access pass.** `doublezero connect` requires the host's public IP to be authorized onchain for
   the chosen environment; otherwise the tunnel won't come up. The installer now checks this **up
   front** (step 2): if the identity has no access pass for the host's public IP or `0.0.0.0` it

scripts/connect-devnet.sh

@@ -377,14 +377,80 @@ if [ "$($SUDO sysctl -n net.core.rmem_max 2>/dev/null || echo 0)" -lt "$RECV_BUF
     || warn "Could not raise net.core.rmem_max to $RECV_BUF_MAX (the bridge will use the host's current ceiling)."
 fi
 
-# best-effort firewall hints (don't auto-edit the user's firewall)
+# best-effort firewall hints (don't auto-edit the user's firewall). Beyond GRE + the liveness
+# port, a *default-deny-incoming* host also has to admit the decapsulated inner traffic: the GRE
+# rule lets the outer encapsulated packets in, but once the kernel decapsulates them the inner
+# multicast UDP re-traverses the INPUT chain on the tunnel interface (doublezero1) and is dropped
+# unless that interface is allowed. The interface only exists once the tunnel is up, but the rule
+# can be added ahead of time.
 if command -v ufw >/dev/null 2>&1 && $SUDO ufw status 2>/dev/null | grep -qi "Status: active"; then
-  warn "ufw is active: ensure IP protocol 47 (GRE) and UDP $LIVENESS_UDP_PORT are allowed."
+  warn "ufw is active: allow GRE (IP protocol 47) and UDP $LIVENESS_UDP_PORT, e.g.
+       sudo ufw allow proto gre from any to any
+       sudo ufw allow $LIVENESS_UDP_PORT/udp
+     If your policy is default-deny-incoming, also admit the decapsulated inner multicast on the tunnel:
+       sudo ufw allow in on doublezero1"
 fi
 if command -v firewall-cmd >/dev/null 2>&1 && $SUDO firewall-cmd --state 2>/dev/null | grep -qi running; then
-  warn "firewalld is running: ensure GRE (protocol 47) and UDP $LIVENESS_UDP_PORT are allowed."
+  warn "firewalld is running: allow GRE (protocol 47) and UDP $LIVENESS_UDP_PORT. If your default zone denies incoming, also place the tunnel interface in a trusted zone once it exists:
+       sudo firewall-cmd --zone=trusted --change-interface=doublezero1"
 fi
 
+# ----------------------------------------------------------------------------
+# 4b. WebSocket sink port preflight (host-side)
+# ----------------------------------------------------------------------------
+# A taken WS port would make the bridge fail to bind that listener. The bridge now degrades
+# gracefully (it logs a warning and runs without the sink — the DoubleZero tunnel is unaffected),
+# but catch the conflict here too so the operator can pick another port or disable the sink up
+# front instead of silently losing the WS output. Also defines ws_disabled/ws_port, reused by the
+# env passthrough and the final status print.
+
+# The WS sink is disabled when WS_BIND is set-but-empty (WS_BIND="").
+ws_disabled() { [ "${WS_BIND+set}" = set ] && [ -z "${WS_BIND}" ]; }
+# The WS port: WS_BIND's trailing :port when set non-empty, else the default WS_PORT.
+ws_port() { if [ -n "${WS_BIND:-}" ]; then printf '%s' "${WS_BIND##*:}"; else printf '%s' "$WS_PORT"; fi; }
+
+# Is a TCP port already bound on this host? Prefer ss, fall back to netstat; if neither exists we
+# can't tell (rc 2 -> skip). A ':'/'.' immediately before the port anchors the match so e.g. 8081
+# doesn't match 18081.
+port_in_use() {
+  local p="$1"
+  if command -v ss >/dev/null 2>&1; then
+    $SUDO ss -H -ltn 2>/dev/null | awk '{print $4}' | grep -qE "[:.]${p}\$"
+  elif command -v netstat >/dev/null 2>&1; then
+    $SUDO netstat -ltn 2>/dev/null | awk '{print $4}' | grep -qE "[:.]${p}\$"
+  else
+    return 2
+  fi
+}
+
+preflight_ws_port() {
+  ws_disabled && return 0                       # sink off -> nothing to bind
+  local p; p="$(ws_port)"
+  case "$p" in *[!0-9]*|'') return 0;; esac      # not a plain numeric port -> let the bridge validate WS_BIND
+  port_in_use "$p"; local rc=$?
+  if [ "$rc" -eq 2 ]; then
+    warn "Can't check whether TCP port $p is free (no ss/netstat installed); if it's in use the WS sink won't bind (the tunnel is unaffected)."
+    return 0
+  fi
+  [ "$rc" -ne 0 ] && return 0                    # free
+  # In use — name the holder when ss can show it.
+  local who=""
+  command -v ss >/dev/null 2>&1 && who="$($SUDO ss -Hltnp 2>/dev/null | awk -v p=":${p}\$" '$4 ~ p {print $NF; exit}')"
+  warn "TCP port $p is already in use${who:+ ($who)}; the WS market-data sink can't bind there."
+  if [ -r "$TTY" ] && [ "$DZ_ASSUME_YES" != 1 ]; then
+    local choice; choice="$(ask 'WS port in use — [p]ick another port, [d]isable the WS sink, or [c]ontinue anyway' 'c')"
+    case "$choice" in
+      p|P) local np; np="$(ask 'New WS port' '8181')"; WS_BIND="0.0.0.0:${np}"; WS_PORT="$np"
+           info "WS sink will use 0.0.0.0:${np}."; preflight_ws_port ;;   # re-check the new choice
+      d|D) WS_BIND=""; info "WS sink disabled (WS_BIND=\"\")." ;;
+      *)   warn "Continuing; the bridge starts without the WS sink (the tunnel is unaffected)." ;;
+    esac
+  else
+    warn "Continuing non-interactively; the bridge starts without the WS sink (the tunnel is unaffected). Re-run with WS_BIND=<host>:<free-port> to serve it, or WS_BIND=\"\" to disable it explicitly."
+  fi
+}
+preflight_ws_port
+
 # ----------------------------------------------------------------------------
 # 5. cloud detection -> warn about provider-level firewall (script can't fix)
 # ----------------------------------------------------------------------------
@@ -427,11 +493,10 @@ mount_args=()
 [ "$KEY_SRC" = file ] && mount_args=(-v "$KEYFILE":"$KEYPAIR_DEST":"$MNT_OPT")
 # Relay bridge env vars to the container. The bridge reads every flag from an env var, so this is
 # the only wiring needed to tune the WS sink, narrow feeds, or raise log level — no per-feature
-# logic here. Only non-empty values are forwarded (so an empty override like WS_BIND="" to disable
-# the WS sink can't be passed this way — use a hand-written `docker run` for that edge case).
+# logic here. Only non-empty values are forwarded, with one exception (WS_BIND, below).
 PASSTHROUGH=(
   DZ_FEEDS DZ_IFACE DZ_RECV_BUF
-  WS_BIND WS_HEARTBEAT_SECS WS_IDLE_TIMEOUT_SECS WS_MAX_CLIENTS
+  WS_HEARTBEAT_SECS WS_IDLE_TIMEOUT_SECS WS_MAX_CLIENTS
   WS_MAX_SUBS WS_MAX_INBOUND_PER_MIN WS_BROADCAST_CAPACITY
   DZ_SHRED_DEDUP_MODE DZ_SHRED_RPC_URL DZ_SHRED_FORWARD DZ_SHRED_SOURCES
   DZ_SHRED_CODE_PREFIX DZ_SHRED_PORT DZ_SHRED_DEDUP_WINDOW_SLOTS
@@ -441,6 +506,10 @@ env_args=()
 for v in "${PASSTHROUGH[@]}"; do
   [ -n "${!v:-}" ] && env_args+=(-e "$v=${!v}")
 done
+# WS_BIND is the one var forwarded whenever it is *set* — including set-but-empty (WS_BIND="") —
+# so the WS sink can be disabled straight from the one-liner (an empty --ws-bind turns the sink
+# off in the bridge). The preflight above may also have set/cleared it in response to a conflict.
+[ "${WS_BIND+set}" = set ] && env_args+=(-e "WS_BIND=${WS_BIND}")
 $SUDO docker run -d --name "$DZ_NAME" \
   --restart unless-stopped \
   --stop-timeout 60 \
@@ -483,7 +552,7 @@ EXEC_TTY=""; [ -t 1 ] && EXEC_TTY="-t"
 # user onchain for $DZ_ENV. If this errors with an access-pass message, that
 # provisioning step still needs to happen. Once the tunnel is up, the bridge
 # self-heals onto the doublezero1 interface within ~30s and quotes begin flowing.
-$SUDO docker exec $EXEC_TTY "$DZ_NAME" doublezero connect multicast || warn "connect failed (often: no access pass for this IP, or provider firewall/NAT). See notes above."
+$SUDO docker exec $EXEC_TTY "$DZ_NAME" doublezero connect multicast || warn "connect failed (often: no access pass for this IP; provider firewall/NAT; or a default-deny host firewall dropping the decapsulated inner multicast on doublezero1). See the firewall notes above."
 
 # ----------------------------------------------------------------------------
 # 8. status + management hints
@@ -496,8 +565,12 @@ echo
 spin_sleep 5 "Waiting for the tunnel to settle"
 $SUDO docker exec "$DZ_NAME" doublezero status || true
 echo
-info "Done. The bridge is serving normalized quotes:"
-echo "  WebSocket : ws://${HOST_IP}:${WS_PORT}            # normalized quotes (see PROTOCOL.md)"
+if ws_disabled; then
+  info "Done. The WebSocket sink is disabled (WS_BIND=\"\"); the bridge ingests DZ Edge and (if configured) forwards shreds, but serves no WebSocket."
+else
+  info "Done. The bridge is serving normalized quotes:"
+  echo "  WebSocket : ws://${HOST_IP}:${WS_PORT}            # normalized quotes (see PROTOCOL.md)"
+fi
 echo
 info "Manage with:"
 echo "  sudo docker logs -f $DZ_NAME                            # bridge + daemon logs"