VScode dropping connection to the cluster

[Naeemkh]

Issue with dead port

[mmshad]

I'm adding the Claude skill here:

---
name: vscode-ssh-doctor
description: Diagnose and fix VS Code Remote-SSH connection failures on hosts that use SSH ControlMaster multiplex. Use when the user reports "dynamic port forwarding failed", "Address already in use", "Session open refused by peer", VS Code suddenly prompting for password on a host that previously connected silently, or VS Code Remote-SSH no longer connecting to a cluster. Covers leaked dynamic-forward listeners on the multiplex master, MaxSessions exhaustion, and stale master sockets.
---

# VS Code Remote-SSH Doctor

Investigate VS Code Remote-SSH failures on hosts using SSH multiplex (ControlMaster). **Always confirm with the user before running any cleanup command.**

## Background — what usually breaks

VS Code Remote-SSH uses `ssh -D <port>` for its SOCKS tunnel. The `-D` listener opens on the multiplex **master**, not the spawned slave. If VS Code's connection fails partway, the master keeps the listener bound — until the master exits or the forward is explicitly canceled. After several failed attempts the master accumulates leaked listeners, eventually colliding with whatever local port VS Code picks: `bind ...: Address already in use` → "dynamic port forwarding failed!".

Concurrent failure mode: too many terminal tabs through one multiplex master can exhaust the server's `MaxSessions` cap. New sessions then fail with `Session open refused by peer`, and VS Code falls back to fresh auth (unexpected password+OTP prompt on a host that used to connect silently).

## Diagnose

Identify the multiplex host. If the user didn't say, read `~/.ssh/config` and ask which one. Then gather evidence in parallel — fix nothing yet.

```bash
ssh -O check <host>                                # master alive? capture PID
ps aux | grep "ssh <host>" | grep -v grep         # slave (terminal-tab) count
lsof -p <master_pid> | grep LISTEN                # leaked dynamic forwards
latest=$(ls -td ~/Library/Application\ Support/Code/logs/*/ | head -1)
find "$latest" -name "*Remote*SSH*.log" | head -1  # path to most recent VS Code log

Read that log. Match the symptom:

Log line	Cause	Recommended fix
bind [127.0.0.1]:N: Address already in use	Leaked dynamic forwards on master	Surgical cancel loop
Could not request local forwarding	Same as above	Surgical cancel loop
Session open refused by peer	Server MaxSessions exhausted by slave count	Close terminal tabs
ControlSocket ... already exists, disabling multiplexing	Stale or degraded master socket	Nuclear restart
Unexpected password / OTP prompt	Multiplex not being used	Check master state, then nuclear if needed

Fix

Surgical (preferred): cancel leaked dynamic forwards

Releases stuck local ports without killing the master or any terminal session.

Ask the user first. Show the list of ports and the exact loop you'll run:

for port in <port1> <port2> ...; do
  ssh -O cancel -D "$port" <host>
done

Verify after: lsof -p <master_pid> | grep LISTEN should be empty.

Nuclear: exit and restart the master

Use only when surgical doesn't apply or the master is in a confused state. Ask the user first — this disconnects every terminal tab on the host.

ssh -O exit <host>
ls -la ~/.ssh/controlmasters/   # confirm socket file is gone
ssh <host>                       # re-auth from terminal

Reduce slave count

If Session open refused by peer is the failure AND there are many slave processes, ask the user to close terminal tabs (target ~3 max) to free server-side session slots. Each tab consumes one slot; VS Code needs at least one free.

Confirmation rule (hard requirement)

Never run ssh -O cancel or ssh -O exit without explicit user approval. Show:

• the exact command(s),
• the resources affected (ports being released, sessions being closed),
• and wait for "go" / "yes" / equivalent before executing.

The diagnostic commands (ssh -O check, lsof, ps, log reads) are safe to run unprompted.