Skip to content

fix(ssh-for-commander): fix clissh and gossh goroutine leaks#36

Merged
090809 merged 7 commits into
mainfrom
fix/ssh-for-commander-goroutine-leaks
Jul 16, 2026
Merged

fix(ssh-for-commander): fix clissh and gossh goroutine leaks#36
090809 merged 7 commits into
mainfrom
fix/ssh-for-commander-goroutine-leaks

Conversation

@faroyam

@faroyam faroyam commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

clissh

pkg/ssh/clissh/process/executor.go

  • Added storage for read-side pipe file descriptors.
  • Close write, read, and handler pipe ends on Stop, timeout, and normal exit.
  • Made Stop idempotent using sync.Once and atomics.
  • The timeout goroutine now terminates after the process exits.
  • readFromStreams exits on os.ErrClosed instead of looping.

pkg/ssh/clissh/tunnel.go

  • Added storage for stdout and stdin pipe file descriptors.
  • Tunnel.Stop() now closes pipes and kills the process even when HealthMonitor was not started.
  • HealthMonitor exits when the tunnel has already been stopped.
  • The wait goroutine closes the pipe set for its own run without affecting the next run.

gossh

pkg/ssh/gossh/client.go

  • Added synchronized lifecycle state for Start, Stop, and keepalive restart.
  • Stop() now marks the client as stopped before cleanup, so new sessions fail fast after stop.
  • Repeated Start() now closes the previous SSH client, sessions, keepalive goroutine, connections, local kube-proxies, and SSH agent socket connection before reconnecting.
  • Keepalive now has its own stopCh and doneCh; stopKeepAlive() waits for shutdown with a timeout instead of blocking forever.
  • Keepalive restart() no longer resurrects a stopped client and ignores stale restarts from an old keepalive run.
  • SSH sessions and SSH client access now use snapshots under mutexes to avoid races.
  • Local kubeProxies access is synchronized and reset during reconnect/stop.
  • Remote kube-proxy cleanup commands can still create SSH sessions during stop via an internal allowStopped path.

pkg/ssh/gossh/keepalive.go

  • Keepalive checks now use thread-safe SSH client/session snapshots.
  • Sleep between retries is interruptible through stopCh.
  • Session keepalive iteration no longer reads sshSessionsList directly.

pkg/ssh/gossh/command.go

  • Added internal newSSHCommand(..., allowStopped) helper.
  • Public commands still reject stopped clients, while stop-cleanup commands can run during client shutdown.

pkg/ssh/gossh/tunnel.go

  • Tunnel now gets the SSH client through a safe snapshot.
  • Concurrent Stop() no longer causes a nil pointer panic while dialing a remote tunnel endpoint.

pkg/ssh/gossh/reverse-tunnel.go

  • ReverseTunnel now gets the SSH client through a safe snapshot.
  • Concurrent Stop() now returns a normal error instead of panicking on GetClient().Listen(...).

pkg/ssh/base/kubeproxy/kube_proxy.go

  • Made the proxy port readiness timeout test-configurable.
  • Stop the proxy command when port readiness times out.
  • Stop the proxy command when the port-ready signal arrives with an empty port.

@faroyam
faroyam force-pushed the fix/ssh-for-commander-goroutine-leaks branch from 5bdc727 to 4b6228e Compare July 3, 2026 06:27
Signed-off-by: Ilya Mayorov <ilya.mayorov@flant.com>
@faroyam
faroyam force-pushed the fix/ssh-for-commander-goroutine-leaks branch 5 times, most recently from a1ea139 to a781c95 Compare July 3, 2026 11:47
Signed-off-by: Ilya Mayorov <ilya.mayorov@flant.com>
@faroyam
faroyam force-pushed the fix/ssh-for-commander-goroutine-leaks branch from a781c95 to 7212f0d Compare July 3, 2026 12:31
@faroyam faroyam changed the title fix(ssh-for-commander): fix clissh goroutine leaks fix(ssh-for-commander): fix clissh and gossh goroutine leaks Jul 12, 2026
090809 added 2 commits July 16, 2026 11:01
…-goroutine-leaks

# Conflicts:
#	pkg/ssh/clissh/process/executor.go
#	pkg/ssh/clissh/tunnel.go
#	pkg/ssh/gossh/client.go
#	pkg/ssh/gossh/command.go
…for-commander-goroutine-leaks

Signed-off-by: pallam <kargin.go@bonch.dev>
@090809
090809 marked this pull request as ready for review July 16, 2026 09:51
@090809
090809 self-requested a review as a code owner July 16, 2026 09:51
090809 added 3 commits July 16, 2026 12:55
…for-commander-goroutine-leaks

Signed-off-by: pallam <kargin.go@bonch.dev>
…ing of race conditions in tests.

Signed-off-by: pallam <kargin.go@bonch.dev>
…finding more robust in Makefile

Signed-off-by: pallam <kargin.go@bonch.dev>
@090809
090809 merged commit 793108b into main Jul 16, 2026
4 checks passed
@090809
090809 deleted the fix/ssh-for-commander-goroutine-leaks branch July 16, 2026 12:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants