[Bug]: SSH Tunnel does not get closed if server is rebooted

[robertpenz]

Describe the bug

Reconnecting to a Database does not work if using SSH Tunnel

To Reproduce

I connect via SSH to a Mysql server. If the server gets rebooted I get a message in Tabularis that the server is not reachable anymore and the orange connection icon is removed. If I than try to reconnect, Tabularis informs me that the port is already in use. Restarting Tabularis makes it work again.

OS Version

Fedora 44

Tabularis Version

0.13.0

Relevant Log Output

[debba]

Thanks for the detailed report — that was enough to track it down.

The problem wasn't the reconnect itself but tunnel cleanup. Tabularis caches SSH tunnels in a global map and reuses them, but they were never being stopped or removed. So when your server rebooted, the tunnel died while the local forward port was still held by the leftover ssh process. On reconnect we picked up that stale entry, pointed at the dead port, and got "port already in use" — and restarting the app worked only because it cleared the map.

The fix (#407) tears the tunnel down at the points where the connection actually goes away:

• the health check that drops the connection after the server becomes unreachable now also kills its tunnel,
• manual disconnect does the same,
• and as a safety net, a cached tunnel is checked for liveness before reuse and recreated if it's dead.

Since you can reproduce this reliably, would you be able to build from the PR branch (fix/ssh-tunnel-reconnect) and confirm on your side? The exact case to check: connect to MySQL over the SSH tunnel, reboot the remote server, wait for the connection indicator to disappear, then reconnect — it should now come back up without needing to restart Tabularis.

[robertpenz]

Works basically but I get multible requests for a pin and after the login it gets an connection close messagen and after all that it works. but not restart needed, here the log

[Logger] Capturing logger initialized successfully with level: Info
[LOG] [2026-07-03 12:45:33.032] [INFO] tabularis_lib - Tabularis application starting...
[LOG] [2026-07-03 12:45:33.034] [INFO] tabularis_lib - Debug mode disabled - standard logging active
[LOG] [2026-07-03 12:45:33.344] [INFO] tabularis_lib::drivers::registry - Registering driver: MySQL (mysql)
[LOG] [2026-07-03 12:45:33.345] [INFO] tabularis_lib::drivers::registry - Registering driver: PostgreSQL (postgres)
[LOG] [2026-07-03 12:45:33.345] [INFO] tabularis_lib::drivers::registry - Registering driver: SQLite (sqlite)
[LOG] [2026-07-03 12:45:33.346] [INFO] tabularis_lib::health_check - Health check: starting ping loop with interval 30s
[LOG] [2026-07-03 12:45:33.349] [INFO] tabularis_lib::ai_approval_watcher - AI approval watcher started on /home/user/.config/tabularis/pending_approvals
[Keychain] Getting SSH password for 78c0b378-999e-4d39-8b4a-f4a165200af5
[Keychain] Error getting SSH password for 78c0b378-999e-4d39-8b4a-f4a165200af5: No matching entry found in secure storage
[Keychain] Getting SSH key passphrase for 78c0b378-999e-4d39-8b4a-f4a165200af5
[Keychain] Error getting SSH key passphrase for 78c0b378-999e-4d39-8b4a-f4a165200af5: No matching entry found in secure storage
[LOG] [2026-07-03 12:49:38.695] [INFO] tabularis_lib::commands - Creating new SSH tunnel for root@myserver:22
[SSH Tunnel] New Request: Host=myserver, Port=22, User=root, UseSystemSSH=true, AllowPrompt=true
[SSH Tunnel] Assigned Local Port: 38905
[SSH Tunnel] Executing: ssh ["-N", "-L", "127.0.0.1:38905:localhost:3306", "-o", "StrictHostKeyChecking=accept-new", "-o", "BatchMode=no", "root@myserver"]
[SSH Tunnel] In-app askpass bridge active at /tmp/tabularis-askpass-3226f71ddc6a42a3b39b0330dbd554db.sock
[SSH Tunnel] Tunnel established successfully on port 38905
[LOG] [2026-07-03 12:49:43.416] [INFO] tabularis_lib::commands - SSH tunnel created successfully on port 38905
[LOG] [2026-07-03 12:49:43.417] [INFO] tabularis_lib::pool_manager - Creating new MySQL connection pool for: root@Some("127.0.0.1") (key: mysql:127.0.0.1:38905:root:information_schema:ssl:preferred::::clear:false:pipes:true)
[LOG] [2026-07-03 12:49:43.469] [INFO] tabularis_lib::pool_manager - MySQL connection pool created successfully for: information_schema (key: mysql:127.0.0.1:38905:root:information_schema:ssl:preferred::::clear:false:pipes:true)
[LOG] [2026-07-03 12:49:51.571] [INFO] tabularis_lib::commands - Saving new connection: vmirakulix
[LOG] [2026-07-03 12:49:51.572] [INFO] tabularis_lib::commands - Connection saved successfully: vmirakulix (ID: bc7170ed-4be2-450d-8615-7b6926787728)
[LOG] [2026-07-03 12:49:55.565] [INFO] tabularis_lib::commands - Testing connection to database: integritycheck
[LOG] [2026-07-03 12:49:55.571] [INFO] tabularis_lib::commands - Connection test successful for database: integritycheck
[LOG] [2026-07-03 12:49:55.593] [INFO] tabularis_lib::health_check - Health check: registering connection bc7170ed-4be2-450d-8615-7b6926787728
[LOG] [2026-07-03 12:49:55.612] [INFO] tabularis_lib::commands - Fetching tables for connection: bc7170ed-4be2-450d-8615-7b6926787728
[LOG] [2026-07-03 12:49:55.612] [INFO] tabularis_lib::commands - Fetching views for connection: bc7170ed-4be2-450d-8615-7b6926787728
[LOG] [2026-07-03 12:49:55.613] [INFO] tabularis_lib::pool_manager - Creating new MySQL connection pool for: root@Some("127.0.0.1") (key: mysql:conn:bc7170ed-4be2-450d-8615-7b6926787728:integritycheck:ssl:preferred::::clear:false:pipes:true)
[LOG] [2026-07-03 12:49:55.613] [INFO] tabularis_lib::pool_manager - Creating new MySQL connection pool for: root@Some("127.0.0.1") (key: mysql:conn:bc7170ed-4be2-450d-8615-7b6926787728:integritycheck:ssl:preferred::::clear:false:pipes:true)
[LOG] [2026-07-03 12:49:55.613] [INFO] tabularis_lib::commands - Fetching routines for connection: bc7170ed-4be2-450d-8615-7b6926787728
[LOG] [2026-07-03 12:49:55.613] [INFO] tabularis_lib::pool_manager - Creating new MySQL connection pool for: root@Some("127.0.0.1") (key: mysql:conn:bc7170ed-4be2-450d-8615-7b6926787728:integritycheck:ssl:preferred::::clear:false:pipes:true)
[LOG] [2026-07-03 12:49:55.613] [INFO] tabularis_lib::commands - Fetching triggers for connection: bc7170ed-4be2-450d-8615-7b6926787728
[LOG] [2026-07-03 12:49:55.613] [INFO] tabularis_lib::pool_manager - Creating new MySQL connection pool for: root@Some("127.0.0.1") (key: mysql:conn:bc7170ed-4be2-450d-8615-7b6926787728:integritycheck:ssl:preferred::::clear:false:pipes:true)
[LOG] [2026-07-03 12:49:55.659] [INFO] tabularis_lib::pool_manager - MySQL connection pool created successfully for: integritycheck (key: mysql:conn:bc7170ed-4be2-450d-8615-7b6926787728:integritycheck:ssl:preferred::::clear:false:pipes:true)
[LOG] [2026-07-03 12:49:55.659] [INFO] tabularis_lib::pool_manager - MySQL connection pool created successfully for: integritycheck (key: mysql:conn:bc7170ed-4be2-450d-8615-7b6926787728:integritycheck:ssl:preferred::::clear:false:pipes:true)
[LOG] [2026-07-03 12:49:55.659] [INFO] tabularis_lib::pool_manager - MySQL connection pool created successfully for: integritycheck (key: mysql:conn:bc7170ed-4be2-450d-8615-7b6926787728:integritycheck:ssl:preferred::::clear:false:pipes:true)
[LOG] [2026-07-03 12:49:55.660] [INFO] tabularis_lib::pool_manager - MySQL connection pool created successfully for: integritycheck (key: mysql:conn:bc7170ed-4be2-450d-8615-7b6926787728:integritycheck:ssl:preferred::::clear:false:pipes:true)
[LOG] [2026-07-03 12:49:55.714] [INFO] tabularis_lib::commands - Retrieved 3 tables from integritycheck
[LOG] [2026-07-03 12:49:55.714] [INFO] tabularis_lib::commands - Retrieved 1 triggers
[LOG] [2026-07-03 12:49:55.714] [INFO] tabularis_lib::commands - Retrieved 0 views from integritycheck
[LOG] [2026-07-03 12:50:03.347] [WARN] tabularis_lib::commands - Discarding dead SSH tunnel for root@myserver:22:localhost->3306, recreating
[SSH Tunnel] Removing tunnel 'root@myserver:22:localhost->3306' (local port 38905)
[LOG] [2026-07-03 12:50:03.347] [INFO] tabularis_lib::commands - Creating new SSH tunnel for root@myserver:22
[SSH Tunnel] New Request: Host=myserver, Port=22, User=root, UseSystemSSH=true, AllowPrompt=true
[SSH Tunnel] Assigned Local Port: 35071
[SSH Tunnel] Executing: ssh ["-N", "-L", "127.0.0.1:35071:localhost:3306", "-o", "StrictHostKeyChecking=accept-new", "-o", "BatchMode=no", "root@myserver"]
[SSH Tunnel] In-app askpass bridge active at /tmp/tabularis-askpass-122c339daf764b609fcad0ecd3533252.sock
[SSH Tunnel Error] SSH process exited prematurely with status: exit status: 255.
Stderr: ssh: connect to host myserver port 22: Connection refused
Stdout: 
[SSH Tunnel Error] System SSH failed: SSH process exited prematurely with status: exit status: 255.
Stderr: ssh: connect to host myserver port 22: Connection refused
Stdout: 
[Connection Error] SSH Tunnel setup failed: SSH process exited prematurely with status: exit status: 255.
Stderr: ssh: connect to host myserver port 22: Connection refused
Stdout: 
[LOG] [2026-07-03 12:50:03.451] [WARN] tabularis_lib::health_check - Health check: ping failed for bc7170ed-4be2-450d-8615-7b6926787728 (1/2): SSH process exited prematurely with status: exit status: 255.
Stderr: ssh: connect to host myserver port 22: Connection refused
Stdout: 
[LOG] [2026-07-03 12:50:33.349] [INFO] tabularis_lib::commands - Creating new SSH tunnel for root@myserver:22
[SSH Tunnel] New Request: Host=myserver, Port=22, User=root, UseSystemSSH=true, AllowPrompt=true
[SSH Tunnel] Assigned Local Port: 40823
[SSH Tunnel] Executing: ssh ["-N", "-L", "127.0.0.1:40823:localhost:3306", "-o", "StrictHostKeyChecking=accept-new", "-o", "BatchMode=no", "root@myserver"]
[SSH Tunnel] In-app askpass bridge active at /tmp/tabularis-askpass-87c6590f8dce4615a9e556f98e5d0e9a.sock
[SSH Tunnel] Tunnel established successfully on port 40823
[LOG] [2026-07-03 12:50:36.965] [INFO] tabularis_lib::commands - SSH tunnel created successfully on port 40823
[LOG] [2026-07-03 12:50:36.965] [INFO] sqlx_core::pool::inner - ping on idle connection returned error error=error communicating with database: expected to read 4 bytes, got 0 bytes at EOF
[LOG] [2026-07-03 12:50:41.966] [WARN] tabularis_lib::health_check - Health check: ping failed for bc7170ed-4be2-450d-8615-7b6926787728 (2/2): Ping timed out
[LOG] [2026-07-03 12:50:41.967] [ERROR] tabularis_lib::health_check - Health check: connection bc7170ed-4be2-450d-8615-7b6926787728 exceeded failure threshold, disconnecting
[LOG] [2026-07-03 12:50:41.967] [INFO] tabularis_lib::health_check - Health check: unregistering connection bc7170ed-4be2-450d-8615-7b6926787728
[SSH Tunnel] Removing tunnel 'root@myserver:22:localhost->3306' (local port 40823)
[LOG] [2026-07-03 12:50:41.969] [INFO] tabularis_lib::commands - Creating new SSH tunnel for root@myserver:22
[SSH Tunnel] New Request: Host=myserver, Port=22, User=root, UseSystemSSH=true, AllowPrompt=true
[SSH Tunnel] Assigned Local Port: 38861
[SSH Tunnel] Executing: ssh ["-N", "-L", "127.0.0.1:38861:localhost:3306", "-o", "StrictHostKeyChecking=accept-new", "-o", "BatchMode=no", "root@myserver"]
[SSH Tunnel] In-app askpass bridge active at /tmp/tabularis-askpass-8f16dad7e51f4d74a2e82daa4bcfa345.sock
[SSH Tunnel] Tunnel established successfully on port 38861
[LOG] [2026-07-03 12:50:46.392] [INFO] tabularis_lib::commands - SSH tunnel created successfully on port 38861
[LOG] [2026-07-03 12:50:46.392] [INFO] tabularis_lib::pool_manager - Closing MySQL connection pool for: integritycheck (key: mysql:conn:bc7170ed-4be2-450d-8615-7b6926787728:integritycheck:ssl:preferred::::clear:false:pipes:true)
[LOG] [2026-07-03 12:50:46.392] [INFO] tabularis_lib::pool_manager - MySQL connection pool closed for: integritycheck (key: mysql:conn:bc7170ed-4be2-450d-8615-7b6926787728:integritycheck:ssl:preferred::::clear:false:pipes:true)
[LOG] [2026-07-03 12:50:52.518] [INFO] tabularis_lib::commands - Testing connection to database: integritycheck
[LOG] [2026-07-03 12:50:52.591] [INFO] tabularis_lib::commands - Connection test successful for database: integritycheck
[LOG] [2026-07-03 12:50:52.598] [INFO] tabularis_lib::health_check - Health check: registering connection bc7170ed-4be2-450d-8615-7b6926787728
[LOG] [2026-07-03 12:50:52.614] [INFO] tabularis_lib::commands - Fetching tables for connection: bc7170ed-4be2-450d-8615-7b6926787728
[LOG] [2026-07-03 12:50:52.614] [INFO] tabularis_lib::commands - Fetching views for connection: bc7170ed-4be2-450d-8615-7b6926787728
[LOG] [2026-07-03 12:50:52.614] [INFO] tabularis_lib::commands - Fetching routines for connection: bc7170ed-4be2-450d-8615-7b6926787728
[LOG] [2026-07-03 12:50:52.614] [INFO] tabularis_lib::commands - Fetching triggers for connection: bc7170ed-4be2-450d-8615-7b6926787728
[LOG] [2026-07-03 12:50:52.615] [INFO] tabularis_lib::pool_manager - Creating new MySQL connection pool for: root@Some("127.0.0.1") (key: mysql:conn:bc7170ed-4be2-450d-8615-7b6926787728:integritycheck:ssl:preferred::::clear:false:pipes:true)
[LOG] [2026-07-03 12:50:52.615] [INFO] tabularis_lib::pool_manager - Creating new MySQL connection pool for: root@Some("127.0.0.1") (key: mysql:conn:bc7170ed-4be2-450d-8615-7b6926787728:integritycheck:ssl:preferred::::clear:false:pipes:true)
[LOG] [2026-07-03 12:50:52.615] [INFO] tabularis_lib::pool_manager - Creating new MySQL connection pool for: root@Some("127.0.0.1") (key: mysql:conn:bc7170ed-4be2-450d-8615-7b6926787728:integritycheck:ssl:preferred::::clear:false:pipes:true)
[LOG] [2026-07-03 12:50:52.615] [INFO] tabularis_lib::pool_manager - Creating new MySQL connection pool for: root@Some("127.0.0.1") (key: mysql:conn:bc7170ed-4be2-450d-8615-7b6926787728:integritycheck:ssl:preferred::::clear:false:pipes:true)
[LOG] [2026-07-03 12:50:52.670] [INFO] tabularis_lib::pool_manager - MySQL connection pool created successfully for: integritycheck (key: mysql:conn:bc7170ed-4be2-450d-8615-7b6926787728:integritycheck:ssl:preferred::::clear:false:pipes:true)
[LOG] [2026-07-03 12:50:52.671] [INFO] tabularis_lib::pool_manager - MySQL connection pool created successfully for: integritycheck (key: mysql:conn:bc7170ed-4be2-450d-8615-7b6926787728:integritycheck:ssl:preferred::::clear:false:pipes:true)
[LOG] [2026-07-03 12:50:52.690] [INFO] tabularis_lib::pool_manager - MySQL connection pool created successfully for: integritycheck (key: mysql:conn:bc7170ed-4be2-450d-8615-7b6926787728:integritycheck:ssl:preferred::::clear:false:pipes:true)
[LOG] [2026-07-03 12:50:52.690] [INFO] tabularis_lib::pool_manager - MySQL connection pool created successfully for: integritycheck (key: mysql:conn:bc7170ed-4be2-450d-8615-7b6926787728:integritycheck:ssl:preferred::::clear:false:pipes:true)
[LOG] [2026-07-03 12:50:52.696] [INFO] tabularis_lib::commands - Retrieved 0 views from integritycheck
[LOG] [2026-07-03 12:50:52.697] [INFO] tabularis_lib::commands - Retrieved 1 triggers
[LOG] [2026-07-03 12:50:52.735] [INFO] tabularis_lib::commands - Retrieved 3 tables from integritycheck

[debba]

A fix for this is now ready in #407. Root cause: SSH tunnels are cached and reused across reconnects, but nothing ever tore down a dead one — after the server rebooted, the stale tunnel kept holding its local forward port, so reconnecting failed with "port already in use" until the app was restarted.

The PR makes tunnels detect a dead SSH session and release the port (including SSH keepalives so a rebooted server is noticed at all), evicts dead tunnels from the cache before reuse, and tears tunnels down on disconnect and on health-check failure. After the fix, once the orange indicator drops you can simply reconnect and a fresh tunnel is created.