Skip to content

Commit 1a15425

Browse files
authored
fix: node graceful shutdown by handling SIGTERM (#2335)
## Fix graceful shutdown in Docker containers ### Problem When running the rollkit node in a Docker container and using `docker stop`, the process doesn't shutdown gracefully and causes database corruption. ### Logs ``` 7:00AM INF creating empty block height=10 module=BlockManager 7:00AM INF creating empty block height=11 module=BlockManager 7:00AM INF creating empty block height=12 module=BlockManager # command: `docker stop <container_name> # command: `docker start <container_name> 7:00AM INF creating new client module=main namespace=000000000000000000000000000000000000002777d4d961c75a526dd8 7:00AM INF using default mempool ttl MempoolTTL=25 module=BlockManager 7:00AM INF starting P2P client module=main 7:00AM INF started RPC server addr=127.0.0.1:7331 module=main 7:00AM INF listening on address=/ip4/127.0.0.1/tcp/7676/p2p/12D3KooWMubR4af4S9VW9xZMix4HYptuM9htnvccLuT6q7BhAGWf module=main 7:00AM INF listening on address=/ip4/172.28.0.4/tcp/7676/p2p/12D3KooWMubR4af4S9VW9xZMix4HYptuM9htnvccLuT6q7BhAGWf module=main 7:00AM INF no peers - only listening for connections module=main 7:00AM INF working in aggregator mode block time=1s module=main 7:00AM INF Reaper started interval=1000 module=Reaper 7:00AM INF creating empty block height=13 module=BlockManager 7:00AM ERR unrecoverable error in one of the go routines... error="error in normal aggregation loop: error while publishing block: failed to broadcast: validation failed" module=main 7:00AM INF halting full node and its sub services... module=main 7:00AM INF data submission loop stopped module=BlockManager 7:00AM INF Reaper stopped module=Reaper 2025-06-04T07:00:46.080Z ERROR header/sync sync/sync.go:229 syncing headers {"from": 2, "to": 13, "err": "header/p2p: exchange is closed"} 7:00AM INF full node halted successfully module=main 7:00AM ERR node error error="context canceled" module=main Error: context canceled ``` ### Solution - Added `SIGTERM` signal handling alongside existing `SIGINT` handling - Added `syscall` import to support `syscall.SIGTERM` - Now responds to both interactive shutdown (Ctrl+C) and Docker stop commands ### Changes - Modified `StartNode()` function in `cmd/run_node.go` to listen for both `os.Interrupt` and `syscall.SIGTERM` - Added `syscall` import ### Testing - [x] Graceful shutdown with Ctrl+C still works - [x] Graceful shutdown with `docker stop` now works - [x] No database corruption after Docker container shutdown ``` 7:00AM INF creating empty block height=10 module=BlockManager 7:00AM INF creating empty block height=11 module=BlockManager 7:00AM INF creating empty block height=12 module=BlockManager # command: `docker stop <container_name> 7:04AM INF shutting down node... module=main 7:04AM INF context canceled, stopping node module=main 7:04AM INF halting full node and its sub services... module=main 7:04AM INF Reaper stopped module=Reaper 7:04AM INF data submission loop stopped module=BlockManager 7:04AM INF full node halted successfully module=main # command: `docker start <container_name> 7:04AM INF creating new client module=main namespace=000000000000000000000000000000000000002777d4d961c75a526dd8 7:04AM INF using default mempool ttl MempoolTTL=25 module=BlockManager 7:04AM INF starting P2P client module=main 7:04AM INF started RPC server addr=127.0.0.1:7331 module=main 7:04AM INF listening on address=/ip4/127.0.0.1/tcp/7676/p2p/12D3KooWS5qA2Soarjzy5CwVukArsqnZ9qyAq1wWxoCyZ9q3egKv module=main 7:04AM INF listening on address=/ip4/172.28.0.4/tcp/7676/p2p/12D3KooWS5qA2Soarjzy5CwVukArsqnZ9qyAq1wWxoCyZ9q3egKv module=main 7:04AM INF no peers - only listening for connections module=main 7:04AM INF working in aggregator mode block time=1s module=main 7:04AM INF Reaper started interval=1000 module=Reaper 7:04AM INF creating empty block height=13 module=BlockManager 7:04AM INF creating empty block height=14 module=BlockManager ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - The application now supports graceful shutdown when receiving SIGTERM signals, in addition to existing support for SIGINT. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
1 parent 907c7df commit 1a15425

1 file changed

Lines changed: 2 additions & 1 deletion

File tree

pkg/cmd/run_node.go

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ import (
88
"os/signal"
99
"path/filepath"
1010
"strings"
11+
"syscall"
1112
"time"
1213

1314
"cosmossdk.io/log"
@@ -168,7 +169,7 @@ func StartNode(
168169

169170
// Wait for interrupt signal to gracefully shut down the server
170171
quit := make(chan os.Signal, 1)
171-
signal.Notify(quit, os.Interrupt)
172+
signal.Notify(quit, os.Interrupt, syscall.SIGTERM)
172173

173174
select {
174175
case <-quit:

0 commit comments

Comments
 (0)