Skip to content

Commit 158d95d

Browse files
committed
Fix: Change timeout pause && resume to 1600s
1 parent bd45d18 commit 158d95d

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

axlearn/common/launch_trainer_main.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@ def train():
4444
train = utils.elastic_manager.pause_resume(
4545
max_retries=10, # Handle up to 10 disruptions before restarting
4646
poll_interval=10, # While paused, checks every 10 seconds for health
47-
timeout=300, # Waits for slices to rejoin for 5 minutes
47+
timeout=1600, # Waits for slices to rejoin for 20 minutes
4848
# on_elastic_event_callback=clean_up_checkpoints,
4949
)(train)
5050

0 commit comments

Comments
 (0)