Failed backup not retried when client back online

[MegaV0lt]

Laptop went offline during backup because of unstable internet connection. After turning on the laptop on the next day the log does not show that agent registers. In clients page the laptop is show as 'Online'
The failed backup is not automatically restarted. Is that the normal behavior?

Should the client not retry to do the failed backup?

Maybe a setting where the admin can choose to just fail or automatically retry is needed?

I also did not get an email that the backup failed. I think that should not be the default

[marcpope]

Hi @MegaV0lt — thanks for sticking with this, the report uncovered three separate gaps that I've now fixed in main (will ship in the next release).

What was actually wrong

1. The reconnect was silent in the log. The heartbeat code did flip the agent back to online, but never wrote a server_log entry — so you saw the "agent went offline" line with no matching "back online" line. Fixed: now writes an info row at the offline → online transition.

2. No auto-retry of offline-induced failures. Scheduled backups that hadn't started yet did wait for the agent to reconnect (queued jobs are left alone), but a backup already in flight when the agent dropped just got marked failed with no follow-up — the next chance for that plan was the next scheduled run, sometimes 24 hours away. Fixed: when the offline sweep marks an in-flight backup failed, it now re-queues the same plan as a fresh job, capped at 3 attempts (configurable). Real errors — borg path missing, repo locked, encryption failure — are reported by the agent via a different path and are not retried, so a genuinely broken backup can't loop forever.

3. The "no email" complaint was actually about deduplication The real cause: NotificationService::notify() only fired email when a notification was newly created. If you had a previous unresolved backup_failed for that plan, every subsequent failure just bumped occurrence_count silently. Fixed: last_emailed_at column added to notifications; the email re-fires after 6 hours of silent accumulation. On terminal retry exhaustion, the email also fires unconditionally, bypassing dedup — so persistent failures can't go quiet.

Bonus: added a banner on Settings → Email Settings that warns if any email_on_* toggle is enabled but SMTP isn't actually configured/tested, so future users don't fall into that trap.

Two things that would help us narrow down what specifically happened on your install

• After upgrading to the next release, on Settings → Email Settings click Test SMTP — that confirms the email path works end-to-end.
• On the Notifications bell, can you tell us whether a backup_failed for /home exists, and what its occurrence_count is? If it's >1, the dedup behaviour I described above was the cause; the new escalation will keep that from happening again.

New settings (Settings → General → Agent)

• Auto-retry backups when agent goes offline — checkbox, default on
• Max Retry Attempts — number, default 3

Thanks again for the detail in the original report — the fix is much better for it.

[marcpope]

Leaving open for feedback.

[MegaV0lt]

Upgraded to the new version.

1. Clicking Test SMTP returns 'Success'

Note: No eMail in my inbox or spam folder. I was expecting something like an '[BBS] Test eMail'

2. The Notification page shows this for 'Darkwing Laptop'

 	Client "Darkwing Laptop" is offline (no heartbeat in 900s) 	Darkwing Laptop 	Warning 	May 6, 15:26 	Resolved 	
	Client "Darkwing Laptop" is offline (no heartbeat in 900s) 	Darkwing Laptop 	Warning 	May 6, 11:25 	Resolved 	
	Client "Darkwing Laptop" is offline (no heartbeat in 900s) 5 occurrences 	Darkwing Laptop 	Warning 	May 6, 05:22 	Resolved 	
...
	Client "Darkwing Laptop" is offline (no heartbeat in 900s) 13 occurrences 	Darkwing Laptop 	Warning 	May 5, 10:39 	Resolved 	

The oldest entry is the one from 5. May

[MegaV0lt]

Here is the log from that time:

May 6, 07:24 	#336 	Darkwing Laptop 	info 	Backup command: 'borg' 'create' '--json' '--log-json' '--list' '--progress' '--lock-wait=600' '--compression=auto,zstd' '--exclude-caches' '--one-file-system' '--noatime' '--exclude-if-present' '.nobackup' '--exclude' '*/[Cc]ache/' '--exclude' '*/Dropbox/' '--exclude' '*/Nextcloud/' '--exclude' '*/pCloudDrive/' '--exclude' '*.tmp' '--exclude' '*.log' '--exclude' '*/.cache' 'ssh://bbs-darkwinglaptop@bbs.darkwing.dynu.net///mnt/usbdisk/_Backup/2/laptop-home::plan1-2026-05-06_05-24-01' '/home'
May 6, 07:24 	#336 	Darkwing Laptop 	info 	Job #336 (backup) sent to agent queue
May 6, 07:23 	#336 	Darkwing Laptop 	info 	Job #336 queued (retry of #329)
May 3, 21:09 	#329 	Darkwing Laptop 	warning 	Job #329 (backup) failed — agent "Darkwing Laptop" went offline
May 3, 20:00 	#329 	Darkwing Laptop 	info 	Agent started backup job #329 for plan "/home"
May 3, 20:00 	#329 	Darkwing Laptop 	info 	Scheduled backup queued for plan "/home"
May 3, 20:00 	#329 	Darkwing Laptop 	info 	Backup command: 'borg' 'create' '--json' '--log-json' '--list' '--progress' '--lock-wait=600' '--compression=auto,zstd' '--exclude-caches' '--one-file-system' '--noatime' '--exclude-if-present' '.nobackup' '--exclude' '*/[Cc]ache/' '--exclude' '*/Dropbox/' '--exclude' '*/Nextcloud/' '--exclude' '*/pCloudDrive/' '--exclude' '*.tmp' '--exclude' '*.log' '--exclude' '*/.cache' 'ssh://bbs-darkwinglaptop@bbs.darkwing.dynu.net///mnt/usbdisk/_Backup/2/laptop-home::plan1-2026-05-03_18-00-01' '/home'
May 3, 20:00 	#329 	Darkwing Laptop 	info 	Job #329 (backup) sent to agent queue

Note: The 'Retry' from May 6 was done from me manually from the log where i clicked 'Retry'