feat(feeds): add consecutive-failure backoff to FEED_REFRESH loop

[jollySleeper]

Summary

• Adds a consecutive-failure counter to the FEED_REFRESH loop — after 5 failures in a row, pauses for 5 minutes before resuming
• Any successful channel refresh resets the counter to zero
• Prevents hammering YouTube when the instance is being rate-limited

Problem

When YouTube rate-limits an instance (HTTP 429, CAPTCHA pages, or bot-detection), the current loop catches the error and immediately moves to the next channel with only the normal per-channel delay. With hundreds of subscribed channels, this means dozens of failed requests in quick succession against a service that is actively rejecting the instance.

Why not exponential backoff?

The loop already has built-in per-channel pacing (window / channel_count), so a single fixed 5-minute pause is sufficient to let rate-limit windows expire. Exponential backoff adds state management complexity for marginal benefit in this architecture.

Why not catch ReCaptchaException specifically?

The current DownloaderImpl does not throw ReCaptchaException on 429 responses — the CAPTCHA handling code is commented out. Rate-limiting manifests as ParsingException or IOException when the extractor fails to parse YouTube's error/CAPTCHA page. A type-agnostic consecutive-failure counter handles this correctly regardless of exception type.

Changes

• Main.java: Add consecutiveFailures counter scoped to each refresh cycle
  • Reset to 0 on success
  • Increment on failure
  • If ≥ 5: log, pause 5 minutes, reset

Test plan

• Build compiles cleanly (./gradlew build)
• Deploy with FEED_REFRESH=true and verify normal refresh cycle works (counter stays at 0)
• Simulate failures (e.g., disconnect network briefly) and verify backoff message appears after 5 consecutive failures
• Verify loop resumes normally after backoff pause

Summary by CodeRabbit

• Bug Fixes
  • Improved feed refresh reliability by handling repeated update failures more gracefully.
  • After several consecutive failures, the app now pauses briefly before continuing, helping reduce rapid retry loops and temporary overload.
  • Successful channel updates reset the failure count so normal processing resumes smoothly.

[coderabbitai]

📝 Walkthrough

Walkthrough

Feed-refresh processing now counts consecutive channel-refresh failures, resets the counter on success, and pauses for five minutes after five back-to-back exceptions before resuming.

Changes

Feed-refresh failure backoff

Layer / File(s)	Summary
Failure counter and pause handling src/main/java/me/kavin/piped/Main.java	A consecutive-failures counter is initialized in the feed-refresh loop, incremented on exceptions, and used to trigger a 5-minute pause after five consecutive failures before resetting.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

• TeamPiped/Piped-Backend#889: Introduced the periodic feed-refresh worker that this loop-level failure backoff extends.

Suggested reviewers

• FireMasterK

Poem

A rabbit counted failures by the hop,
then paused for tea when five would not stop.
I twitched my nose, waited a spell,
then bounced back on the feed-refresh trail. 🐇

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly describes the main change: adding consecutive-failure backoff to the feed refresh loop.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/main/java/me/kavin/piped/Main.java`:
- Around line 279-280: The failure streak in the refresh loop is being reset
each time the outer while pass restarts, so move consecutiveFailures to a scope
that survives across full refresh passes in Main’s refresh logic. Update the
handling around the loop that processes subscribed channels so the same counter
is reused across iterations, and keep the backoff reset only when a successful
refresh occurs; this ensures the streak can cross pass boundaries and trigger
the intended delay.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 8cb7b320-3dbc-4916-b84b-736b2b1b4ea7

📥 Commits

Reviewing files that changed from the base of the PR and between 6d0ad06 and 5858b26.

📒 Files selected for processing (1)

• src/main/java/me/kavin/piped/Main.java

[coderabbitai]

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Keep the failure streak across full refresh passes.

consecutiveFailures is recreated on each outer while iteration, so a streak that spans the end of one pass and the start of the next is lost. That means instances with fewer than 5 subscribed channels can fail forever without ever hitting the new backoff.

Proposed fix

         if (Constants.FEED_REFRESH)
             Thread.ofVirtual().name("feed-refresh").start(() -> {
+                int consecutiveFailures = 0;
                 while (true) {
                     try {
                         List<String> channelIds;
@@
-                        int consecutiveFailures = 0;
-
                         for (String channelId : channelIds) {
                             try {
                                 var info = ChannelInfo.getInfo("https://youtube.com/channel/" + channelId);

Also applies to: 290-299

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/main/java/me/kavin/piped/Main.java` around lines 279 - 280, The failure
streak in the refresh loop is being reset each time the outer while pass
restarts, so move consecutiveFailures to a scope that survives across full
refresh passes in Main’s refresh logic. Update the handling around the loop that
processes subscribed channels so the same counter is reused across iterations,
and keep the backoff reset only when a successful refresh occurs; this ensures
the streak can cross pass boundaries and trigger the intended delay.