Skip to content

feat: prune idle volume browser helper containers#2767

Merged
kmendell merged 9 commits into
getarcaneapp:mainfrom
Zgrill2:feat/volume-helper-reaper
Jun 4, 2026
Merged

feat: prune idle volume browser helper containers#2767
kmendell merged 9 commits into
getarcaneapp:mainfrom
Zgrill2:feat/volume-helper-reaper

Conversation

@Zgrill2

@Zgrill2 Zgrill2 commented May 30, 2026

Copy link
Copy Markdown
Contributor

Checklist

  • This PR is not opened from my fork’s main branch

What This PR Implements

Volume Browser Helper Reaper

Context / Motivation

The volume Browser tab (/volumes/[volumeName] → Browser) browses a volume's contents read-only. To do this the backend spins up a Docker helper
container that mounts the volume read-only at /volume

The problem: these helpers leak. If the user closes the tab, loses network, or the browser crashes, nothing tells the backend to tear the
helper down — it keeps running and holds the volume, which can block volume deletes/prunes and waste resources.

This PR adds layered cleanup: eager client-side teardown when the user leaves the browser, plus a backend reaper job as a guaranteed backstop that
removes any helper left idle past a configurable timeout.

Fixes: #2765

Changes Made

How it works

Helper lifecycle

  • Created per volume on the first read-only browse request; tracked in helperByVolume[volumeName] with a lastUsedAt timestamp.
  • Subsequent requests reuse the same helper and refresh lastUsedAt.
  • Helpers are independent per volume — no "one at a time" global constraint.

Teardown paths (layered, intentional)

  1. Eager client-side (VolumeBrowser.svelte):
    - onDestroy → stopBrowsing(volumeName) on tab switch, in-app nav, back button, and opening another volume (unmount).
    - pagehide → navigator.sendBeacon(.../browse/stop) on hard close/refresh.
  2. Manual — "Stop browsing" button → same stopBrowsing.
  3. Backend reaper (backstop) — VolumeHelperReaperJob runs every minute (0 * * * * *, fixed), removing any helper idle ≥ the timeout. Catches leaks
    the client paths miss.
  4. Implicit — DeleteVolume / PruneVolumes stop helpers first so Docker doesn't see the volume as in-use.

▎ Design note: teardown is eager, not lazy. The helper dies the moment you leave the browser; the 10-min idle timeout is the safety net, not the primary mechanism.

Testing Done

  • Development environment started: ./scripts/development/dev.sh start
  • Frontend verified at http://localhost:3000
  • Backend verified at http://localhost:3552
  • Manual testing completed (describe):
  • No linting errors (e.g., just lint all)
  • Backend tests pass: just test backend

AI Tool Used (if applicable)

AI Tool: Claude Code
Assistance Level: Moderate
What AI helped with: Design implementation plan, doing the actual code, running the linting and just tests.
I reviewed and edited all AI-generated output: Yes always. I had the design and all implementation plans were reviewed by me
I ran all required tests and manually verified changes: Yes, tested on my local laptop

Additional Context

My partner uses Arcane and really wants this feature

Disclaimer Greptiles Reviews use AI, make sure to check over its work.

To better help train Greptile on our codebase, if the comment is useful and valid Like the comment, if its not helpful or invalid Dislike

To have Greptile Re-Review the changes, mention greptileai.

Greptile Summary

This PR introduces a layered cleanup mechanism for volume-browser helper containers: eager client-side teardown in VolumeBrowser.svelte (onDestroy + pagehide beacon), explicit "Stop browsing" button, pre-cleanup in DeleteVolume/PruneVolumesWithOptions, and a background reaper job (PruningVolumeHelperJob) as the final backstop.

  • The helperByVolume map is promoted from map[string]string to map[string]*volumeHelper, adding a lastUsedAt timestamp. StopHelper, ReapIdleHelpers, and touchHelperInternal are carefully split so all Docker calls happen outside the mutex, with map entries pre-deleted before container removal — concurrent browse requests get a clean inspect-fail path and re-create a fresh helper.
  • A new VolumeBrowserHelperIdleTimeout setting (default 10 minutes, 0 disables) is wired through models, defaults, the TypeScript type, and the schema test; the reaper job runs on a fixed 5-minute schedule and reads this knob each invocation.
  • Pure-Go helper functions (collectStaleHelperIDsInternal, takeHelperIDInternal, touchHelperInternal) are tested in isolation without Docker, covering fresh/stale/edge/nil entries and idempotent absent cases.

Confidence Score: 5/5

The change is safe to merge. All Docker calls are outside the mutex, force removal is used throughout, and concurrent browse/reap races result in a clean inspect-fail path rather than undefined behavior.

The concurrency design is sound: map entries are pre-deleted before container removal so concurrent requests transparently re-create a fresh helper, and force removal via volumeHelperRemoveOptionsInternal ensures running containers are stopped even when the timeout fires. DeleteVolume and PruneVolumes correctly stop helpers first to avoid "volume in use" conflicts. New unexported functions all carry the required Internal suffix. Unit tests cover edge cases (exact-timeout boundary, nil entries, idempotent take) without requiring Docker. The new setting is consistently plumbed through models, defaults, TypeScript types, and schema tests.

No files require special attention.

Reviews (7): Last reviewed commit: "rename" | Re-trigger Greptile

Zgrill2 added 2 commits May 30, 2026 10:32
Reused read-only helper containers leaked when a client vanished, holding the volume open and blocking deletes/prunes. Now track one helper per volume, stop it before DeleteVolume and on PruneVolumes, and add a reaper job (runs every minute) that removes helpers idle past volumeBrowserHelperIdleTimeout (minutes; default 10, 0 disables). Adds a POST browse/stop endpoint for explicit and page-unload teardown.
Stop the volume browser helper as soon as the user leaves the browser, instead of waiting on the backend reaper (now only a backstop). Stops on onDestroy (tab switch, in-app nav, back, opening another volume) and via a pagehide sendBeacon on hard close/refresh. Adds a Stop browsing button, the stopBrowsing service call, the settings type, and i18n messages.
@Zgrill2 Zgrill2 requested a review from a team May 30, 2026 14:42
Comment thread backend/api/handlers/volumes.go Outdated
Comment thread frontend/src/lib/components/file-browser/VolumeBrowser.svelte Outdated
Comment thread frontend/src/lib/components/file-browser/VolumeBrowser.svelte Outdated
Address review feedback: move the pagehide teardown beacon to a declarative <svelte:window onpagehide> handler instead of a manual onMount addEventListener pair, and show an error toast (new volumes_browser_stop_error message) when the explicit Stop button fails so it no longer fails silently.
@Zgrill2

Zgrill2 commented May 30, 2026

Copy link
Copy Markdown
Contributor Author

Fixed issues #2 and #3. Issue #1 about admin is a non-issue based on my analysis. The endpoint only allows stopping the helper container, not any arbitrary container and so does not need to be explicitly admin protected

@Zgrill2

Zgrill2 commented May 30, 2026

Copy link
Copy Markdown
Contributor Author

greptileai please re-review

@cabaucom376

Copy link
Copy Markdown
Collaborator

@greptileai

@Zgrill2

Zgrill2 commented May 30, 2026

Copy link
Copy Markdown
Contributor Author

@cabaucom376 do I have to do anything to get the E2E Test workflows to run? I ran all just tests on my local, is the report expected to upload or its running in a workflow?

please let me know if any additional changes requested

@Zgrill2

Zgrill2 commented May 31, 2026

Copy link
Copy Markdown
Contributor Author

@kmendell please review

Comment thread backend/api/handlers/volumes.go Outdated
@kmendell kmendell changed the title feat(volumes): reap idle volume browser helper containers feat: prune idle volume browser helper containers Jun 4, 2026
@kmendell kmendell enabled auto-merge (squash) June 4, 2026 00:56
@kmendell kmendell merged commit d3b2795 into getarcaneapp:main Jun 4, 2026
21 checks passed
github-actions Bot added a commit to ShobuPrime/home-assistant-apps that referenced this pull request Jun 5, 2026
## Arcane Docker Manager Update

This automated PR updates Arcane from `1.19.5` to `1.20.0`.

### Changelog


### New features

* add removeOrphans option to project deploy/redeploy ([#2785](getarcaneapp/arcane#2785) by `khanhx`)
* prune idle volume browser helper containers ([#2767](getarcaneapp/arcane#2767) by `Zgrill2`)

### Bug fixes

* slog-go nil pointer dereference ([#2781](getarcaneapp/arcane#2781) by `lohrbini`)
* dashboard card buttons paddings overlaps([c1a0bda](getarcaneapp/arcane@c1a0bda) by `kmendell`)
* disable schema display on text selection([058f062](getarcaneapp/arcane@058f062) by `kmendell`)
* clear / check for default jwt secret([ae914bd](getarcaneapp/arcane@ae914bd) by `kmendell`)

### Dependencies

* bump date-fns from 4.2.1 to 4.3.0 ([#2745](getarcaneapp/arcane#2745) by `dependabot`[bot])
* bump `sveltejs`/ki

### Changes

- Updated `config.yaml` version
- Updated `build.yaml` ARCANE_VERSION
- Updated `Dockerfile` ARCANE_VERSION
- Updated documentation files
- Updated CHANGELOG.md

### Release Notes

Full release notes: https://github.com/getarcaneapp/arcane/releases/tag/v1.20.0

---

This PR was automatically generated by the Update Arcane workflow

Auto-merged by GitHub Actions
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

⚡️ Feature: Volume helpers get orphaned

3 participants