Fix JetBrains persistence and toolbox backend reuse

Author: chapterjason

README.md

@@ -30,7 +30,7 @@ the workspace in Coder.
 | `rtk` | [rtk](https://github.com/rtk-ai/rtk), token-reducing Claude proxy | Auto-patches Claude Code via a post-create hook at workspace start |
 | `nvm` | [nvm](https://github.com/nvm-sh/nvm) at `/usr/local/share/nvm` | Default Node = LTS, `node`/`npm`/`npx` in `/usr/local/bin` |
 | `web-shell` | [web-shell](https://github.com/SoureCode/web-shell), persistent browser terminal | systemd unit, registered as a Coder app |
-| `jetbrains` | JetBrains Gateway remote backend persistence | Headless-only: Toolbox/Gateway runs on the user's local machine, opens a `jetbrains-gateway://` URL that SSHes in and runs `remote-dev-server.sh` here. Declares `~/.cache/JetBrains/`, `~/.config/JetBrains/`, `~/.local/share/JetBrains/`, `~/.java/.userPrefs/jetbrains/` to `home-persist` so the downloaded IDE backend, per-IDE settings, plugins, project indexes and JetProfile login survive workspace restarts. |
+| `jetbrains` | JetBrains Toolbox workspace integration | Uses Coder's JetBrains Toolbox module (`registry.coder.com/coder/jetbrains/coder`). Persists `~/.config/JetBrains/`, `~/.local/share/JetBrains/`, and `~/.java/.userPrefs/jetbrains/` for settings/plugins and JetProfile/license state. On startup, writes Toolbox `environment.json` (`allowUpdate=false`) and pins backend install location to `/mnt/home-persist/.jetbrains-dist` to avoid per-restart re-downloads, enforces `idea.properties` path split (`config/plugins` persisted, `system/log` in `/tmp`), prunes persisted `Daemon`, Toolbox `download/backup`, and per-IDE `caches/logs`, and does not persist `~/.cache/JetBrains/`. |
 | `home-persist` | Manifest-driven `$HOME` persistence | Reads `/etc/home-persist.d/*.json`, symlinks declared paths under `/mnt/home-persist` (per-owner volume). Add extra per-workspace paths via the `home_persist_paths` Coder parameter. See [`docs/persistence.md`](docs/persistence.md). |
 | `llvm` (cpp) | Clang toolchain via [apt.llvm.org](https://apt.llvm.org/) | `CC=clang`, `CXX=clang++` via `/etc/profile.d/llvm-env.sh` |
 | `cmake` (cpp) | CMake from [Kitware's GitHub releases](https://cmake.org/) | latest by default |

docs/persistence.md

@@ -66,14 +66,6 @@ opt in.
      single-writer semantics (lock files, unix sockets, per-project
      indexes) — otherwise two concurrent workspaces race and one fails.
 
-   ```json
-   {
-     "source": "jetbrains-local",
-     "scope": "workspace",
-     "paths": [".cache/JetBrains/"]
-   }
-   ```
-
    **Sibling rule**: owner-scoped and workspace-scoped paths must not nest
    under each other. A path symlinked at the parent already points into the
    shared volume; a child symlink would land inside that target and leak
@@ -108,14 +100,13 @@ opt in.
 │     │   .config/…   .local/share/…   .claude/   …               │
 │     │                                                           │
 │     └─ .workspaces/<CODER_WORKSPACE_ID>/   workspace-scoped     │
-│         ├─ <id-A>/  .cache/JetBrains/  …    (private to ws A)   │
-│         ├─ <id-B>/  .cache/JetBrains/  …    (private to ws B)   │
-│         └─ <id-C>/  .cache/JetBrains/  …    (private to ws C)   │
+│         ├─ <id-A>/  <workspace-only paths>                      │
+│         ├─ <id-B>/  <workspace-only paths>                      │
+│         └─ <id-C>/  <workspace-only paths>                      │
 │                                                                 │
 │   workspace A mounts /mnt/home-persist                          │
 │     └─► $HOME/.config/JetBrains → /mnt/home-persist/.config/... │
-│     └─► $HOME/.cache/JetBrains  → /mnt/home-persist/.workspaces │
-│                                     /<ws-A-id>/.cache/JetBrains │
+│     └─► $HOME/.local/share/JetBrains → /mnt/home-persist/...    │
 └─────────────────────────────────────────────────────────────────┘
 ```
 
@@ -136,8 +127,7 @@ Properties that fall out:
 | Source            | Scope       | Paths                         | Why                                  |
 | ----------------- | ----------- | ----------------------------- | ------------------------------------ |
 | `claude-code`     | owner       | `.claude/`, `.claude.json`    | Login credentials, sessions, plugins |
-| `jetbrains`       | owner       | `.config/JetBrains/`, `.local/share/JetBrains/`, `.java/.userPrefs/jetbrains/` | Settings, plugins, and JetProfile state that should follow the user across workspaces. Keymaps, color schemes, installed plugins, license acceptance. |
-| `jetbrains-local` | workspace   | `.cache/JetBrains/` | Per-workspace runtime: the SSH-deployed Toolbox Agent (`Toolbox-CLI-dist/`), its IPC lock and unix socket under `Toolbox/ports/`, the downloaded IDE backend (`RemoteDev/dist/`), and per-IDE system caches and project indexes. Must be per-workspace — concurrent workspaces that share `.cache/JetBrains/` race on the Toolbox Agent's `UnixApplicationStartLock` and fail to connect ("main instance is alive, cannot bind twice"). |
+| `jetbrains`       | owner       | `.config/JetBrains/`, `.local/share/JetBrains/`, `.java/.userPrefs/jetbrains/` | Settings, plugins, and JetProfile/license state that should follow the user across workspaces. |
 
 Anything not declared is image-owned (or per-workspace-home-volume-owned)
 and resets on image rebuild — git config, SSH keys, bash history, caches.
@@ -244,22 +234,44 @@ ls /mnt/home-persist/.workspaces/
 rm -rf /mnt/home-persist/.workspaces/<stale-id>
 ```
 
-## Migrating an owner-scoped path to workspace-scoped
+## Migrating or Dropping a Persisted Path
 
-Flipping a path from `scope: "owner"` to `scope: "workspace"` leaves the old
-`/mnt/home-persist/<path>` dir behind — the resolver retargets the symlink
-but doesn't touch the previous target. Tens to hundreds of MB can accumulate
-(JetBrains caches, Docker-ish state, etc.).
+Changing persistence scope or dropping persistence for a path leaves the old
+`/mnt/home-persist/<path>` dir behind — the resolver retargets or removes
+symlink ownership, but doesn't clean the previous target automatically.
 
 Add a `migration_sweep` line to `coder_script.lifecycle_init` in `main.tf`,
 keyed by a unique sentinel name:
 
 ```bash
 migration_sweep <sentinel-name> <path-relative-to-/mnt/home-persist>
 # e.g.
-migration_sweep jetbrains-cache-owner-to-workspace .cache/JetBrains
+migration_sweep jetbrains-cache-owner-to-ephemeral .cache/JetBrains
+migration_sweep jetbrains-toolbox-download-owner-to-ephemeral .local/share/JetBrains/Toolbox/download
+migration_sweep jetbrains-toolbox-backup-owner-to-ephemeral .local/share/JetBrains/Toolbox/backup
 ```
 
+For JetBrains backend reuse between restarts, `main.tf` creates
+`/mnt/home-persist/.jetbrains-dist` and writes Toolbox `environment.json`
+with a fixed `tools.location` pointing to that path.
+
+Because `.local/share/JetBrains/` is persisted broadly, `main.tf` startup also
+prunes subpaths that should remain ephemeral:
+
+`~/.local/share/JetBrains/Daemon/`
+`~/.local/share/JetBrains/Toolbox/download/`
+`~/.local/share/JetBrains/Toolbox/backup/`
+`~/.local/share/JetBrains/*/{caches,logs}/`
+
+`main.tf` startup also writes `~/.local/share/JetBrains/Toolbox/environment.json`
+with `tools.allowUpdate=false` and a pinned tool location, and writes `idea.properties` in each
+`~/.config/JetBrains/<IDE>/` directory with:
+
+`idea.config.path=~/.config/JetBrains/<IDE>`
+`idea.plugins.path=~/.local/share/JetBrains/<IDE>/plugins`
+`idea.system.path=/tmp/jetbrains/system/<IDE>`
+`idea.log.path=/tmp/jetbrains/log/<IDE>`
+
 The sweep runs once per owner volume (the sentinel
 `/mnt/home-persist/.workspaces/.migrated/<sentinel-name>` blocks reruns) and
 is a no-op if the orphan is already gone. Delete the line from `main.tf`

main.tf

@@ -370,10 +370,60 @@ resource "coder_script" "lifecycle_init" {
       touch "$sentinel"
     }
     if [ -w /mnt/home-persist ]; then
-      migration_sweep jetbrains-cache-owner-to-workspace .cache/JetBrains
+      migration_sweep jetbrains-cache-owner-to-ephemeral .cache/JetBrains
+      mkdir -p /mnt/home-persist/.jetbrains-dist
+      rm -rf /mnt/home-persist/.local/share/JetBrains/Daemon
+      rm -rf /mnt/home-persist/.local/share/JetBrains/Toolbox/download
+      rm -rf /mnt/home-persist/.local/share/JetBrains/Toolbox/backup
+      if [ -d /mnt/home-persist/.local/share/JetBrains ]; then
+        find /mnt/home-persist/.local/share/JetBrains -mindepth 2 -maxdepth 2 -type d \
+          \( -name caches -o -name logs \) -exec rm -rf {} +
+      fi
     fi
 
     [ -x /usr/local/bin/home-persist-resolve ]          && /usr/local/bin/home-persist-resolve
+
+    mkdir -p "$HOME/.local/share/JetBrains/Toolbox"
+    printf '%s\n' \
+      '{' \
+      '  "tools": {' \
+      '    "allowUpdate": false,' \
+      '    "location": [' \
+      '      {' \
+      '        "path": "/mnt/home-persist/.jetbrains-dist",' \
+      '        "levels": 1' \
+      '      }' \
+      '    ]' \
+      '  }' \
+      '}' \
+      > "$HOME/.local/share/JetBrains/Toolbox/environment.json"
+
+    mkdir -p "$HOME/.config/JetBrains" "$HOME/.local/share/JetBrains"
+    if [ -d "$HOME/.local/share/JetBrains" ]; then
+      for share_dir in "$HOME"/.local/share/JetBrains/*; do
+        [ -d "$share_dir" ] || continue
+        share_name="$(basename "$share_dir")"
+        case "$share_name" in
+          Toolbox|Daemon|consentOptions)
+            continue
+            ;;
+        esac
+        mkdir -p "$HOME/.config/JetBrains/$share_name"
+      done
+    fi
+    if [ -d "$HOME/.config/JetBrains" ]; then
+      for ide_dir in "$HOME"/.config/JetBrains/*; do
+        [ -d "$ide_dir" ] || continue
+        ide_name="$(basename "$ide_dir")"
+        printf '%s\n' \
+          "idea.config.path=$ide_dir" \
+          "idea.plugins.path=$HOME/.local/share/JetBrains/$ide_name/plugins" \
+          "idea.system.path=/tmp/jetbrains/system/$ide_name" \
+          "idea.log.path=/tmp/jetbrains/log/$ide_name" \
+          > "$ide_dir/idea.properties"
+      done
+    fi
+
     [ -x "$HOME/.local/share/rtk/post-create.sh" ]      && "$HOME/.local/share/rtk/post-create.sh"
     exit 0
   EOT

scripts/jetbrains/install.sh

@@ -1,47 +1,8 @@
 #!/usr/bin/env bash
-# jetbrains installer.
-#
-# How the Coder JetBrains flow works: the workspace is headless. Clicking the
-# IDE button in the Coder UI opens a `jetbrains-gateway://` URL that the
-# user's *local* Toolbox/Gateway handles; Gateway then SSHes into the
-# workspace and runs remote-dev-server.sh, which downloads the IDE backend
-# (IntelliJ IDEA, PyCharm, CLion, WebStorm, GoLand, RubyMine, PhpStorm,
-# Rider, DataGrip) on-demand into $HOME. Toolbox itself never runs on the
-# workspace. Nothing to install in the image — this script only declares the
-# HOME paths the remote-dev backend writes, so Gateway doesn't redownload
-# hundreds of MB and users don't lose settings, plugins, and project indexes
-# on every restart.
-#
-# JetBrains' remote-dev backend follows the XDG scheme on Linux with a
-# RemoteDev-<Code> subdir per IDE flavor (PS = PhpStorm, PY = PyCharm, IU =
-# IDEA Ultimate, CL = CLion, ...) under these three roots:
-#
-#   ~/.cache/JetBrains/         — RemoteDev/dist/<build>/ (downloaded backend),
-#                                 RemoteDev-<Code>/ system caches, indexes,
-#                                 LocalHistory, log, Toolbox-CLI-dist/ (the
-#                                 SSH-deployed Toolbox Agent), Toolbox/ports/
-#                                 (agent IPC / lock)
-#   ~/.config/JetBrains/        — RemoteDev-<Code>/ IDE settings, keymaps,
-#                                 schemes, options, workspace, .lock
-#   ~/.local/share/JetBrains/   — RemoteDev-<Code>/ installed plugins,
-#                                 Daemon, consentOptions
-#
-# Plus the Java Preferences store the IDEs use for JetBrains Account
-# (JetProfile) login, license activation, and non-commercial-license
-# acceptance — skipping this forces re-login + re-accept every restart:
-#
-#   ~/.java/.userPrefs/jetbrains/
-#
-# .cache/JetBrains/ is scoped per-workspace because the Toolbox Agent's
-# UnixApplicationStartLock + IPC socket live under it and would collide
-# across concurrent workspaces ("main instance is alive, cannot bind twice").
-# Settings, plugins, and JetProfile state stay owner-scoped so they follow
-# the user across workspaces.
 set -eo pipefail
 
 mkdir -p /etc/home-persist.d
 
-# Owner-scoped: things we want shared across all of the user's workspaces.
 tee /etc/home-persist.d/jetbrains.json >/dev/null <<'EOF'
 {
   "source": "jetbrains",
@@ -53,16 +14,3 @@ tee /etc/home-persist.d/jetbrains.json >/dev/null <<'EOF'
   ]
 }
 EOF
-
-# Workspace-scoped: runtime IPC and per-project indexes. Must be private
-# per workspace, otherwise concurrent workspaces race on the Toolbox Agent
-# lock. Survives restarts of the same workspace.
-tee /etc/home-persist.d/jetbrains-local.json >/dev/null <<'EOF'
-{
-  "source": "jetbrains-local",
-  "scope": "workspace",
-  "paths": [
-    ".cache/JetBrains/"
-  ]
-}
-EOF