feat: support multi-process deployments (web + workers)

A backend deployment can now declare additional long-running processes
alongside the web server. PM2 supervises them all under one deployment,
with worker names namespace-prefixed to prevent collisions between
multiple shipnode deployments on the same host.

Adds .worker() to the builder, generalises pm2 config into pm2.apps[],
moves the ecosystem file into each release (ADR-0001), and enriches the
post-deploy health check with a pm2 jlist status assertion so workers
that crash on startup don't slip past the HTTP probe.

Author: devalade

CHANGELOG.md

@@ -4,6 +4,32 @@ All notable changes to `@devalade/shipnode` will be documented here.
 
 ## [Unreleased]
 
+### Added
+- **Workers / multi-process deployments** — a backend can now declare additional long-running processes (workers, cron consumers, queues) alongside the web server. PM2 supervises all of them under one deployment.
+  - New `.worker({ name, command, instances?, maxMemory?, env? })` builder method appends a worker to the deployment.
+  - New `pm2.apps` config shape: each entry has `name`, optional `command` (custom entry like `node dist/worker.js`; defaults to `<pkgManager> start`), `port` (web app only), `instances`, `maxMemory`, `env`.
+  - The entry with a `port` is the web app; entries without are workers (see [ADR-0002](docs/adr/0002-port-presence-determines-web-app.md)).
+  - Worker-only deployments are legal — a backend with no web app skips the HTTP health check and Caddy site config.
+  - `restart`, `stop`, `logs` accept `--process <name>` to target a single app; without it they operate on the whole deployment via the PM2 namespace.
+  - Worker names are automatically prefixed with the deployment namespace when registered with PM2 (e.g. `.pm2('api')` + `.worker({name: 'mailer'})` shows up in `pm2 list` as `api` and `api-mailer`). Prevents collisions between multiple shipnode deployments on the same host. Users always refer to the short name in shipnode commands.
+- **PM2 status check** — after every deploy, `pm2 jlist` is parsed and each declared app must be `online` with `restart_time === 0`. Catches workers that boot and crash before the HTTP health check would notice anything. Runs even on worker-only deployments.
+- **Per-app `env`** — each `pm2.apps` entry can set its own env vars (`{ WORKER_QUEUE: 'emails' }`); PM2 loads the shared `.env` via `env_file` and applies per-app overrides on top.
+
+### Changed
+- **Ecosystem file is per-release** — `ecosystem.config.cjs` now lives inside each release directory instead of `<remotePath>/shared/`. PM2 references it via `<remotePath>/current/ecosystem.config.cjs` so rollback restores the exact process set that was active for that release (a worker added in v2 won't crash-loop after rolling back to v1). See [ADR-0001](docs/adr/0001-per-release-ecosystem-file.md).
+- **Builder back-compat preserved** — `.pm2(name, opts)` and `.port(n)` still work and now act as sugar on the first app. Existing `shipnode.config.ts` files don't need changes.
+- `shipnode config show` now lists each PM2 app with its full per-entry shape.
+- `shipnode status` shows every declared app, not just the one matching the deployment name.
+
+### Removed
+- `ShipnodeConfig.backend` and the `BackendConfig` type — the port now lives on the web `pm2.apps` entry. Users of the builder/loader are unaffected; only direct consumers of the `ShipnodeConfig` TypeScript shape need to read `pm2.apps.find(a => a.port !== undefined)?.port` instead of `config.backend?.port`.
+- The legacy `pm2: { name, instances, maxMemory }` object shape on `ShipnodeConfig` — same migration: read `pm2.apps[0]` instead. (The legacy *input* shape is still accepted by `assembleConfig` and folded onto `apps[0]`.)
+
+### Migration
+- **No config changes required** for existing deployments — your current `shipnode.config.ts` keeps working.
+- The first deploy under this version silently cleans up any pre-existing PM2 process with the deployment's name (legacy single-app deploys), so the migration is invisible.
+- To add a worker: chain `.worker({ name: 'mailer', command: 'node dist/worker.js' })` on your builder.
+
 ## [2.2.0] - 2026-05-17
 
 ### Added

CONTEXT.md

@@ -53,6 +53,12 @@ User-provided function that runs at a fixed point in the deployment lifecycle. `
 ### HealthCheck
 A remote curl-based check that verifies the backend application is responding after a deployment. Only runs for backend apps in zero-downtime mode.
 
+### Pm2Config
+The process-supervision section of a `ShipnodeConfig`. Contains an `apps` list — one entry per long-running process PM2 should supervise (web server, workers, etc.). Backend-only; frontend apps don't have one.
+
+### Pm2App
+A single PM2-supervised process within a `Pm2Config`. Carries its own `name`, optional `command` (defaults to the package manager's `start` script), `instances`, `maxMemory`, `env`, and optionally a `port`. **The entry with a `port` is the web app** — it receives the HTTP health check and Caddy upstream wiring. Entries without a `port` are workers: PM2 supervises them, but shipnode doesn't curl them. At most one entry may carry a `port`; zero is legal (worker-only deployments).
+
 ### Caddy
 The reverse proxy / static file server configured per-deployment. `CaddyService` writes config to `/etc/caddy/conf.d/` and reloads the service.
 

README.md

@@ -41,6 +41,42 @@ export default shipnode
   .build();
 ```
 
+### Web + workers
+
+A backend can run additional long-running processes alongside the web server. PM2 supervises all of them under one deployment.
+
+```ts
+export default shipnode
+  .backend()
+  .ssh({ host: '1.2.3.4', user: 'deploy' })
+  .deployTo('/var/www/myapp')
+  .pm2('myapp', { instances: 2 })
+  .port(3000)
+  .worker({ name: 'mailer', command: 'node dist/worker.js' })
+  .worker({ name: 'cron', command: 'node dist/cron.js', env: { JOB: 'cleanup' } })
+  .build();
+```
+
+After deploy, `shipnode status` lists every app, and `shipnode restart` / `stop` / `logs` operate on the whole deployment by default. Use `--process <name>` to target a single one (use the short name from your config — shipnode handles the PM2-side naming):
+
+```bash
+shipnode logs --process mailer
+shipnode restart --process cron
+```
+
+> **Note on PM2 naming.** PM2 has a flat global namespace, so two deployments on the same host with a `worker` entry would collide. To prevent that, shipnode prefixes worker names with the deployment namespace when registering them with PM2. A config with `.pm2('api')` + `.worker({name: 'mailer'})` shows up in `pm2 list` as `api` and `api-mailer`. You always use the short name (`mailer`) in shipnode commands.
+
+A backend can also be **worker-only** (no web server, no domain) — useful for queue consumers or cron runners. Just omit `.port(...)` and `.domain(...)`:
+
+```ts
+export default shipnode
+  .backend()
+  .ssh({ host: '1.2.3.4', user: 'deploy' })
+  .deployTo('/var/www/queue-runner')
+  .worker({ name: 'jobs', command: 'node dist/worker.js' })
+  .build();
+```
+
 ### Frontend apps
 
 ```ts
@@ -62,8 +98,9 @@ export default shipnode
 | `.backend()` / `.frontend()` | — | App type |
 | `.ssh({ host, user, port?, identityFile? })` | port 22 | SSH connection |
 | `.deployTo(path)` | `/var/www/app` | Remote deploy path |
-| `.pm2(name, opts?)` | — | PM2 process name + options |
-| `.port(n)` | `3000` | Backend port |
+| `.pm2(name, opts?)` | — | PM2 process name + options (the web app) |
+| `.port(n)` | `3000` | Backend port (marks the entry as the web app) |
+| `.worker({ name, command, ... })` | — | Add a worker process supervised alongside the web app |
 | `.domain(d)` | — | Domain for Caddy config |
 | `.nodeVersion(v)` | `lts` | Node version (via mise) |
 | `.pkgManager(pm)` | auto-detected | `npm` \| `yarn` \| `pnpm` \| `bun` |
@@ -115,11 +152,14 @@ shipnode run bash              # Open an interactive shell
 ### Process management
 
 ```bash
-shipnode logs                  # Tail PM2 logs (last 100 lines)
-shipnode logs --lines 500      # Show 500 lines
-shipnode restart               # Reload PM2 with --update-env
-shipnode stop                  # Stop the application
-shipnode metrics               # Open PM2 monit dashboard
+shipnode logs                       # Tail PM2 logs for the whole deployment
+shipnode logs --process mailer      # Tail logs for one app
+shipnode logs --lines 500           # Show 500 lines
+shipnode restart                    # Reload every app with --update-env
+shipnode restart --process mailer   # Reload one app
+shipnode stop                       # Stop every app in the deployment
+shipnode stop --process mailer      # Stop one app
+shipnode metrics                    # Open PM2 monit dashboard
 ```
 
 ### Security & maintenance

docs/adr/0001-per-release-ecosystem-file.md

@@ -0,0 +1,7 @@
+# Per-release PM2 ecosystem file
+
+The PM2 ecosystem file describes which processes constitute a deployed app (web + workers). It used to live at `/remotePath/shared/ecosystem.config.cjs` — shared across releases — which was fine when the ecosystem was effectively constant. Once apps can declare arbitrary workers, the ecosystem becomes part of the application's shape and can differ from one release to the next.
+
+We write the ecosystem into `<releasePath>/ecosystem.config.cjs` instead, and PM2 commands target `<remotePath>/current/ecosystem.config.cjs` (stable across rollbacks because the `current` symlink always points at the active release). Rollback then restores the symlink *and*, transparently, the ecosystem that matches that release's code — a worker added in v2 won't crash-loop after rolling back to v1.
+
+The alternative was snapshotting the shared ecosystem inside `ReleaseManager` metadata and restoring it on rollback. That works but invents a new "ecosystem snapshot" concept parallel to the existing release boundary, when the release boundary already means exactly this.

docs/adr/0002-port-presence-determines-web-app.md

@@ -0,0 +1,7 @@
+# Port presence determines the web app; no `type` discriminator on `Pm2App`
+
+`pm2.apps` is a uniform list of processes — web servers and workers share the same shape. The web app is identified by carrying a `port`; workers omit it. There is deliberately no `type: 'web' | 'worker'` field.
+
+The discriminator would be a second source of truth that can disagree with itself: it would still be the `port` that drives the health check and Caddy upstream, so a `type: 'web'` entry without a port (or a `type: 'worker'` entry with one) would have to be rejected anyway. Encoding the same fact twice invites bugs. `assembleConfig` enforces: at most one port-bearing entry; zero is legal (worker-only deployments); a `domain` requires a web entry.
+
+The cost is that "which entry is the web app?" isn't visible from a glance at one entry — you have to look at the list. We accept that in exchange for not maintaining a tagged union whose tag is redundant with its data.

src/cli/commands/cloudflare.ts

@@ -1,5 +1,6 @@
 import { runRemoteCommand } from '../runner.js';
 import { ui } from '../ui.js';
+import { getWebApp } from '../../domain/pm2/apps.js';
 
 const CF_API = 'https://api.cloudflare.com/client/v4';
 
@@ -108,8 +109,12 @@ export async function cmdCloudflareInit(
       }
 
       // Generate config and start as systemd service
-      const appIngress = cf.appHostname
-        ? `  - hostname: ${cf.appHostname}\n    service: http://localhost:${config.backend?.port ?? 3000}`
+      const webApp = getWebApp(config);
+      if (cf.appHostname && !webApp) {
+        throw new Error('cloudflare.appHostname is set but no pm2.apps entry declares a port — nothing to route HTTP to.');
+      }
+      const appIngress = cf.appHostname && webApp
+        ? `  - hostname: ${cf.appHostname}\n    service: http://localhost:${webApp.port}`
         : '';
       const sshIngress = cf.sshHostname
         ? `  - hostname: ${cf.sshHostname}\n    service: ssh://localhost:22`
@@ -138,19 +143,23 @@ ${sshIngress}
 
       // Setup firewall lockdown if requested
       if (cf.lockdownFirewall) {
-        ui.info('Locking down firewall to Cloudflare IPs only...');
-        const cfIpv4 = await cfFetch('/ips', cfApiOpts) as { ipv4_cidrs: string[] };
-        for (const cidr of cfIpv4.ipv4_cidrs ?? []) {
+        if (!webApp) {
+          ui.warn('Skipping firewall lockdown: no web app (no pm2.apps entry with a port) to expose.');
+        } else {
+          ui.info('Locking down firewall to Cloudflare IPs only...');
+          const cfIpv4 = await cfFetch('/ips', cfApiOpts) as { ipv4_cidrs: string[] };
+          for (const cidr of cfIpv4.ipv4_cidrs ?? []) {
+            await executor.exec(
+              `SUDO=""; [ "$EUID" -ne 0 ] && SUDO="sudo"; ` +
+              `$SUDO ufw allow from ${cidr} to any port ${webApp.port} 2>/dev/null || true`,
+            );
+          }
           await executor.exec(
             `SUDO=""; [ "$EUID" -ne 0 ] && SUDO="sudo"; ` +
-            `$SUDO ufw allow from ${cidr} to any port ${config.backend?.port ?? 3000} 2>/dev/null || true`,
+            `$SUDO ufw deny ${webApp.port} 2>/dev/null || true`,
           );
+          ui.success('Firewall locked to Cloudflare IPs');
         }
-        await executor.exec(
-          `SUDO=""; [ "$EUID" -ne 0 ] && SUDO="sudo"; ` +
-          `$SUDO ufw deny ${config.backend?.port ?? 3000} 2>/dev/null || true`,
-        );
-        ui.success('Firewall locked to Cloudflare IPs');
       }
 
       // Setup Access policy if accessEmails present

src/cli/commands/config.ts

@@ -23,17 +23,17 @@ export async function cmdConfigShow(cwd: string, options: { config?: string }):
       ]);
 
       if (config.pm2) {
-        ui.section('PM2', [
-          ['name', config.pm2.name],
-          ...(config.pm2.instances !== undefined ? [['instances', String(config.pm2.instances)] as [string, string]] : []),
-          ...(config.pm2.maxMemory !== undefined ? [['maxMemory', config.pm2.maxMemory] as [string, string]] : []),
-        ]);
-      }
-
-      if (config.backend) {
-        ui.section('Backend', [
-          ['port', String(config.backend.port)],
-        ]);
+        for (const app of config.pm2.apps) {
+          const rows: [string, string][] = [['name', app.name]];
+          if (app.command) rows.push(['command', app.command]);
+          if (app.port !== undefined) rows.push(['port', String(app.port)]);
+          if (app.instances !== undefined) rows.push(['instances', String(app.instances)]);
+          if (app.maxMemory !== undefined) rows.push(['maxMemory', app.maxMemory]);
+          if (app.env) {
+            for (const [k, v] of Object.entries(app.env)) rows.push([`env.${k}`, v]);
+          }
+          ui.section(app.port !== undefined ? `PM2 app: ${app.name} (web)` : `PM2 app: ${app.name}`, rows);
+        }
       }
 
       if (config.domain) {

src/cli/commands/deploy.ts

@@ -5,6 +5,7 @@ import { LoggingExecutor } from '../../infrastructure/ssh/logging-executor.js';
 import { runRemoteCommand } from '../runner.js';
 import { ui } from '../ui.js';
 import type { ShipnodeConfig } from '../../shared/types.js';
+import { getDeploymentName, getWebApp } from '../../domain/pm2/apps.js';
 
 export async function cmdDeploy(cwd: string, options: { dryRun?: boolean; skipBuild?: boolean; config?: string }): Promise<void> {
   const config = await loadConfig(cwd, options.config);
@@ -18,7 +19,7 @@ export async function cmdDeploy(cwd: string, options: { dryRun?: boolean; skipBu
     cwd,
     async ({ config, executor }) => {
       ui.banner();
-      ui.step(`Deploying ${chalk.bold(config.pm2?.name ?? config.app)} → ${config.ssh.user}@${config.ssh.host}`);
+      ui.step(`Deploying ${chalk.bold(getDeploymentName(config) ?? config.app)} → ${config.ssh.user}@${config.ssh.host}`);
 
       const deployer = new DeployService(new LoggingExecutor(executor), config, cwd);
       await deployer.execute(options.skipBuild ?? false);
@@ -46,8 +47,13 @@ function printDryRun(config: ShipnodeConfig, skipBuild: boolean): void {
   ];
 
   if (config.app === 'backend') {
-    if (config.pm2?.name) serverRows.push(['PM2 name', config.pm2.name]);
-    serverRows.push(['Port', String(config.backend?.port ?? 3000)]);
+    const apps = config.pm2?.apps ?? [];
+    if (apps.length) {
+      serverRows.push(['PM2 deployment', getDeploymentName(config) ?? '']);
+      serverRows.push(['PM2 apps', apps.map((a) => a.port !== undefined ? `${a.name}(web:${a.port})` : a.name).join(', ')]);
+    }
+    const web = getWebApp(config);
+    if (web) serverRows.push(['Port', String(web.port)]);
   }
 
   if (config.domain) serverRows.push(['Domain', config.domain]);

src/cli/commands/doctor.ts

@@ -19,7 +19,7 @@ export async function cmdDoctor(cwd: string, options: { config?: string; securit
   );
 }
 
-function checkLocal(config: { ssh: { host?: string; user?: string }; remotePath?: string; app: string; pm2?: { name?: string } }): void {
+function checkLocal(config: { ssh: { host?: string; user?: string }; remotePath?: string; app: string; pm2?: { apps: unknown[] } }): void {
   ui.info('Checking local configuration...');
 
   const issues: string[] = [];
@@ -36,8 +36,8 @@ function checkLocal(config: { ssh: { host?: string; user?: string }; remotePath?
     issues.push('Remote path is not configured');
   }
 
-  if (config.app === 'backend' && !config.pm2?.name) {
-    issues.push('PM2 app name is not configured for backend app');
+  if (config.app === 'backend' && !config.pm2?.apps.length) {
+    issues.push('PM2 apps are not configured for backend app');
   }
 
   if (issues.length === 0) {

src/cli/commands/env.ts

@@ -3,6 +3,7 @@ import { pathExists } from 'fs-extra';
 import { resolve } from 'path';
 import { runRemoteCommand } from '../runner.js';
 import { ui } from '../ui.js';
+import { getDeploymentName } from '../../domain/pm2/apps.js';
 
 export async function cmdEnv(
   cwd: string,
@@ -38,21 +39,22 @@ export async function cmdEnv(
         ui.success('Linked shared .env to current release');
       }
 
-      if (config.app === 'backend' && config.pm2?.name) {
+      const namespace = getDeploymentName(config);
+      if (config.app === 'backend' && namespace) {
         const nodeVersion = config.nodeVersion === 'lts' ? '24' : config.nodeVersion;
         const mise = `export PATH="$HOME/.local/bin:$HOME/.local/share/mise/shims:$PATH"`;
         const checkResult = await executor.exec(
-          `${mise}; mise exec node@${nodeVersion} -- pm2 describe ${config.pm2.name} 2>/dev/null && echo "running" || echo "stopped"`,
+          `${mise}; mise exec node@${nodeVersion} -- pm2 describe ${namespace} 2>/dev/null && echo "running" || echo "stopped"`,
         );
 
         if (checkResult.stdout.includes('running')) {
-          ui.info('Restarting app to reload environment variables...');
+          ui.info('Reloading deployment to pick up environment variables...');
           await executor.exec(
-            `${mise}; mise exec node@${nodeVersion} -- pm2 reload ${config.pm2.name} --update-env`,
+            `${mise}; mise exec node@${nodeVersion} -- pm2 reload ${namespace} --update-env`,
           );
-          ui.success('App restarted with new environment variables');
+          ui.success('Deployment reloaded with new environment variables');
         } else {
-          ui.warn('App not running. Variables will be loaded on next deploy.');
+          ui.warn('Deployment not running. Variables will be loaded on next deploy.');
         }
       }
     },