Skip to content

fix(cli): surface deploy remote output, write the audit log, seed flat-alias ssh config (#2957) #3166

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
bpamiri merged 1 commit into from
Jun 12, 2026
Merged

fix(cli): surface deploy remote output, write the audit log, seed flat-alias ssh config (#2957) #3166

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions changelog.d/deploy-w3-observability.fixed.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
- Deploy read verbs (`wheels deploy audit`, `details`, `app logs`/`details`/`containers`/`images`, `lock status`) now surface the remote output host-prefixed (`[host] line`) instead of returning only a host-count summary — the dispatch layer previously discarded every `ssh.run()` result ([#2957](https://github.com/wheels-dev/wheels/issues/2957) DEP-6a).
- `wheels deploy` / `rollback` / `setup` now write the on-server audit trail (`/tmp/kamal-audit.log`): deploys are bracketed by `started deploy of version X` / `completed deploy of version X` records, rollbacks and setup's accessory boots are recorded, and audit writes are tolerated-on-failure so observability can never fail a deploy. Previously `AuditorCommands.record()` had zero call sites and `wheels deploy audit` tailed a file the tool never created ([#2957](https://github.com/wheels-dev/wheels/issues/2957) DEP-6b).
- The flat `wheels deploy bootstrap` / `wheels deploy exec` aliases — the only CLI-reachable form of these verbs ([#2677](https://github.com/wheels-dev/wheels/issues/2677)) — now seed their SSH pool from `deploy.yml`'s `ssh:` block like every other deploy verb; previously they constructed a bare pool that ignored `ssh.user`, `ssh.port`, and `keys:` and always connected as `root@host:22` ([#2957](https://github.com/wheels-dev/wheels/issues/2957) DEP-7).
8 changes: 6 additions & 2 deletions cli/lucli/Module.cfc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2416,8 +2416,11 @@ component extends="modules.BaseModule" {
// branch below is retained for Kamal parity and direct callers
// (MCP, internal tests) that don't go through LuCLI's picocli root.
case "bootstrap":
// #2957 DEP-7: build the pool from deploy.yml's ssh: block like
// every other verb — a bare `new SshPool()` here meant the only
// CLI-reachable bootstrap form ignored ssh.user/port/keys.
var bootstrapCli = new modules.wheels.services.deploy.cli.DeployServerCli(
new modules.wheels.services.deploy.lib.SshPool()
$deployBuildSshPool(opts.configPath)
);
return bootstrapCli.bootstrap(opts);
case "exec":
Expand All @@ -2430,8 +2433,9 @@ component extends="modules.BaseModule" {
arrayAppend(execCmdParts, positional[ei]);
}
opts.cmd = arrayToList(execCmdParts, " ");
// #2957 DEP-7: same ssh-config seeding as the nested `server` branch.
var execCli = new modules.wheels.services.deploy.cli.DeployServerCli(
new modules.wheels.services.deploy.lib.SshPool()
$deployBuildSshPool(opts.configPath)
);
return execCli.exec(opts);
case "server":
Expand Down
52 changes: 47 additions & 5 deletions cli/lucli/services/deploy/cli/DeployAppCli.cfc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ component {
variables.sshPool = arguments.sshPool;
variables.loader = new modules.wheels.services.deploy.config.ConfigLoader();
variables.dryRunBuffer = [];
// Host-prefixed remote stdout collected by read verbs (#2957 DEP-6a).
variables.liveOutput = [];
return this;
}

Expand Down Expand Up @@ -46,21 +48,21 @@ component {
public string function details(required struct opts) {
var n = $forEachHost(arguments.opts, function(cmds, role, version) {
return cmds.status(role, version);
});
}, {collect: true});
return $renderResult(arguments.opts, "Collected app details on " & n & " host(s)");
}

public string function containers(required struct opts) {
var n = $forEachHost(arguments.opts, function(cmds, role, version) {
return cmds.containers();
}, {versionOptional: true});
}, {versionOptional: true, collect: true});
return $renderResult(arguments.opts, "Listed app containers on " & n & " host(s)");
}

public string function images(required struct opts) {
var n = $forEachHost(arguments.opts, function(cmds, role, version) {
return cmds.images();
}, {versionOptional: true});
}, {versionOptional: true, collect: true});
return $renderResult(arguments.opts, "Listed app images on " & n & " host(s)");
}

Expand All @@ -72,7 +74,7 @@ component {
};
var n = $forEachHost(arguments.opts, function(cmds, role, version) {
return cmds.logs(logOpts);
}, {versionOptional: true});
}, {versionOptional: true, collect: true});
return $renderResult(arguments.opts, "Tailed app logs on " & n & " host(s)");
}

Expand Down Expand Up @@ -101,6 +103,7 @@ component {

private numeric function $forEachHost(required struct opts, required any cmdFn, struct flags = {}) {
arrayClear(variables.dryRunBuffer);
arrayClear(variables.liveOutput);
var cfg = variables.loader.load(
arguments.opts.configPath,
{destination: arguments.opts.destination ?: ""}
Expand All @@ -112,6 +115,9 @@ component {
message="This verb requires --version (e.g. --version=v1.2.3). On older wrappers that pre-date the picocli rewrite, pass --release instead.");
}
var dryRun = arguments.opts.dryRun ?: false;
// collect=true marks a read verb: remote stdout is gathered into
// liveOutput and appended to the live-mode summary (#2957 DEP-6a).
var collect = arguments.flags.collect ?: false;
var appCmds = new modules.wheels.services.deploy.commands.AppCommands(cfg);
var roleFilter = arguments.opts.role ?: "";
var hostCount = 0;
Expand All @@ -120,7 +126,13 @@ component {
if (len(roleFilter) && role.name() != roleFilter) continue;
for (var host in role.hosts()) {
var cmd = arguments.cmdFn(appCmds, role, version);
$dispatch([host], cmd, dryRun);
if (collect) {
for (var l in $dispatchCollect([host], cmd, dryRun)) {
arrayAppend(variables.liveOutput, l);
}
} else {
$dispatch([host], cmd, dryRun);
}
hostCount++;
}
}
Expand All @@ -131,6 +143,9 @@ component {
if (arguments.opts.dryRun ?: false) {
return arrayToList(variables.dryRunBuffer, chr(10));
}
if (arrayLen(variables.liveOutput)) {
return arguments.summary & chr(10) & arrayToList(variables.liveOutput, chr(10));
}
return arguments.summary;
}

Expand All @@ -145,4 +160,31 @@ component {
var doRaise = !arguments.allowFail;
variables.sshPool.onEach(arguments.hosts, function(ssh, host) { ssh.run(c, {raise: doRaise}); });
}

/**
* Dispatch + collect host-prefixed remote output (`[host] line`) for
* read verbs (#2957 DEP-6a). Sequential so output order is stable and
* the collection stays single-threaded. Mirrors
* DeployMainCli.$dispatchCollect — keep the two in lockstep.
*/
private array function $dispatchCollect(required array hosts, required string cmd, required boolean dryRun, boolean allowFail = false) {
if (arguments.dryRun) {
for (var h in arguments.hosts) arrayAppend(variables.dryRunBuffer, "[" & h & "] " & arguments.cmd);
return [];
}
var c = arguments.cmd;
var doRaise = !arguments.allowFail;
var ctx = {lines: []};
variables.sshPool.sequential(arguments.hosts, function(ssh, host) {
var res = ssh.run(c, {raise: doRaise});
var text = trim(res.stdout ?: "");
if (!len(text)) text = trim(res.stderr ?: "");
if (!len(text)) return;
text = replace(text, chr(13), "", "all");
for (var line in listToArray(text, chr(10))) {
arrayAppend(ctx.lines, "[" & host & "] " & line);
}
});
return ctx.lines;
}
}
37 changes: 35 additions & 2 deletions cli/lucli/services/deploy/cli/DeployLockCli.cfc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,12 @@ component {
// readlink exits nonzero when the lock file is missing — which is
// exactly what the operator wants to learn from `status`. Treat that
// as advisory output, not a thrown error. #2696.
$dispatchAny($allHosts(cfg), lock.status(), dryRun, true);
return $renderResult(arguments.opts, "Checked deploy lock status for " & cfg.service());
// #2957 DEP-6a: surface readlink's output (the lock holder on stdout,
// or the "No such file" diagnostic on stderr) instead of dropping it.
var lines = $dispatchAnyCollect($allHosts(cfg), lock.status(), dryRun, true);
var summary = "Checked deploy lock status for " & cfg.service();
if (arrayLen(lines)) summary &= chr(10) & arrayToList(lines, chr(10));
return $renderResult(arguments.opts, summary);
}

private string function $renderResult(required struct opts, required string summary) {
Expand Down Expand Up @@ -87,6 +91,35 @@ component {
variables.sshPool.onAny(arguments.hosts, function(ssh, host) { ssh.run(c, {raise: doRaise}); });
}

/**
* Like $dispatchAny, but returns the remote output host-prefixed
* (`[host] line`). Stdout wins; stderr is the fallback so tolerated
* failures (allowFail) still surface their diagnostic. #2957 DEP-6a.
*/
private array function $dispatchAnyCollect(required array hosts, required string cmd, required boolean dryRun, boolean allowFail = false) {
if (arguments.dryRun) {
if (arrayLen(arguments.hosts)) {
arrayAppend(variables.dryRunBuffer, "[" & arguments.hosts[1] & "] " & arguments.cmd);
}
return [];
}
var c = arguments.cmd;
var doRaise = !arguments.allowFail;
// Closures can't write outer locals reliably — collect via a shared struct.
var ctx = {lines: []};
variables.sshPool.onAny(arguments.hosts, function(ssh, host) {
var res = ssh.run(c, {raise: doRaise});
var text = trim(res.stdout ?: "");
if (!len(text)) text = trim(res.stderr ?: "");
if (!len(text)) return;
text = replace(text, chr(13), "", "all");
for (var line in listToArray(text, chr(10))) {
arrayAppend(ctx.lines, "[" & host & "] " & line);
}
});
return ctx.lines;
}

private string function $currentUser() {
var sys = createObject("java", "java.lang.System");
var user = sys.getenv("USER");
Expand Down
84 changes: 76 additions & 8 deletions cli/lucli/services/deploy/cli/DeployMainCli.cfc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ component {
var builder = new modules.wheels.services.deploy.commands.BuilderCommands(cfg);
var dockerCmds = new modules.wheels.services.deploy.commands.DockerCommands(cfg);
var lock = new modules.wheels.services.deploy.commands.LockCommands(cfg);
var auditor = new modules.wheels.services.deploy.commands.AuditorCommands(cfg);
var hooks = new modules.wheels.services.deploy.commands.HookCommands(
cfg,
{projectRoot: variables.projectRoot}
Expand All @@ -102,6 +103,11 @@ component {
);

try {
// #2957 DEP-6b: write the on-server audit trail that the
// `audit` verb tails. Records bracket the work (started /
// completed) and are dispatched allowFail — observability
// must never fail a deploy.
$dispatch(hosts, auditor.record("started deploy of version " & ver), dryRun, true);
$dispatch(hosts, builder.pull(ver), dryRun);
// Fresh-host bootstrap (#2957 DEP-5c): every `docker run`
// below joins --network kamal, so the network must exist
Expand Down Expand Up @@ -130,6 +136,7 @@ component {
}
}
}
$dispatch(hosts, auditor.record("completed deploy of version " & ver), dryRun, true);
} finally {
// Tolerate release failures so they can never shadow the
// original deploy exception inside this finally block. rm -f
Expand Down Expand Up @@ -196,6 +203,10 @@ component {
arrayAppend(hostList, host);
}
}
// #2957 DEP-6b: rollbacks are exactly what an operator greps the audit
// log for later. allowFail — never let the log line fail the rollback.
var auditor = new modules.wheels.services.deploy.commands.AuditorCommands(cfg);
$dispatch(hostList, auditor.record("rolled back to version " & arguments.opts.version), dryRun, true);

return $renderResult(
arguments.opts,
Expand All @@ -222,12 +233,15 @@ component {
if (arrayLen(cfg.accessories())) {
var dockerCmds = new modules.wheels.services.deploy.commands.DockerCommands(cfg);
var accCmds = new modules.wheels.services.deploy.commands.AccessoryCommands(cfg);
var auditor = new modules.wheels.services.deploy.commands.AuditorCommands(cfg);
for (var acc in cfg.accessories()) {
// Accessories join --network kamal too, and may live on
// hosts outside the app roles — ensure the network there
// before the accessory container runs (#2957 DEP-5c).
$dispatch(acc.hosts(), dockerCmds.ensure_network("kamal"), dryRun);
$dispatch(acc.hosts(), accCmds.run(acc), dryRun);
// #2957 DEP-6b: audit trail for setup's accessory phase.
$dispatch(acc.hosts(), auditor.record("booted accessory " & acc.name()), dryRun, true);
}
}

Expand All @@ -248,11 +262,15 @@ component {
var cmd = "tail -n " & tail & " /tmp/kamal-audit.log";
var hosts = $allHosts(cfg);
var dryRun = arguments.opts.dryRun ?: false;
$dispatch(hosts, cmd, dryRun);
// #2957 DEP-6a: surface the actual log content, not just a summary.
var lines = $dispatchCollect(hosts, cmd, dryRun);
return $renderResult(
arguments.opts,
"Tailed audit log (last " & tail & " lines) on "
& arrayLen(hosts) & " host(s): " & arrayToList(hosts, ", ")
$withRemoteOutput(
"Tailed audit log (last " & tail & " lines) on "
& arrayLen(hosts) & " host(s): " & arrayToList(hosts, ", "),
lines
)
);
}

Expand Down Expand Up @@ -295,21 +313,26 @@ component {
var proxyCmds = new modules.wheels.services.deploy.commands.ProxyCommands(cfg);
var hosts = $allHosts(cfg);

// #2957 DEP-6a: read verbs surface the remote stdout, host-prefixed.
var lines = [];
// app details: docker ps filtered by service label
$dispatch(hosts, appCmds.containers(), dryRun);
for (var l in $dispatchCollect(hosts, appCmds.containers(), dryRun)) arrayAppend(lines, l);
// proxy details: docker ps filtered by kamal-proxy name
$dispatch(hosts, proxyCmds.details(), dryRun);
for (var l in $dispatchCollect(hosts, proxyCmds.details(), dryRun)) arrayAppend(lines, l);
// accessory details (if any)
if (arrayLen(cfg.accessories())) {
var accCmds = new modules.wheels.services.deploy.commands.AccessoryCommands(cfg);
for (var acc in cfg.accessories()) {
$dispatch(acc.hosts(), accCmds.details(acc), dryRun);
for (var l in $dispatchCollect(acc.hosts(), accCmds.details(acc), dryRun)) arrayAppend(lines, l);
}
}
return $renderResult(
arguments.opts,
"Collected app + proxy + accessory details from "
& arrayLen(hosts) & " host(s): " & arrayToList(hosts, ", ")
$withRemoteOutput(
"Collected app + proxy + accessory details from "
& arrayLen(hosts) & " host(s): " & arrayToList(hosts, ", "),
lines
)
);
}

Expand Down Expand Up @@ -499,6 +522,51 @@ component {
});
}

/**
* Dispatch a command and collect the remote output, host-prefixed
* (`[host] line`), in host order. Used by read verbs (`audit`,
* `details`) so live mode surfaces what the remote actually said
* instead of just a host-count summary (#2957 DEP-6a). Runs hosts
* sequentially — read output should arrive in a predictable order,
* and serial execution keeps the collection single-threaded.
* Falls back to stderr when a tolerated (allowFail) command produced
* no stdout, so e.g. "No such file" diagnostics still reach the user.
*/
private array function $dispatchCollect(
required array hosts,
required string cmd,
required boolean dryRun,
boolean allowFail = false
) {
if (arguments.dryRun) {
for (var h in arguments.hosts) {
arrayAppend(variables.dryRunBuffer, "[" & h & "] " & arguments.cmd);
}
return [];
}
var c = arguments.cmd;
var doRaise = !arguments.allowFail;
// Closures can't write outer locals reliably — collect via a shared struct.
var ctx = {lines: []};
variables.sshPool.sequential(arguments.hosts, function(ssh, host) {
var res = ssh.run(c, {raise: doRaise});
var text = trim(res.stdout ?: "");
if (!len(text)) text = trim(res.stderr ?: "");
if (!len(text)) return;
text = replace(text, chr(13), "", "all");
for (var line in listToArray(text, chr(10))) {
arrayAppend(ctx.lines, "[" & host & "] " & line);
}
});
return ctx.lines;
}

/** Append the remote output block to a live-mode summary (no-op when empty). */
private string function $withRemoteOutput(required string summary, required array lines) {
if (!arrayLen(arguments.lines)) return arguments.summary;
return arguments.summary & chr(10) & arrayToList(arguments.lines, chr(10));
}

/**
* Dispatch a single command to "any one" host — used for operations
* that only need to happen once across the fleet (lock acquire/release).
Expand Down
Loading
Loading