Skip to content

Commit 1d69d1c

Browse files
AlexgodorojaAlex Godorojaclaude
authored
cli: full CLI-app support (adapter hardening, publish-api, proc.exec) (#31)
Make the cli backend archetype real and end-to-end publishable, so an existing CLI can be fronted as an app: 'pilotctl appstore call <app> <args>' translates into a local subprocess invocation. Scaffolder / generated adapter: - Fix the cli main: it never compiled (http-only cfg leaked in) — now builds. - Passthrough method shape: a verbatim {"args":[...]} argv fronts the whole CLI, so every subcommand is reachable without enumerating it. - Hardened runner: scrubbed child env (+ backend.env_passthrough allowlist), bounded output, structured non-zero exit, deterministic flags, missing-param errors, stdin. No shell (argv exec'd directly). - Manifest emits the proc.exec grant (scoped to the command) and protection guarded; bump app-store to the version that knows proc.exec. publish-api (internal/publish + admin): - Submission model gains a cli backend (type/command/env_passthrough) and a per-method cli route (args/params_as_flags/passthrough); Validate, ToConfig, and the live HelpPreview branch on backend type. Admin case report is backend-aware. Tests: - compile guard (parse-only test missed the unused-var that broke cli builds) - publish-path e2e: a proc.exec cli app builds + clears the catalogue gate - runtime e2e: the built adapter execs the fronted CLI over real IPC (enumerated + passthrough + non-zero exit). Co-authored-by: Alex Godoroja <alex@vulturelabs.io> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 1701c4a commit 1d69d1c

13 files changed

Lines changed: 819 additions & 94 deletions

File tree

cmd/publish-server/templates/case.html

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,15 +16,20 @@ <h1>{{.C.Submission.Listing.DisplayName}} <span class="soft">{{.C.Submission.Ver
1616
<tr><th>ID</th><td class="mono">{{.C.Submission.ID}}</td></tr>
1717
<tr><th>Version</th><td class="mono">{{.C.Submission.Version}}</td></tr>
1818
<tr><th>Description</th><td>{{.C.Submission.Description}}</td></tr>
19+
{{if .C.Submission.Backend.IsCLI}}
20+
<tr><th>Backend</th><td class="mono">cli: {{join .C.Submission.Backend.Command}}</td></tr>
21+
{{if .C.Submission.Backend.EnvPassthrough}}<tr><th>Env passthrough</th><td class="mono">{{join .C.Submission.Backend.EnvPassthrough}}</td></tr>{{end}}
22+
{{else}}
1923
<tr><th>Backend</th><td class="mono">{{.C.Submission.Backend.BaseURL}}</td></tr>
2024
{{range .C.Submission.Backend.Headers}}<tr><th>Header</th><td class="mono">{{.Name}}: {{.Value}}</td></tr>{{end}}
25+
{{end}}
2126
</table></div>
2227

2328
<div class="card review"><h2>Methods</h2><table class="methods">
24-
<tr><th>Method</th><th>Latency</th><th>Path</th><th>Description</th></tr>
29+
<tr><th>Method</th><th>Latency</th><th>Route</th><th>Description</th></tr>
2530
{{range .C.Submission.Methods}}<tr>
2631
<td class="mono">{{.Name}}</td><td>{{.Latency}}</td>
27-
<td class="mono">{{.HTTP.Verb}} {{.HTTP.Path}}</td><td>{{.Description}}</td>
32+
<td class="mono">{{if $.C.Submission.Backend.IsCLI}}{{if .CLI.Passthrough}}passthrough {args[]}{{else}}{{join .CLI.Args}}{{if .CLI.ParamsAsFlags}} +flags{{end}}{{end}}{{else}}{{.HTTP.Verb}} {{.HTTP.Path}}{{end}}</td><td>{{.Description}}</td>
2833
</tr>{{end}}
2934
</table></div>
3035

docs/CLI-ADAPTER.md

Lines changed: 47 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,57 @@
11
# CLI adapter archetype — design & the `proc.exec` capability
22

3-
`pilot-app` already **generates** a working CLI-backed adapter (`backend.type:
4-
cli`): `internal/backend/exec.go` runs a subprocess per method, substitutes
5-
`${field}` placeholders from the IPC payload, and returns stdout as the JSON
6-
reply. The generated Go compiles and serves IPC exactly like the HTTP archetype.
3+
`pilot-app` **generates** a working CLI-backed adapter (`backend.type: cli`):
4+
`internal/backend/exec.go` runs a subprocess per method and returns its output
5+
as the JSON reply. The generated Go compiles and serves IPC exactly like the
6+
HTTP archetype (a `go build ./...` regression test pins this).
77

88
What it can't do **yet** is install through the catalogue with permission to
99
exec, because the platform has no capability for "run a local process." This
1010
doc specifies the one platform change that unblocks it.
1111

12+
## Invocation model — enumerated vs passthrough
13+
14+
A method maps to a subprocess one of two ways:
15+
16+
- **Enumerated** (`cli: {args: [...]}`): a baked argv with `${field}`
17+
placeholders filled from the payload, optionally `params_as_flags: true` to
18+
append `--key value` for each payload field (sorted, deterministic). A missing
19+
`${field}` is a hard error, not a silently-empty arg. Use this to expose a
20+
curated, named surface (`weather.current`, `weather.forecast`).
21+
22+
- **Passthrough** (`cli: {passthrough: true}`): the caller supplies a verbatim
23+
`args` array, so **every** subcommand of the fronted CLI is reachable without
24+
enumerating it:
25+
26+
```
27+
pilotctl appstore call io.pilot.toolx toolx.exec '{"args":["status","--short"]}'
28+
```
29+
30+
is exactly `toolx status --short`. This is the "translate all CLI commands"
31+
shape: one method fronts the whole tool. Both shapes accept an optional
32+
`stdin` string piped to the child.
33+
34+
Because argv is exec'd directly (no shell), payload values can never be
35+
re-parsed as shell metacharacters. Passthrough is strictly more powerful than an
36+
enumerated surface — the caller chooses the subcommand and flags — so reserve it
37+
for trusted callers and keep CLI apps `guarded` (below).
38+
39+
## Hardening built into the runner
40+
41+
The generated `exec.go` is defensive by default:
42+
43+
- **Scrubbed environment** — the child inherits only a minimal baseline (`PATH`,
44+
`HOME`, locale, `TMPDIR`) plus vars the spec opts in via
45+
`backend.env_passthrough: [TOKEN, ...]`. The adapter's own environment (app
46+
identity, broker secrets) never leaks to the fronted CLI.
47+
- **Bounded output** — stdout/stderr are capped (4 MiB) so a runaway child can't
48+
OOM the adapter; truncation is flagged in the reply.
49+
- **Structured failures** — a non-zero exit is returned as
50+
`{"stdout","stderr","exit","truncated"}` rather than an opaque error, so the
51+
caller sees everything the CLI produced. Only spawn failures (binary missing)
52+
and timeouts surface as IPC errors; the per-method `timeout`/`duration` bounds
53+
the run and the child is killed on cancel.
54+
1255
## Why HTTP works today and CLI doesn't
1356

1457
An app's grants are the *only* thing the daemon authorizes (`pkg/manifest`).

go.mod

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ module github.com/pilot-protocol/app-template
33
go 1.25.0
44

55
require (
6-
github.com/pilot-protocol/app-store v1.0.1-beta.1.0.20260609061942-8852c785a264
6+
github.com/pilot-protocol/app-store v1.0.1-beta.1.0.20260622180016-07b4170265dc
77
github.com/sendgrid/sendgrid-go v3.16.1+incompatible
88
gopkg.in/yaml.v3 v3.0.1
99
modernc.org/sqlite v1.52.0

go.sum

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,8 +12,8 @@ github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWE
1212
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
1313
github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w=
1414
github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
15-
github.com/pilot-protocol/app-store v1.0.1-beta.1.0.20260609061942-8852c785a264 h1:NL9rFdakbVQ0V7xfJbCk8RJZSaQ1AmvdhAJwFIouMsk=
16-
github.com/pilot-protocol/app-store v1.0.1-beta.1.0.20260609061942-8852c785a264/go.mod h1:zoCxHYoNdj0V44OkG3Yzcye0jnwZDVUcJgAvR5Z1kwc=
15+
github.com/pilot-protocol/app-store v1.0.1-beta.1.0.20260622180016-07b4170265dc h1:Ze7h3rEPMhFaAyjNH9riySBs8HEeeoB3wODwtoLQ4Eo=
16+
github.com/pilot-protocol/app-store v1.0.1-beta.1.0.20260622180016-07b4170265dc/go.mod h1:leZPtX43gE2JB7xeljexXri81g6qhdZfYExLtzI+bhg=
1717
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
1818
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
1919
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=

internal/publish/cli_e2e_test.go

Lines changed: 160 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,160 @@
1+
package publish
2+
3+
import (
4+
"archive/tar"
5+
"bytes"
6+
"compress/gzip"
7+
"encoding/json"
8+
"io"
9+
"strings"
10+
"testing"
11+
)
12+
13+
// sampleCLISubmission is a CLI-backed submission with both method shapes:
14+
// an enumerated subcommand (status) and a passthrough that fronts the whole
15+
// tool (exec) — the "translate all CLI commands" surface.
16+
func sampleCLISubmission() Submission {
17+
return Submission{
18+
ID: "io.pilot.gh", Version: "0.1.0", Description: "Fronts the gh CLI over the app store.", Email: "dev@acme.example",
19+
Backend: SubBackend{Type: "cli", Command: []string{"gh"}, EnvPassthrough: []string{"GH_TOKEN"}},
20+
Methods: []SubMethod{
21+
{Name: "gh.status", Description: "Show gh auth status.", Latency: "fast",
22+
CLI: SubCLIRoute{Args: []string{"auth", "status"}}},
23+
{Name: "gh.exec", Description: "Run any gh subcommand.", Latency: "med",
24+
Params: []SubParam{{Name: "args", Type: "array", Description: "verbatim argv forwarded to gh"}},
25+
CLI: SubCLIRoute{Passthrough: true}},
26+
},
27+
Listing: SubListing{DisplayName: "GitHub CLI", License: "MIT", Categories: []string{"dev"}},
28+
Vendor: SubVendor{Name: "Acme", AgentUsage: "agents drive gh", Capabilities: "github"},
29+
}
30+
}
31+
32+
// fileFromTarball pulls one path out of a gzipped bundle tarball.
33+
func fileFromTarball(t *testing.T, tarball []byte, name string) []byte {
34+
t.Helper()
35+
gz, err := gzip.NewReader(bytes.NewReader(tarball))
36+
if err != nil {
37+
t.Fatal(err)
38+
}
39+
tr := tar.NewReader(gz)
40+
for {
41+
h, err := tr.Next()
42+
if err == io.EOF {
43+
break
44+
}
45+
if err != nil {
46+
t.Fatal(err)
47+
}
48+
if h.Name == name {
49+
b, _ := io.ReadAll(tr)
50+
return b
51+
}
52+
}
53+
t.Fatalf("%s not found in tarball", name)
54+
return nil
55+
}
56+
57+
// TestCLISubmissionValidates pins the submission-level CLI rules: a well-formed
58+
// cli submission passes, and the common misconfigurations are rejected with a
59+
// friendly message (fast, no build).
60+
func TestCLISubmissionValidates(t *testing.T) {
61+
t.Parallel()
62+
if errs := sampleCLISubmission().Validate(); len(errs) != 0 {
63+
t.Fatalf("a well-formed cli submission must validate, got: %v", errs)
64+
}
65+
66+
noCmd := sampleCLISubmission()
67+
noCmd.Backend.Command = nil
68+
if !hasSubErr(noCmd.Validate(), "CLI backend requires a command") {
69+
t.Errorf("missing command should be rejected, got: %v", noCmd.Validate())
70+
}
71+
72+
badPass := sampleCLISubmission()
73+
badPass.Methods[1].CLI.Args = []string{"oops"} // passthrough + args is contradictory
74+
if !hasSubErr(badPass.Validate(), "passthrough takes argv") {
75+
t.Errorf("passthrough+args should be rejected, got: %v", badPass.Validate())
76+
}
77+
78+
emptyRoute := sampleCLISubmission()
79+
emptyRoute.Methods[0].CLI = SubCLIRoute{} // no args, no flags, no passthrough
80+
if !hasSubErr(emptyRoute.Validate(), "needs args, params_as_flags, or passthrough") {
81+
t.Errorf("empty cli route should be rejected, got: %v", emptyRoute.Validate())
82+
}
83+
}
84+
85+
// TestCLIHelpPreviewShowsPassthroughArgs verifies the live preview renders the
86+
// correct pilotctl line for a passthrough method: an {"args":[...]} payload, not
87+
// a params skeleton.
88+
func TestCLIHelpPreviewShowsPassthroughArgs(t *testing.T) {
89+
t.Parallel()
90+
_, cmds := sampleCLISubmission().HelpPreview()
91+
joined := strings.Join(cmds, "\n")
92+
if !strings.Contains(joined, `call io.pilot.gh gh.exec '{"args":[`) {
93+
t.Errorf("passthrough method should preview an args[] payload; got:\n%s", joined)
94+
}
95+
if !strings.Contains(joined, "call io.pilot.gh gh.status") {
96+
t.Errorf("enumerated method missing from preview:\n%s", joined)
97+
}
98+
}
99+
100+
// TestCLISubmissionBuildsAndVerifies is the publish-path e2e: a CLI submission
101+
// builds through the real pipeline (scaffold → cross-compile → sign → catalogue
102+
// self-verify) for every platform. Because BuildBundle self-verifies through the
103+
// exact catalogue gate, a successful build PROVES the proc.exec manifest passes
104+
// validation — the gate that rejected cli apps before this capability landed.
105+
// It then asserts the shipped manifest declares proc.exec (scoped to the command)
106+
// and is guarded.
107+
func TestCLISubmissionBuildsAndVerifies(t *testing.T) {
108+
if testing.Short() {
109+
t.Skip("cross-compiles the cli adapter for all platforms; skipped under -short")
110+
}
111+
priv, err := LoadOrCreateKey(t.TempDir() + "/k.key")
112+
if err != nil {
113+
t.Fatal(err)
114+
}
115+
sub := sampleCLISubmission()
116+
if errs := sub.Validate(); len(errs) != 0 {
117+
t.Fatalf("submission invalid: %v", errs)
118+
}
119+
120+
b, err := BuildBundle(sub.ToConfig(), priv)
121+
if err != nil {
122+
t.Fatalf("BuildBundle (implies catalogue self-verify) failed for a proc.exec app: %v", err)
123+
}
124+
if len(b.Platforms) != len(DefaultPlatforms) {
125+
t.Fatalf("want %d platforms, got %d", len(DefaultPlatforms), len(b.Platforms))
126+
}
127+
128+
// The shipped manifest must carry the hardened proc.exec grant + guarded.
129+
mfRaw := fileFromTarball(t, b.Primary().Tarball, "./manifest.json")
130+
var mf struct {
131+
Protection string `json:"protection"`
132+
Grants []struct {
133+
Cap, Target string
134+
} `json:"grants"`
135+
}
136+
if err := json.Unmarshal(mfRaw, &mf); err != nil {
137+
t.Fatalf("parse shipped manifest: %v", err)
138+
}
139+
if mf.Protection != "guarded" {
140+
t.Errorf("cli app must ship protection=guarded, got %q", mf.Protection)
141+
}
142+
var procExec string
143+
for _, g := range mf.Grants {
144+
if g.Cap == "proc.exec" {
145+
procExec = g.Target
146+
}
147+
}
148+
if procExec != "gh" {
149+
t.Errorf("manifest must declare proc.exec scoped to the command (target=gh), got %q", procExec)
150+
}
151+
}
152+
153+
func hasSubErr(errs []string, substr string) bool {
154+
for _, e := range errs {
155+
if strings.Contains(e, substr) {
156+
return true
157+
}
158+
}
159+
return false
160+
}

0 commit comments

Comments
 (0)