feat(appstore): pilotctl install --local for sideloaded apps (#240)

* feat(appstore): pilotctl install --local for sideloaded apps

Pairs with pilot-protocol/app-store#15. Local-path installs now
require an explicit --local flag and produce a `.sideloaded` marker
in the install dir; the supervisor uses that marker to skip
publisher-signature verification and enforce the sideload allow-list.

Behaviour changes:

  - `pilotctl appstore install <path>`                  → refused
  - `pilotctl appstore install <path> --local`          → sideload
  - `pilotctl appstore install <id>`  (catalogue ID)    → unchanged

When --local is passed:

  - manifest.EnforceSideloadPolicy(m) runs before staging; any
    grant outside the allow-list refuses the install up-front so
    users get a clear error instead of a silent supervisor-skip
    after the next rescan
  - `.sideloaded` is planted in the staging dir as mode 0o400 BEFORE
    the atomic rename, so there is no window where the dir appears
    under InstallRoot without the marker
  - install output prints the honest "manifest gate, not OS sandbox"
    caveat so users know what `--local` does and does not protect

`appstore list` surfaces a `[sideloaded]` badge on the version line
and adds the `sideloaded` field to the JSON output.

A small internal refactor: resolveInstallTarget now returns an
installSource tag (catalogue|local) so the install command can
branch on trust regime without re-doing catalogue lookup.

Test updates:

  - validManifestJSON now uses fs.read $APP/data instead of /tmp/data
    so the helper-built bundles satisfy the sideload policy and can
    be used in both the catalogue and sideload test paths
  - install/duplicate/text-mode tests pass --local since they install
    from a local bundle directory
  - SIDELOADED warning text is asserted in the text-mode install test
    so removing the warning would surface as a regression

Bumps the app-store dep to the feat/sideload-policy SHA. Will rebase
the version pin once that PR merges and a tagged release lands.

* docs(appstore): show --local in install help

Splits the install command into two lines in the appstore help —
catalogue install vs sideload — so users discover --local from
`pilotctl appstore` rather than via the trial-and-error of a
local-path install that gets refused.

* fix(ci): cli-reference-check captures stderr and tolerates --help exit 2

Two pre-existing bugs were masking each other in scripts/gen-cli-reference.sh:

1. \`pilotctl --help\` writes its banner to stderr (so that the same
   text shows for \`pilotctl <bad-flag>\`). The script only redirected
   stdout, so the help block was always empty.

2. \`pilotctl --help\` exits with code 2 by Go's flag convention. Under
   \`set -euo pipefail\` that aborted the whole script before the diff
   step ran.

Net effect: the workflow never modified docs/cli-reference.md, so the
\`git diff --exit-code\` step trivially passed. Any PR that touches
\`cmd/pilotctl/**\` triggered the gen step, hit the same exit-2 abort,
and \"passed\" for the wrong reason. PR #237/#238/#239 (catalogue-only
bumps) skipped the gate entirely because they don't match the path
filter, but the run history on main shows the check has been failing
silently on every code change since the workflow was added.

Fix:
  - 2>&1 so stderr lands in the doc
  - `|| true` so a non-zero exit doesn't abort the script

No content change to docs/cli-reference.md — the current committed
version already matches what the fixed script produces.

* deps: pin app-store to post-merge main SHA

Bumps github.com/pilot-protocol/app-store from the feat/sideload-policy
SHA to the post-merge main SHA (#15 squash-merged as 8852c785).
No code changes — the manifest API + supervisor scan code that ships
in v1.0.1-beta.1.0.20260609061942-8852c785a264 is byte-equivalent to
what was on the feat branch.

---------

Co-authored-by: Teodor Calin <teodor@vulturelabs.io>

Author: TeoSlayer

cmd/pilotctl/appstore.go

@@ -102,9 +102,11 @@ Usage:
   pilotctl appstore uninstall <id> --yes     remove an installed app from the install root
   pilotctl appstore verify <bundle-dir>      sha256-check a pre-install bundle against its manifest
   pilotctl appstore catalogue                list apps available for one-command install
-  pilotctl appstore install <app-id-or-dir> [--force]
+  pilotctl appstore install <app-id> [--force]
                                              install by catalogue ID (fetches + verifies + extracts)
-                                             OR by local bundle directory (offline / dev path)
+  pilotctl appstore install <bundle-dir> --local [--force]
+                                             sideload a local bundle (sandbox: fs.read/fs.write
+                                             under $APP, audit.log; no net, no key.sign, no hooks)
   pilotctl appstore gen-key <key-file>       generate a fresh ed25519 publisher keypair; prints the public side
   pilotctl appstore sign --key <key-file> <manifest>
                                              sign (or re-sign) a manifest's store.signature so the supervisor accepts it
@@ -145,6 +147,7 @@ type appListEntry struct {
 	BinaryPresent   bool     `json:"binary_present"`
 	Suspended       bool     `json:"suspended,omitempty"`
 	ManifestValid   bool     `json:"manifest_valid"`
+	Sideloaded      bool     `json:"sideloaded,omitempty"`
 }
 
 func cmdAppStoreList(_ []string) {
@@ -185,6 +188,7 @@ func cmdAppStoreList(_ []string) {
 		// budget is spent. Detecting it from disk lets list report
 		// "suspended" without daemon IPC.
 		_, suspErr := os.Stat(filepath.Join(dir, ".suspended"))
+		_, sideErr := os.Stat(filepath.Join(dir, manifest.SideloadMarkerName))
 		// Run the manifest's semantic Validate too — an app the
 		// supervisor will silently skip should be VISIBLY broken in
 		// list output, not look like a normal "stopped" app.
@@ -199,6 +203,7 @@ func cmdAppStoreList(_ []string) {
 			BinaryPresent:   binErr == nil,
 			Suspended:       suspErr == nil,
 			ManifestValid:   validationOK,
+			Sideloaded:      sideErr == nil,
 		})
 	}
 	sort.Slice(apps, func(i, j int) bool { return apps[i].ID < apps[j].ID })
@@ -226,7 +231,11 @@ func cmdAppStoreList(_ []string) {
 			state = "ready"
 		}
 		fmt.Printf("  %s\n", a.ID)
-		fmt.Printf("    version:  %s (manifest v%d, %s)\n", a.AppVersion, a.ManifestVersion, a.Protection)
+		versionLine := fmt.Sprintf("%s (manifest v%d, %s)", a.AppVersion, a.ManifestVersion, a.Protection)
+		if a.Sideloaded {
+			versionLine += " [sideloaded]"
+		}
+		fmt.Printf("    version:  %s\n", versionLine)
 		fmt.Printf("    state:    %s\n", state)
 		if len(a.Methods) > 0 {
 			fmt.Printf("    methods:  %v\n", a.Methods)
@@ -971,32 +980,45 @@ type installReport struct {
 func cmdAppStoreInstall(args []string) {
 	if len(args) < 1 {
 		fatalHint("invalid_argument",
-			"usage: pilotctl appstore install <app-id-or-dir> [--force]",
+			"usage: pilotctl appstore install <app-id-or-dir> [--force] [--local]",
 			"missing app id or bundle dir")
 	}
 	target := args[0]
 	force := false
+	allowLocal := false
 	for i := 1; i < len(args); i++ {
 		switch args[i] {
 		case "--force", "-f":
 			force = true
+		case "--local":
+			// Required acknowledgement when installing from a local
+			// directory. Catalogue installs ignore this; path installs
+			// without --local fail closed so a typo'd app id (which
+			// happens to also exist as a directory in the cwd) can't
+			// silently sideload an unsigned bundle.
+			allowLocal = true
 		default:
 			fatalHint("invalid_argument",
-				"available flags: --force",
+				"available flags: --force, --local",
 				"unknown install flag: %s", args[i])
 		}
 	}
 
-	// Resolve `target` to a local bundle dir. If it's a catalogue ID
-	// (e.g. "io.pilot.wallet"), fetch + verify the tarball and unpack
-	// into a tempdir. If it's already a directory, use it as-is. The
-	// rest of this function operates only on the directory.
-	bundleDir, err := resolveInstallTarget(target)
+	// Resolve `target` to a local bundle dir and a source tag.
+	// Catalogue path = signed, runs the standard signature gate.
+	// Local path = sideload, requires --local AND must satisfy the
+	// sideload allow-list before the supervisor will load it.
+	bundleDir, source, err := resolveInstallTarget(target)
 	if err != nil {
 		fatalHint("invalid_argument",
 			"the argument must be either a catalogue ID (`pilotctl appstore catalogue` to list) or a path to a bundle dir containing manifest.json",
 			"%v", err)
 	}
+	if source == installSourceLocal && !allowLocal {
+		fatalHint("invalid_argument",
+			"local sideloads carry no catalogue signature; pass --local to confirm you trust this bundle's source. The supervisor will clamp the manifest to a small allow-list (fs.read/fs.write under $APP, audit.log). No net.dial, key.sign, ipc.call to other apps, or daemon hooks.",
+			"refusing to install local path %q without --local", target)
+	}
 
 	// 1. Validate the bundle — same shape as verify. Reusing the
 	//    surface manifest.Parse + sha256File makes the trust check
@@ -1026,6 +1048,18 @@ func cmdAppStoreInstall(args []string) {
 			"refusing to install a manifest the supervisor will reject; fix the listed issues and re-run",
 			"manifest validation: %s", validationErrSummary(errs))
 	}
+	if source == installSourceLocal {
+		// Same allow-list the supervisor enforces at scan time. We
+		// check it here too so users find out at install time, not at
+		// the next supervisor poll, what they need to strip from the
+		// manifest. Failing closed: no marker is planted unless the
+		// manifest passes.
+		if err := manifest.EnforceSideloadPolicy(m); err != nil {
+			fatalHint("invalid_argument",
+				"sideloaded apps may only declare audit.log, fs.read $APP/*, fs.write $APP/*. Remove the offending grant or use the catalogue install path for a reviewed app.",
+				"%v", err)
+		}
+	}
 	srcBin := filepath.Join(bundleDir, m.Binary.Path)
 	if _, err := os.Stat(srcBin); err != nil {
 		fatalHint("invalid_argument",
@@ -1092,6 +1126,21 @@ func cmdAppStoreInstall(args []string) {
 			"staged binary sha256 mismatch: manifest=%s staged=%s", m.Binary.SHA256, got)
 	}
 
+	if source == installSourceLocal {
+		// Plant the sentinel before the atomic rename so the moment
+		// the dir appears under InstallRoot it's already tagged
+		// sideloaded — there's no window where the supervisor could
+		// scan a sideloaded dir and treat it as catalogue-trusted.
+		// 0o400: read-only to the owning user; the file's mere
+		// existence is the signal, no content needed.
+		markerPath := filepath.Join(stagingDir, manifest.SideloadMarkerName)
+		if err := os.WriteFile(markerPath, nil, 0o400); err != nil {
+			_ = os.RemoveAll(stagingDir)
+			fatalHint("io_error", "check install root permissions",
+				"write sideload marker: %v", err)
+		}
+	}
+
 	// 4. Atomic swap. Rename of directories is atomic on Linux/macOS
 	//    as long as the destination does not already exist; with
 	//    --force we have to step aside the previous install first.
@@ -1162,6 +1211,11 @@ func cmdAppStoreInstall(args []string) {
 	}
 	fmt.Printf("installed %s v%s (manifest v%d) → %s\n",
 		report.AppID, report.AppVersion, report.ManifestVersion, report.InstalledTo)
+	if source == installSourceLocal {
+		fmt.Println("mode: SIDELOADED — manifest-level allow-list applied (audit.log, fs.read/$APP, fs.write/$APP).")
+		fmt.Println("      this is NOT an OS sandbox: a malicious binary that ignores its manifest can still")
+		fmt.Println("      misbehave at the syscall level. Only install paths from sources you trust.")
+	}
 	fmt.Println("note: the daemon rescans the install root periodically —")
 	fmt.Println("      this app will be picked up within ~30s (no daemon restart needed)")
 }

cmd/pilotctl/appstore_catalogue.go

@@ -133,11 +133,30 @@ func cmdAppStoreCatalogue(_ []string) {
 	}
 }
 
+// installSource tags how a bundle reached the install command.
+// The install path uses this to switch between catalogue-signed and
+// sideloaded trust regimes.
+type installSource int
+
+const (
+	// installSourceCatalogue: target matched a catalogue entry; the
+	// bundle was downloaded and sha-verified against the catalogue
+	// pin. Goes through the standard signed-manifest install.
+	installSourceCatalogue installSource = iota
+	// installSourceLocal: target was a local directory path. No
+	// publisher signature is expected; the install command applies
+	// the sideload allow-list policy and plants `.sideloaded` so the
+	// supervisor uses the sideloaded trust regime at runtime.
+	installSourceLocal
+)
+
 // resolveInstallTarget turns the user's `target` arg into a local
-// bundle directory the existing install code can consume. If `target`
-// matches a catalogue ID, the catalogue entry is fetched, verified,
-// and unpacked. Otherwise `target` is treated as a local path.
-func resolveInstallTarget(target string) (string, error) {
+// bundle directory the existing install code can consume, plus a
+// source tag indicating which trust regime the bundle came from. If
+// `target` matches a catalogue ID, the catalogue entry is fetched,
+// verified, and unpacked. Otherwise `target` is treated as a local
+// path and the caller is expected to apply sideload policy.
+func resolveInstallTarget(target string) (string, installSource, error) {
 	c, err := loadCatalogue()
 	if err != nil {
 		// Catalogue-lookup failure doesn't preclude a local-dir install
@@ -148,15 +167,16 @@ func resolveInstallTarget(target string) (string, error) {
 	} else {
 		for _, e := range c.Apps {
 			if target == e.ID {
-				return fetchAndUnpackBundle(e)
+				dir, err := fetchAndUnpackBundle(e)
+				return dir, installSourceCatalogue, err
 			}
 		}
 	}
 	info, err := os.Stat(target)
 	if err == nil && info.IsDir() {
-		return target, nil
+		return target, installSourceLocal, nil
 	}
-	return "", fmt.Errorf("not a catalogue ID or a bundle dir: %q (try `pilotctl appstore catalogue` to list installable apps)", target)
+	return "", installSourceLocal, fmt.Errorf("not a catalogue ID or a bundle dir: %q (try `pilotctl appstore catalogue` to list installable apps)", target)
 }
 
 // fetchAndUnpackBundle downloads the catalogue entry's tarball,

cmd/pilotctl/zz_appstore_cmds_test.go

@@ -9,6 +9,8 @@ import (
 	"strings"
 	"testing"
 	"time"
+
+	"github.com/pilot-protocol/app-store/pkg/manifest"
 )
 
 // minimalManifestJSON returns a manifest that passes JSON Parse but
@@ -54,7 +56,7 @@ func validManifestJSON(id, binSHA string) []byte {
 		"grants": []any{
 			map[string]any{
 				"cap":    "fs.read",
-				"target": "/tmp/data",
+				"target": "$APP/data",
 			},
 		},
 		"protection": "shareable",
@@ -511,22 +513,28 @@ func TestCmdAppStoreInstallHappyPath(t *testing.T) {
 	prev := jsonOutput
 	defer func() { jsonOutput = prev }()
 	jsonOutput = true
-	out := captureStdout(t, func() { cmdAppStoreInstall([]string{bundleDir}) })
+	// Local-path installs now require --local; without it the install
+	// command refuses (see TestCmdAppStoreInstallRefusesLocalPathWithoutFlag).
+	out := captureStdout(t, func() { cmdAppStoreInstall([]string{bundleDir, "--local"}) })
 	var rpt installReport
 	if err := json.Unmarshal([]byte(out), &rpt); err != nil {
 		t.Fatalf("parse: %v\n%s", err, out)
 	}
 	if rpt.AppID != "io.test.install" {
 		t.Errorf("AppID = %q", rpt.AppID)
 	}
-	// App should live under root/<id>/
+	// App should live under root/<id>/ and carry the .sideloaded marker
+	// — local-path installs are sideloads by definition.
 	installedPath := filepath.Join(root, "io.test.install")
 	if _, err := os.Stat(filepath.Join(installedPath, "manifest.json")); err != nil {
 		t.Errorf("manifest not placed: %v", err)
 	}
 	if _, err := os.Stat(filepath.Join(installedPath, "bin", "app")); err != nil {
 		t.Errorf("binary not placed: %v", err)
 	}
+	if _, err := os.Stat(filepath.Join(installedPath, manifest.SideloadMarkerName)); err != nil {
+		t.Errorf("sideload marker not planted: %v", err)
+	}
 }
 
 func TestCmdAppStoreInstallRefusesDuplicate(t *testing.T) {
@@ -537,10 +545,10 @@ func TestCmdAppStoreInstallRefusesDuplicate(t *testing.T) {
 	defer func() { jsonOutput = prev }()
 	jsonOutput = true
 	// First install OK.
-	_ = captureStdout(t, func() { cmdAppStoreInstall([]string{bundleDir}) })
+	_ = captureStdout(t, func() { cmdAppStoreInstall([]string{bundleDir, "--local"}) })
 	// Second install would fatalCode — can't catch os.Exit inline. But
 	// install with --force should succeed.
-	out := captureStdout(t, func() { cmdAppStoreInstall([]string{bundleDir, "--force"}) })
+	out := captureStdout(t, func() { cmdAppStoreInstall([]string{bundleDir, "--force", "--local"}) })
 	var rpt installReport
 	if err := json.Unmarshal([]byte(out), &rpt); err != nil {
 		t.Fatalf("force install parse: %v\n%s", err, out)
@@ -699,8 +707,8 @@ func TestCmdAppStoreInstallTextMode(t *testing.T) {
 	prev := jsonOutput
 	defer func() { jsonOutput = prev }()
 	jsonOutput = false
-	out := captureStdout(t, func() { cmdAppStoreInstall([]string{bundleDir}) })
-	for _, frag := range []string{"installed", "io.test.install.text", "daemon rescans"} {
+	out := captureStdout(t, func() { cmdAppStoreInstall([]string{bundleDir, "--local"}) })
+	for _, frag := range []string{"installed", "io.test.install.text", "daemon rescans", "SIDELOADED"} {
 		if !strings.Contains(out, frag) {
 			t.Errorf("missing %q in: %s", frag, out)
 		}

go.mod

@@ -4,7 +4,7 @@ go 1.25.10
 
 require (
 	github.com/coder/websocket v1.8.14
-	github.com/pilot-protocol/app-store v0.2.0
+	github.com/pilot-protocol/app-store v1.0.1-beta.1.0.20260609061942-8852c785a264
 	github.com/pilot-protocol/beacon v0.2.5
 	github.com/pilot-protocol/common v0.4.8
 	github.com/pilot-protocol/dataexchange v0.2.0

go.sum

@@ -4,6 +4,10 @@ github.com/expr-lang/expr v1.17.8 h1:W1loDTT+0PQf5YteHSTpju2qfUfNoBt4yw9+wOEU9VM
 github.com/expr-lang/expr v1.17.8/go.mod h1:8/vRC7+7HBzESEqt5kKpYXxrxkr31SaO8r40VO/1IT4=
 github.com/pilot-protocol/app-store v0.2.0 h1:/SbLPa2dKnvhv0ITeaYMZw/2KXXxWDRuYUdM5z9eyKk=
 github.com/pilot-protocol/app-store v0.2.0/go.mod h1:f0umeJxswDG8/CctHpSFMlr5GLtE2GlPKkijIQErZuc=
+github.com/pilot-protocol/app-store v1.0.0-rc1.0.20260609015400-d02db7da3924 h1:pUK7tOsFIqYr2vwcfKT4t81+ntuVDbyffajkJa+2A3U=
+github.com/pilot-protocol/app-store v1.0.0-rc1.0.20260609015400-d02db7da3924/go.mod h1:f0umeJxswDG8/CctHpSFMlr5GLtE2GlPKkijIQErZuc=
+github.com/pilot-protocol/app-store v1.0.1-beta.1.0.20260609061942-8852c785a264 h1:NL9rFdakbVQ0V7xfJbCk8RJZSaQ1AmvdhAJwFIouMsk=
+github.com/pilot-protocol/app-store v1.0.1-beta.1.0.20260609061942-8852c785a264/go.mod h1:zoCxHYoNdj0V44OkG3Yzcye0jnwZDVUcJgAvR5Z1kwc=
 github.com/pilot-protocol/beacon v0.2.5 h1:5+pkSPoA35r+u4Hfrph/ZfOltOyiy8lh1sCfK5XqXKs=
 github.com/pilot-protocol/beacon v0.2.5/go.mod h1:I/UhEv097g1z/qtAVDZbEhf3R5tzM0Dp71vGHah52A4=
 github.com/pilot-protocol/common v0.4.8 h1:eS2Bc+XcZWJ/qhwwOZbXwIWhtNdOijuoEp716kQE+/c=

scripts/gen-cli-reference.sh

@@ -25,7 +25,12 @@ TMP="$(mktemp)"
     echo '> Source: [`cmd/pilotctl/main.go`](../cmd/pilotctl/main.go).'
     echo ''
     echo '```text'
-    ./pilotctl --help
+    # pilotctl --help writes to stderr (so the same banner appears
+    # for `pilotctl <bad-flag>`) and exits non-zero by Go's flag
+    # convention. Capture stderr → stdout so it lands in the doc,
+    # and swallow the exit code so `set -euo pipefail` doesn't
+    # abort before the diff step runs.
+    ./pilotctl --help 2>&1 || true
     echo '```'
 } > "$TMP"