Skip to content

Commit e2f4f70

Browse files
drewrdrewr
authored
Merge branch 'main' into fix/nix-workflow-pr
2 parents 73892bb + 660bb80 commit e2f4f70

46 files changed

Lines changed: 6189 additions & 11 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎docs/developer/plugins.md‎

Lines changed: 277 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,277 @@
1+
---
2+
title: "Plugin System Design"
3+
sidebar:
4+
order: 6
5+
---
6+
7+
# datumctl Plugin System
8+
9+
This document covers the architectural design for the datumctl plugin system.
10+
It is intended as a reference for contributors building the plugin infrastructure
11+
and for teams authoring first-party plugins.
12+
13+
---
14+
15+
## Goals
16+
17+
- Let domain teams (compute, networking, billing, audit) ship CLI extensions
18+
independently without modifying core `datumctl`.
19+
- Establish a stable contract that survives datumctl version upgrades.
20+
- Keep the security blast radius of a third-party plugin small.
21+
- Give plugin authors a clear, low-friction path to ship.
22+
23+
---
24+
25+
## How plugins work
26+
27+
A plugin is any executable named `datumctl-<name>` that is either:
28+
29+
1. **Managed** — installed via `datumctl plugin install` into
30+
`~/.datumctl/plugins/` and recorded in `plugins.json`.
31+
2. **Unmanaged** — placed on the user's `PATH` by any other means.
32+
33+
When a user runs `datumctl <name>`, datumctl resolves the command using this
34+
precedence order:
35+
36+
1. **Built-in commands** — always win. A plugin named `datumctl-get` or
37+
`datumctl-login` will never shadow a built-in. `datumctl plugin install`
38+
rejects any plugin whose name collides with a built-in at install time.
39+
2. **Managed plugins** — binaries in `~/.datumctl/plugins/`.
40+
3. **Unmanaged plugins** — binaries named `datumctl-<name>` found on `PATH`.
41+
42+
If a matching plugin is found (steps 2–3), datumctl execs it, passing through
43+
all remaining arguments.
44+
45+
Unmanaged plugins trigger a one-time warning:
46+
47+
```
48+
warning: 'datumctl-compute' is not a managed plugin and has not been verified.
49+
```
50+
51+
---
52+
53+
## Directory layout
54+
55+
```
56+
~/.datumctl/
57+
config # CLI configuration
58+
credentials.json # stored credentials
59+
plugins/ # managed plugin binaries
60+
plugins/plugins.json # install record (see below)
61+
plugins/plugin-index.json # cached plugin index
62+
```
63+
64+
datumctl searches `plugins/` before `PATH` so managed and unmanaged plugins
65+
are always distinguishable.
66+
67+
---
68+
69+
## Installation
70+
71+
### From the curated index
72+
73+
The curated plugin index lives at
74+
[datum-cloud/datumctl-plugins](https://github.com/datum-cloud/datumctl-plugins).
75+
Plugins listed there can be installed by name:
76+
77+
```sh
78+
datumctl plugin install compute
79+
datumctl plugin search
80+
datumctl plugin list
81+
datumctl plugin upgrade compute
82+
datumctl plugin remove compute
83+
```
84+
85+
The index is cached locally at `~/.datumctl/plugins/plugin-index.json`
86+
and refreshed automatically when stale (default TTL: 1 hour). Override the
87+
index URL with `DATUMCTL_PLUGIN_INDEX_URL` for testing.
88+
89+
### From a GitHub Release
90+
91+
Any plugin can be installed directly from a GitHub Release without being listed
92+
in the curated index:
93+
94+
```sh
95+
datumctl plugin install datum-cloud/datumctl-compute # latest release
96+
datumctl plugin install datum-cloud/datumctl-compute@v1.2.0 # specific version
97+
```
98+
99+
This path requires a `checksums.txt` file alongside the release archives in
100+
goreleaser's default two-column format.
101+
102+
### Restoring all plugins
103+
104+
Running `datumctl plugin install` with no arguments restores all plugins
105+
recorded in `plugins.json` — useful for reproducing a plugin set on a new
106+
machine.
107+
108+
---
109+
110+
## plugins.json schema
111+
112+
`plugins.json` records every managed install. It is the source of truth for
113+
`plugin list`, `plugin upgrade`, and `plugin remove`.
114+
115+
```json
116+
{
117+
"plugins": {
118+
"compute": {
119+
"source": "compute",
120+
"version": "v0.8.0",
121+
"sha256": "abc123...",
122+
"installed_at": "2026-05-26T00:00:00Z",
123+
"manifest": {
124+
"name": "compute",
125+
"version": "v0.8.0",
126+
"description": "Deploy and manage containerized workloads on Datum Cloud",
127+
"api_version": 1
128+
}
129+
}
130+
}
131+
}
132+
```
133+
134+
`source` is either the short index name (e.g. `compute`) or a
135+
`github.com/owner/repo` path for direct GitHub installs.
136+
137+
---
138+
139+
## Plugin manifest
140+
141+
Every plugin binary must respond to `--plugin-manifest` with a JSON document
142+
on stdout:
143+
144+
```json
145+
{
146+
"name": "compute",
147+
"version": "v0.8.0",
148+
"description": "Deploy and manage containerized workloads on Datum Cloud",
149+
"min_datumctl_version": "v0.10.0",
150+
"api_version": 1,
151+
"min_api_version": 1
152+
}
153+
```
154+
155+
datumctl reads this manifest at install time to validate compatibility. If a
156+
plugin does not respond to `--plugin-manifest`, datumctl treats it as
157+
unversioned and skips compatibility checks.
158+
159+
---
160+
161+
## Context passthrough
162+
163+
datumctl sets the following environment variables before execing a plugin:
164+
165+
| Variable | Value |
166+
|----------------------------|----------------------------------------------------|
167+
| `DATUM_ORG` | Current organization slug |
168+
| `DATUM_PROJECT` | Current project slug (may be empty) |
169+
| `DATUM_API_HOST` | API base URL (e.g. `api.datum.net`) |
170+
| `DATUM_PLUGIN_API_VERSION` | Integer API version (currently `1`) |
171+
| `DATUM_CREDENTIALS_HELPER` | Absolute path to the datumctl binary |
172+
| `DATUM_SESSION` | Active session name (may be empty) |
173+
174+
**Tokens are not passed as environment variables.** Plugins fetch a token on
175+
demand via the credentials helper:
176+
177+
```sh
178+
$DATUM_CREDENTIALS_HELPER auth get-token --session $DATUM_SESSION
179+
```
180+
181+
Omit `--session` when `DATUM_SESSION` is empty. The Go SDK's `plugin.Token()`
182+
handles this automatically.
183+
184+
### Why not `DATUM_TOKEN`?
185+
186+
Passing a raw token in an environment variable freezes the auth mechanism —
187+
every plugin that reads `DATUM_TOKEN` directly must be updated if tokens become
188+
shorter-lived, audience-scoped, or replaced by a different credential type.
189+
The credentials helper insulates plugins from these changes entirely.
190+
191+
---
192+
193+
## `DATUM_PLUGIN_API_VERSION`
194+
195+
This integer increments only when the plugin contract (env var names, manifest
196+
schema, credentials helper interface) changes in a breaking way. Plugin authors
197+
check this value if they need to handle multiple datumctl generations. It is
198+
independent of datumctl's own semver version.
199+
200+
Current version: **1**
201+
202+
---
203+
204+
## Go SDK
205+
206+
First-party and community plugins written in Go should use:
207+
208+
```
209+
go.datum.net/datumctl/plugin
210+
```
211+
212+
The SDK provides:
213+
214+
- `plugin.Context()` — reads all `DATUM_*` env vars into a typed struct.
215+
- `plugin.Token()` — calls the credentials helper and returns a token string.
216+
- `plugin.NewRootCmd(name, short)` — returns a pre-configured `*cobra.Command`
217+
with `--org`, `--project`, and `--output` flags wired to the injected context.
218+
- `plugin.ServeManifest(m)` — handles `--plugin-manifest` and exits before
219+
Cobra runs.
220+
221+
See `examples/plugin-dns/` for a working reference implementation.
222+
223+
Plugins written in other languages can implement the same contract manually —
224+
the protocol is just environment variables and a subprocess call.
225+
226+
---
227+
228+
## Security model
229+
230+
| Plugin type | Token access | Verification |
231+
|-------------|--------------|--------------|
232+
| Managed (index) | On demand via helper | SHA256 verified against index manifest |
233+
| Managed (GitHub) | On demand via helper | SHA256 verified against `checksums.txt` |
234+
| Unmanaged | On demand via helper | None — user warning shown |
235+
236+
Because tokens are fetched on demand rather than injected at startup, a plugin
237+
process that exfiltrates its environment variables does not automatically
238+
capture a usable credential. A determined attacker can still call the helper,
239+
but this raises the bar meaningfully over raw env var injection.
240+
241+
Future: audience-scoped tokens (e.g., `datumctl auth get-token
242+
--audience=dns.datum.net`) will let datumctl issue tokens that are only valid
243+
for a specific plugin's API surface.
244+
245+
---
246+
247+
## Compatibility and versioning
248+
249+
Plugin authors declare a minimum datumctl version in their manifest. datumctl
250+
validates this at install time and warns (but does not block) at invocation if
251+
the running version is below the declared minimum.
252+
253+
datumctl guarantees that `DATUM_PLUGIN_API_VERSION=1` env vars and the
254+
credentials helper interface are stable for the lifetime of API version 1.
255+
Breaking changes increment the version and are announced in release notes with
256+
a migration guide.
257+
258+
---
259+
260+
## V1 scope
261+
262+
| Component | Status |
263+
|-----------|--------|
264+
| PATH shim (`datumctl <name>`) | V1 |
265+
| Managed install dir + `plugins.json` | V1 |
266+
| `datumctl plugin install/list/upgrade/remove` | V1 |
267+
| Curated plugin index (`datum-cloud/datumctl-plugins`) | V1 |
268+
| Direct GitHub Release install (`owner/repo[@version]`) | V1 |
269+
| ENV context passthrough | V1 |
270+
| Credentials helper (`datumctl auth get-token`) | V1 |
271+
| Plugin manifest (`--plugin-manifest`) | V1 |
272+
| Go SDK (`go.datum.net/datumctl/plugin`) | V1 |
273+
| Reference first-party plugin (`compute`) | V1 |
274+
| TUI panel extension points | V2 |
275+
| MCP tool registration | V2 |
276+
| `datumctl plugin new` scaffolding | V2 |
277+
| Audience-scoped tokens | V2 |

‎examples/plugin-dns/.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
plugin-dns

0 commit comments

Comments
 (0)