You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
feat: migrate config system from dotenv to koanf/YAML (#102)
* feat: migrate config system from dotenv to koanf/YAML
Replace godotenv-based .env config loading with koanf and YAML config
files across the entire stack:
Server (hypeman-api):
- Config struct now uses koanf tags for YAML unmarshaling
- Loads config.yaml from platform-specific paths with env var overrides
- CONFIG_PATH env var for explicit config file location
Token tool (hypeman-token):
- Reads jwt_secret directly from config.yaml (no more wrapper scripts)
- Added -duration flag for configurable token expiry
Install script:
- Generates config.yaml instead of dotenv-style config
- Generates ~/.config/hypeman/cli.yaml with pre-authenticated token
- Removed all wrapper scripts, replaced with symlinks on Linux
- Updated launchd/systemd service definitions
Also adds config.example.yaml, config.darwin.example.yaml, and
cli.example.yaml as reference templates.
* fix: address bot review feedback on config loading
- Pass JWT_SECRET explicitly to hypeman-token in install.sh (fixes
Linux installs where config.yaml is root-only)
- Return error from config.Load() when explicit CONFIG_PATH fails
instead of silently falling back to defaults
- Deduplicate config paths: gen-jwt now imports and uses
config.GetDefaultConfigPaths() instead of maintaining its own copy
* fix: skip empty env vars to preserve config/default values
Use env.ProviderWithValue instead of env.Provider so that empty
environment variables (e.g. PORT="") don't override valid defaults
or YAML config values. This preserves the old getEnv() behavior.
* fix: gen-jwt respects CONFIG_PATH, install.sh uses sudo for config read
- hypeman-token now checks CONFIG_PATH env var before default paths,
matching hypeman-api behavior for custom config locations
- install.sh uses $SUDO when reading jwt_secret from existing config
file on Linux reinstalls (file is 640 root:root)
* fix: robust YAML parsing in install.sh and restore make gen-jwt dev workflow
- Make jwt_secret and port grep/sed pipelines handle leading whitespace,
single/double quotes, trailing whitespace, and multiple matches
- Update make gen-jwt to auto-detect local config.yaml via CONFIG_PATH,
restoring the dev workflow that previously relied on godotenv/.env
Addresses bugbot review comments on #102.
* refactor: use __ delimiter for nested env vars, rename example files
- Replace explicit envKeyMap with koanf's __ delimiter for auto-mapping
env vars to nested config paths (e.g. CADDY__LISTEN_ADDRESS -> caddy.listen_address)
- Rename config.darwin.example.yaml -> config.example.darwin.yaml for
consistent naming
- Remove .env.example and .env.darwin.example references
- Update DEVELOPMENT.md to document __ convention and YAML-first config
- Update README.md configuration table
BREAKING: Old flat env var names (CADDY_LISTEN_ADDRESS, BRIDGE_NAME, etc.)
no longer work. Use double-underscore for nested keys (CADDY__LISTEN_ADDRESS,
NETWORK__BRIDGE_NAME) or configure via config.yaml instead.
* docs: simplify config docs to reference YAML keys, not env vars
README and DEVELOPMENT.md now document configuration using YAML key
names (dot notation for nested keys). Env var override convention
mentioned once as a footnote rather than being the primary reference.
* chore: default build.builder_image to "none" (built on first run)
* fix: BSD sed compat for docker_socket injection in install.sh
The macOS BSD sed `a\` (append) command was inserting the docker_socket
line on the same line as builder_image, producing invalid YAML. Fix by:
1. Including docker_socket in example config templates so the simpler
sed s| replacement path is used instead of sed a\.
2. Fixing the fallback sed a\ to use BSD-compatible s| with literal
newline.
Also updates builder_image default to "none" in example files.
* fix: export CLI env vars in e2e test for released CLI compat
The released CLI (v0.11.0) doesn't support cli.yaml yet, so the e2e
test needs to export HYPEMAN_BASE_URL and HYPEMAN_API_KEY env vars.
Once the CLI release with koanf/yaml support ships, cli.yaml will
handle this and the env vars become redundant.
* Revert "fix: export CLI env vars in e2e test for released CLI compat"
This reverts commit a6e82a5.
* feat: support CLI_BRANCH to build CLI from source in install.sh
Add CLI_BRANCH env var to install.sh that clones and builds the CLI
from the specified branch of kernel/hypeman-cli instead of downloading
a release binary. Useful for testing unreleased CLI features.
The e2e test now passes CLI_BRANCH through to install.sh. Temporarily
set to koanf-yaml-config in CI so the e2e test uses the CLI with
cli.yaml config file support.
* fix: update GPU test files to use nested config struct
Missed gpu_module_test.go, gpu_e2e_test.go, and gpu_inference_test.go
during the config nesting migration. Uses config.NetworkConfig for
BridgeName, SubnetCIDR, and DNSServer fields.
* fix: remove fallback config generation from install.sh
The inline heredoc configs duplicated defaults defined in the example
YAML files. Treat a failed config template download as a hard error
instead of silently generating a potentially stale config.
* cleanup: remove redundant OS check inside darwin block
* fix: hard error if jwt_secret not found in config template
Without jwt_secret the server can't authenticate API requests, so
silently skipping it leaves the install in a broken state.
* fix: remove accidentally committed gen-jwt binary
* docs: remove token tool row from config table
* refactor: remove dead fallbacks and "none" sentinel for builder_image
- Remove unreachable zero-value fallbacks in ProvideBuildManager that
duplicated defaults already set by defaultConfig().
- Replace the "none" sentinel for builder_image with empty string.
Empty string now canonically means "build from embedded Dockerfile
on first run", which is simpler than a magic string.
* fix: validate build.timeout and build.max_concurrent_source_builds
Reject zero/negative values at startup rather than passing them through
to the build manager where they'd cause immediate timeouts or a
zero-capacity semaphore that blocks all builds.
|`CADDY_LISTEN_ADDRESS`| Address for Caddy ingress listeners |`0.0.0.0`|
114
-
|`CADDY_ADMIN_ADDRESS`| Address for Caddy admin API |`127.0.0.1`|
115
-
|`CADDY_ADMIN_PORT`| Port for Caddy admin API |`2019`|
116
-
|`CADDY_STOP_ON_SHUTDOWN`| Stop Caddy when hypeman shuts down (set to `true` for dev) |`false`|
117
-
|`ACME_EMAIL`| Email for ACME certificate registration (required for TLS ingresses) |_(empty)_|
118
-
|`ACME_DNS_PROVIDER`| DNS provider for ACME challenges: `cloudflare`|_(empty)_|
119
-
|`ACME_CA`| ACME CA URL (empty = Let's Encrypt production) |_(empty)_|
120
-
|`TLS_ALLOWED_DOMAINS`| Comma-separated allowed domains for TLS (e.g., `*.example.com,api.other.com`) |_(empty)_|
121
-
|`DNS_PROPAGATION_TIMEOUT`| Max time to wait for DNS propagation (e.g., `2m`) |_(empty)_|
122
-
|`DNS_RESOLVERS`| Comma-separated DNS resolvers for propagation checking |_(empty)_|
123
-
|`CLOUDFLARE_API_TOKEN`| Cloudflare API token (when using `cloudflare` provider) |_(empty)_|
124
-
|`DOCKER_SOCKET`| Path to Docker socket (for builder image builds) |`/var/run/docker.sock`|
92
+
Hypeman reads configuration from a YAML config file. See `config.example.yaml` (Linux) and `config.example.darwin.yaml` (macOS) for all available settings with comments.
93
+
94
+
The config file is searched in these locations (first match wins):
95
+
- Path specified by `CONFIG_PATH` environment variable
96
+
-`/etc/hypeman/config.yaml` (Linux)
97
+
-`~/.config/hypeman/config.yaml` (all platforms)
98
+
99
+
Common settings:
100
+
101
+
| Key | Description | Default |
102
+
|-----|-------------|---------|
103
+
|`port`| HTTP server port |`8080`|
104
+
|`data_dir`| Data directory for VM images, volumes, etc. |`/var/lib/hypeman`|
105
+
|`jwt_secret`| Secret key for JWT authentication (required) |_(empty)_|
|`limits.max_concurrent_builds`| Max concurrent image builds |`1`|
119
+
|`limits.max_overlay_size`| Max overlay filesystem size |`100GB`|
120
+
|`acme.email`| Email for ACME certificate registration |_(empty)_|
121
+
|`acme.dns_provider`| DNS provider for ACME challenges |_(empty)_|
122
+
|`acme.cloudflare_api_token`| Cloudflare API token |_(empty)_|
123
+
|`build.docker_socket`| Path to Docker socket |`/var/run/docker.sock`|
124
+
125
+
Environment variables can also override any config key using `__` as the nesting separator (e.g. `CADDY__LISTEN_ADDRESS` overrides `caddy.listen_address`).
125
126
126
127
**Important: Subnet Configuration**
127
128
128
129
The default subnet `10.100.0.0/16` is chosen to avoid common conflicts. Hypeman will detect conflicts with existing routes on startup and fail with guidance.
129
130
130
-
If you need a different subnet, set `SUBNET_CIDR` in your environment. The gateway is automatically derived as the first IP in the subnet (e.g., `10.100.0.0/16` → `10.100.0.1`).
131
+
If you need a different subnet, set `network.subnet_cidr` in your config file. The gateway is automatically derived as the first IP in the subnet (e.g., `10.100.0.0/16` → `10.100.0.1`).
131
132
132
133
**Alternative subnets if needed:**
133
134
@@ -136,14 +137,15 @@ If you need a different subnet, set `SUBNET_CIDR` in your environment. The gatew
136
137
137
138
**Example:**
138
139
139
-
```bash
140
-
# In your .env file
141
-
SUBNET_CIDR=172.30.0.0/16
140
+
```yaml
141
+
# In your config.yaml
142
+
network:
143
+
subnet_cidr: 172.30.0.0/16
142
144
```
143
145
144
-
**Finding the uplink interface (`UPLINK_INTERFACE`)**
146
+
**Finding the uplink interface (`network.uplink_interface`)**
145
147
146
-
`UPLINK_INTERFACE` tells Hypeman which host interface to use for routing VM traffic to the outside world (for iptables MASQUERADE rules). On many hosts this is `eth0`, but laptops and more complex setups often use Wi‑Fi or other names.
148
+
`network.uplink_interface`tells Hypeman which host interface to use for routing VM traffic to the outside world (for iptables MASQUERADE rules). On many hosts this is `eth0`, but laptops and more complex setups often use Wi-Fi or other names.
147
149
148
150
**Quick way to discover it:**
149
151
@@ -158,10 +160,11 @@ Look for the `dev` field in the output, for example:
158
160
1.1.1.1 via 192.168.12.1 dev wlp2s0 src 192.168.12.98
159
161
```
160
162
161
-
In this case, `wlp2s0` is the uplink interface, so you would set:
163
+
In this case, `wlp2s0` is the uplink interface, so you would set in your config file:
162
164
163
-
```bash
164
-
UPLINK_INTERFACE=wlp2s0
165
+
```yaml
166
+
network:
167
+
uplink_interface: wlp2s0
165
168
```
166
169
167
170
You can also inspect all routes:
@@ -178,15 +181,13 @@ Hypeman uses Caddy with automatic ACME certificates for TLS termination. Certifi
178
181
179
182
To enable TLS ingresses:
180
183
181
-
1. Configure ACME credentials in your `.env`:
184
+
1. Configure ACME credentials in your `config.yaml`:
182
185
183
-
```bash
184
-
# Required for any TLS ingress
185
-
ACME_EMAIL=admin@example.com
186
-
187
-
# For Cloudflare
188
-
ACME_DNS_PROVIDER=cloudflare
189
-
CLOUDFLARE_API_TOKEN=your-api-token
186
+
```yaml
187
+
acme:
188
+
email: admin@example.com
189
+
dns_provider: cloudflare
190
+
cloudflare_api_token: your-api-token
190
191
```
191
192
192
193
2. Create an ingress with TLS enabled:
@@ -205,13 +206,13 @@ curl -X POST http://localhost:8080/v1/ingresses \
205
206
}'
206
207
```
207
208
208
-
Certificates are stored in `$DATA_DIR/caddy/data/` and auto-renewed by Caddy.
209
+
Certificates are stored in `<data_dir>/caddy/data/` and auto-renewed by Caddy.
209
210
210
211
### Setup
211
212
212
213
```bash
213
-
cp .env.example .env
214
-
# Edit .env and set JWT_SECRET and other configuration values
# Edit config.yaml and set jwt_secret and other configuration values
215
216
```
216
217
217
218
### Data directory
@@ -253,15 +254,16 @@ make gen-jwt
253
254
make dev
254
255
```
255
256
256
-
The server will start on port 8080 (configurable via `PORT` environment variable).
257
+
The server will start on port 8080 (configurable via `port` in config.yaml).
257
258
258
259
### Setting Up the Builder Image (for Dockerfile builds)
259
260
260
-
The builder image is required for `hypeman build` to work. When `BUILDER_IMAGE` is unset or empty, the server will automatically build and push the builder image on startup using Docker. This is the easiest way to get started — just ensure Docker is available and run `make dev`. If a build is requested while the builder image is still being prepared, the server returns a clear error asking you to retry shortly.
261
+
The builder image is required for `hypeman build` to work. When `build.builder_image` is unset or empty, the server will automatically build and push the builder image on startup using Docker. This is the easiest way to get started — just ensure Docker is available and run `make dev`. If a build is requested while the builder image is still being prepared, the server returns a clear error asking you to retry shortly.
261
262
262
-
On macOS with Colima, set the Docker socket path:
263
-
```bash
264
-
DOCKER_SOCKET=$HOME/.colima/default/docker.sock
263
+
On macOS with Colima, set the Docker socket path in your config file:
264
+
```yaml
265
+
build:
266
+
docker_socket: ~/.colima/default/docker.sock
265
267
```
266
268
267
269
### Local OpenTelemetry (optional)
@@ -287,9 +289,11 @@ docker run -d --name lgtm \
287
289
# If developing on a remote server, forward the port to your local machine (or YOLO):
288
290
# ssh -L 3001:localhost:3000 your-server (then open http://localhost:3001)
289
291
290
-
# Enable OTel in .env (set ENV to your name to filter your telemetry)
291
-
echo"OTEL_ENABLED=true">> .env
292
-
echo"ENV=yourname">> .env
292
+
# Enable OTel in config.yaml (set env to your name to filter your telemetry)
Copy file name to clipboardExpand all lines: README.md
+37-7Lines changed: 37 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -35,29 +35,59 @@ Install Hypeman (Linux and macOS supported):
35
35
curl -fsSL https://get.hypeman.sh | bash
36
36
```
37
37
38
-
This installs both the Hypeman server and CLI. The installer handles all dependencies and configuration automatically.
38
+
This installs the Hypeman server, CLI, and token tool. The installer:
39
+
- Generates a YAML config file with a random JWT secret
40
+
- Starts the server as a system service (launchd on macOS, systemd on Linux)
41
+
- Creates a CLI config file (`~/.config/hypeman/cli.yaml`) with a pre-authenticated token
39
42
40
-
## CLI Installation
43
+
No environment variables needed -- just run `hypeman` commands immediately after install.
41
44
42
-
To use Hypeman via the CLI on a separate machine:
45
+
## Remote CLI Access
43
46
44
-
**Homebrew:**
47
+
To use the Hypeman CLI from a **different machine** than the server:
48
+
49
+
**Homebrew (macOS):**
45
50
```bash
46
51
brew install kernel/tap/hypeman
47
52
```
48
53
54
+
**Linux:**
55
+
```bash
56
+
curl -fsSL https://get.hypeman.sh/cli | bash
57
+
```
58
+
49
59
**Go:**
50
60
```bash
51
61
go install 'github.com/kernel/hypeman-cli/cmd/hypeman@latest'
52
62
```
53
63
54
-
**Configure CLI access:**
64
+
Then create a CLI config file at `~/.config/hypeman/cli.yaml`:
65
+
66
+
```yaml
67
+
base_url: http://<server-host>:8080
68
+
api_key: "<token>"
69
+
```
70
+
71
+
To generate a token, run `hypeman-token` on the server:
55
72
56
73
```bash
57
-
export HYPEMAN_API_KEY="<token>"
58
-
export HYPEMAN_BASE_URL="http://<host>:8080"
74
+
hypeman-token -user-id "my-user" -duration 8760h
59
75
```
60
76
77
+
Environment variables (`HYPEMAN_BASE_URL`, `HYPEMAN_API_KEY`) and CLI flags (`--base-url`) also work and take precedence over the config file.
78
+
79
+
## Configuration
80
+
81
+
Hypeman is configured via YAML config files.
82
+
83
+
| Component | Config File |
84
+
|-----------|-------------|
85
+
| Server | `/etc/hypeman/config.yaml` (Linux) or `~/.config/hypeman/config.yaml` (macOS) |
86
+
| CLI | `~/.config/hypeman/cli.yaml` |
87
+
88
+
89
+
See [`config.example.yaml`](config.example.yaml) (Linux) and [`config.example.darwin.yaml`](config.example.darwin.yaml) (macOS) for all available server options.
0 commit comments