Skip to content

Commit 4541afd

Browse files
miharpclaude
andcommitted
docs: fix and modernize dev_running_from_source
Rewrite the Docker agent section to the actual working flow and apply related corrections found while testing the guide end-to-end against a from-source server. - Docker agent: connect via `--add-host puppet:host-gateway` (default `--server puppet`), persist a named SSL volume, autosign by default, and document the manual `puppetserver-ca` path and its requirement that the server name resolve on the host. - Quick Start: rename from "Step 0"; replace the no-op `rm -rf ~/.puppetserver` with a recoverable timestamped `mv` backup; run the source agent through Bundler; point to the Step 1 prerequisites. - Prerequisites: add Ruby 3.1+; reword the stale Leiningen version. - Step 3: enable autosign with `puppet config set ... --section server`; update deprecated `jruby-puppet.master-*` settings to `server-*`. - OpenVoxDB section: `[master]` -> `[server]`, routes `master:` -> `server:`, grammar fix. - Structure/conventions: nest the Step 4c gotchas as `###`; "puppet master" -> "Puppet Server"; convert terminal `bash` fences to `console` per CONTRIBUTING. Signed-off-by: Michael Harp <mike@mikeharp.com> Assisted-by: Claude Opus 4.8 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 82a30da commit 4541afd

1 file changed

Lines changed: 95 additions & 46 deletions

File tree

docs/_openvox-server_8x/dev_running_from_source.markdown

Lines changed: 95 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -7,16 +7,19 @@ title: "Running OpenVox Server from Source"
77

88
The following steps will help you get OpenVox Server up and running from source.
99

10-
## Step 0: Quick Start for Developers
10+
## Quick Start for Developers
1111

12-
```bash
12+
This quick start assumes the [prerequisites](#step-1-install-prerequisites) are already installed: Java and Leiningen to run the server, Git to check out the source, and Ruby 3.1 or later to run the
13+
agent. See [Step 1](#step-1-install-prerequisites) for details.
14+
15+
```console
1316
# clone git repository and initialize submodules
1417
$ git clone --recursive https://github.com/OpenVoxProject/openvox-server
1518
$ cd openvox-server
1619

17-
# Remove any old config if you want to make sure you're using the latest
18-
# defaults
19-
$ rm -rf ~/.puppetserver
20+
# Move any old config aside (rename rather than delete, so you can restore
21+
# it) if you want to be sure you're starting from the latest defaults
22+
$ mv ~/.puppetlabs ~/.puppetlabs.bak.$(date +%Y%m%d-%H%M%S)
2023

2124
# Run the `dev-setup` script to initialize all required configuration
2225
$ ./dev-setup
@@ -33,16 +36,16 @@ values that `puppet` uses (for non-root users).
3336

3437
You can find the specific paths in the `dev/puppetserver.conf` file.
3538

36-
In another shell, you can run the agent:
39+
In another shell, you can run the agent from source. Run it through Bundler so the source tree is isolated from any `puppet` or `openvox` gems installed in your Ruby (otherwise the two copies of
40+
Puppet collide):
3741

38-
```bash
39-
# Go to the directory where you checked out openvox-server
40-
$ cd openvox-server
41-
# Set ruby and bin paths
42-
$ export RUBYLIB=./ruby/puppet/lib:./ruby/facter/lib
43-
$ export PATH=./ruby/puppet/bin:./ruby/facter/bin:$PATH
44-
# Run the agent
45-
$ puppet agent -t
42+
```console
43+
# Go to the agent source in your openvox-server checkout
44+
$ cd openvox-server/ruby/puppet
45+
# Install the source tree's dependencies (first time only)
46+
$ bundle install
47+
# Run the source agent against your running server
48+
$ bundle exec puppet agent -t --confdir ~/.puppetlabs/etc/puppet
4649
```
4750

4851
More detailed instructions follow.
@@ -52,12 +55,14 @@ More detailed instructions follow.
5255
Use your system's package tools to ensure that the following prerequisites are installed:
5356

5457
- Java 17 or 21
55-
- [Leiningen 2.9.1 (latest)](http://leiningen.org/)
58+
- [Leiningen 2.9.1 or later](http://leiningen.org/)
5659
- Git (for checking out the source code)
60+
- Ruby 3.1 or later -- only needed to run a Puppet Agent from source on the host (the source agent is run through Bundler, which isolates it from any Puppet gems in your Ruby). You can skip this if you
61+
[run the agent in a Docker container](#running-the-agent-inside-a-docker-container) instead.
5762

5863
## Step 2: Clone Git Repo and Set Up Working Tree
5964

60-
```bash
65+
```console
6166
git clone --recursive https://github.com/OpenVoxProject/openvox-server
6267
cd openvox-server
6368
```
@@ -66,7 +71,7 @@ cd openvox-server
6671

6772
The easiest way to do this is to just run:
6873

69-
```bash
74+
```console
7075
./dev-setup
7176
```
7277

@@ -78,19 +83,27 @@ The default paths should all align with the default values that are used by `pup
7883
If you'd like to customize your environment, here are a few things you can do:
7984

8085
- Before running `./dev-setup`, set an environment variable called `MASTERHOST`. If this variable is found during `dev-setup`, it will configure your `puppet.conf` file to use this value for your certname (both
81-
for Puppet Server and for `puppet`) and for the `server` configuration (so that your agent runs will automatically use this hostname as their puppet master).
86+
for Puppet Server and for `puppet`) and for the `server` configuration (so that your agent runs will automatically use this hostname as their Puppet Server).
87+
- After running `./dev-setup`, you can enable autosigning so that agent certificate requests are signed automatically -- convenient when [running agents in Docker
88+
containers](#running-the-agent-inside-a-docker-container). Configure this before you start the server in Step 4 so the change is picked up without a restart. It is best suited to development
89+
environments rather than production:
90+
91+
```console
92+
puppet config set autosign true --section server
93+
```
94+
8295
- Create a file called `dev/user.clj`. This file will be automatically loaded when you run Puppet Server from the REPL. In it, you can define a function called `get-config`, and use it to override the default
8396
values of various settings from `dev/puppetserver.conf`. For an example of what this file should look like, see `./dev/user.clj.sample`.
8497

8598
You don't need to create a `user.clj` in most cases; settings most likely to warrant it are:
8699

87100
- `jruby-puppet.max-active-instances`: the number of JRuby instances to put into the pool. This can usually be set to 1 for dev purposes, unless you're working on something that involves concurrency.
88101
- `jruby-puppet.splay-instance-flush`: Do not attempt to splay JRuby flushing, set when testing if using multiple JRuby instances and you need to control when they are flushed from the pool
89-
- `jruby-puppet.master-conf-dir`: the puppet master confdir (where `puppet.conf`, `modules`, `manifests`, etc. should be located).
90-
- `jruby-puppet.master-code-dir`: the puppet master codedir
91-
- `jruby-puppet.master-var-dir`: the puppet master vardir
92-
- `jruby-puppet.master-run-dir`: the puppet master rundir
93-
- `jruby-puppet.master-log-dir`: the puppet master logdir
102+
- `jruby-puppet.server-conf-dir`: the OpenVox Server confdir (where `puppet.conf`, `modules`, `manifests`, etc. should be located).
103+
- `jruby-puppet.server-code-dir`: the OpenVox Server codedir
104+
- `jruby-puppet.server-var-dir`: the OpenVox Server vardir
105+
- `jruby-puppet.server-run-dir`: the OpenVox Server rundir
106+
- `jruby-puppet.server-log-dir`: the OpenVox Server logdir
94107

95108
## Step 4a: Run the server from the clojure REPL
96109

@@ -135,13 +148,13 @@ Have a look at `dev-tools.clj` if you're interested in seeing what other utility
135148

136149
If you prefer not to run the server interactively in the REPL, you can launch it as a normal process. To start the OpenVox Server when running from source, simply run the following:
137150

138-
```bash
151+
```console
139152
lein run -c /path/to/puppetserver.conf
140153
```
141154

142155
## Step 4c: Development environment gotchas
143156

144-
## Missing git submodules
157+
### Missing git submodules
145158

146159
If you get an error like the following:
147160

@@ -153,7 +166,7 @@ Execution error (LoadError) at org.jruby.RubyKernel/require
153166

154167
Then you've probably forgotten to fetch the git submodules.
155168

156-
## Failing tests
169+
### Failing tests
157170

158171
If you change the `:webserver :ssl-port` config option from the default value of `8140`, tests will fail with errors like the following:
159172

@@ -181,34 +194,70 @@ Changing the `ssl-port` variable back to `8140` makes the tests run properly.
181194

182195
Use a command like the one below to run an agent against your running OpenVox Server:
183196

184-
```bash
197+
```console
185198
puppet agent --confdir ~/.puppetlabs/etc/puppet \
186199
--debug -t
187200
```
188201

189202
Note that a system installed Puppet Agent is ok for use with source-based OpenVoxDB and OpenVox Server. The `--confdir` above specifies the same confdir that OpenVox Server is using. Because the Puppet Agent and
190203
OpenVox Server instances are both using the same confdir, they're both using the same certificates as well. This alleviates the need to sign certificates as a separate step.
191204

205+
To run the agent from source instead -- for example, to exercise local changes to the agent code -- use the Bundler approach shown in [Quick Start for Developers](#quick-start-for-developers).
206+
192207
## Running the Agent inside a Docker container
193208

194-
You can easily run a Puppet Agent inside a Docker container, either by using the `host` network profile or by accessing the OpenVox Server service using the Docker host IP:
209+
You can run a Puppet Agent inside a Docker container to test against an OpenVox Server you are running from source. Unlike the host agent described above, a containerized agent has its own certificate
210+
identity, so it must connect to the server by the **hostname** the server certificate was issued for. Connecting by the Docker host IP (for example `--server 172.17.0.1`) fails with a TLS hostname
211+
mismatch, because the server certificate is issued for a name rather than an address.
212+
213+
### Prerequisites
214+
215+
Before starting the server (Step 4), configure it as described in [Step 3: Set up Config Files](#step-3-set-up-config-files):
216+
217+
- Run `MASTERHOST=puppet ./dev-setup` so the server certificate covers the `puppet` hostname the agent connects to.
218+
- Enable autosigning so each new agent certificate is signed automatically, rather than signing each one by hand.
219+
220+
If you would rather sign certificates manually, see [Sign the agent certificate](#sign-the-agent-certificate) below.
221+
222+
### Start the agent
195223

196-
```bash
197-
docker run -ti \
198-
--name agent1 \
199-
ghcr.io/openvoxproject/openvoxagent:8 \
200-
agent -t --server 172.17.0.1
224+
The agent image already defaults to the server name `puppet`, so you only need to make that name resolve to your Docker host. Map it with `--add-host`, and persist the agent's SSL directory in a
225+
named volume so it keeps a stable key and certificate across runs:
201226

202-
docker run -ti \
203-
--name agent2 \
204-
--network host \
205-
--add-host puppet:127.0.0.1 \
206-
ghcr.io/openvoxproject/openvoxagent:8
227+
```console
228+
docker run -ti \
229+
--name agent1 \
230+
--add-host puppet:host-gateway \
231+
-v agent1-ssl:/etc/puppetlabs/puppet/ssl \
232+
ghcr.io/openvoxproject/openvoxagent:8 \
233+
agent -t --certname agent1
207234
```
208235

209-
To start another Puppet Agent run in a previous container you can use the `docker start` command:
236+
- `--add-host puppet:host-gateway` resolves the `puppet` hostname inside the container to your Docker host, where the server you are running from source listens on port `8140`.
237+
- `-v agent1-ssl:/etc/puppetlabs/puppet/ssl` persists the agent's keys and certificates in a named volume. Without it, each `docker run` generates a new private key, causing certificate/key
238+
mismatches on subsequent runs.
239+
240+
On Linux you can instead share the host's network stack with `--network host --add-host puppet:127.0.0.1`, which points the `puppet` hostname at the loopback address the server is bound to.
241+
242+
### Sign the agent certificate
243+
244+
With autosigning enabled in the [Prerequisites](#prerequisites), each agent certificate is signed automatically and you can skip this step.
245+
246+
If you turn autosigning off, the agent's first run submits a certificate signing request and then exits without compiling a catalog (`Certificate for agent1 has not been signed yet`). Sign the
247+
pending request on the server, then run the agent again. From source, use the `puppetserver-ca` CLI and point it at your development config:
248+
249+
```console
250+
puppetserver-ca sign --certname agent1 --config ~/.puppetlabs/etc/puppet/puppet.conf
251+
```
252+
253+
The CLI signs by connecting to the running server under the name in that config (`puppet`), so that name must also resolve on the host running the command -- for example, with a `127.0.0.1 puppet`
254+
entry in `/etc/hosts`. Because the SSL directory is persisted in the named volume, the agent reuses its signed certificate on every subsequent run.
255+
256+
### Re-running a container
257+
258+
To start a previously created agent container again, use the `docker start` command:
210259

211-
```bash
260+
```console
212261
docker start -a agent1
213262
```
214263

@@ -223,7 +272,7 @@ arguments for the `:jvm-opts` `defproject` key within the `project.clj` file, it
223272

224273
1. An environment variable named `PUPPETSERVER_HEAP_SIZE`. For example, to use a heap size of 6 GiB for a `lein test` run, you could run the following:
225274

226-
```bash
275+
```console
227276
PUPPETSERVER_HEAP_SIZE=6G lein test
228277
```
229278

@@ -260,7 +309,7 @@ From here, the instructions are similar to installing OpenVoxDB manually via pac
260309
Update `~/.puppetlabs/etc/puppet/puppet.conf` to include:
261310

262311
```ini
263-
[master]
312+
[server]
264313
storeconfigs = true
265314
storeconfigs_backend = puppetdb
266315
reports = store,puppetdb
@@ -277,15 +326,15 @@ Then create a new routes file at `~/.puppetlabs/etc/puppet/routes.yaml` that con
277326

278327
```yaml
279328
---
280-
master:
329+
server:
281330
facts:
282331
terminus: puppetdb
283332
cache: yaml
284333
```
285334
286-
Assuming you have a OpenVoxDB instance up and running, start your OpenVox Server instance with the new puppetserver.conf file that you changed:
335+
Assuming you have an OpenVoxDB instance up and running, start your OpenVox Server instance with the new puppetserver.conf file that you changed:
287336
288-
```bash
337+
```console
289338
lein run -c ~/<YOUR CONFIG DIR>/puppetserver.conf
290339
```
291340

@@ -304,7 +353,7 @@ ssl-ca-cert = <home dir>/.puppetlabs/etc/puppet/ssl/certs/ca.pem
304353

305354
After the SSL config is in place, start (or restart) OpenVoxDB:
306355

307-
```bash
356+
```console
308357
lein run services -c <path to PDB config>/conf.d
309358
```
310359

0 commit comments

Comments
 (0)