Skip to content

Commit 3f40359

Browse files
authored
Merge pull request #166 from miharp/feat/modernize-dev-running-from-source
Modernize dev_running_from_source page
2 parents 17645d8 + 101d67e commit 3f40359

1 file changed

Lines changed: 147 additions & 106 deletions

File tree

Lines changed: 147 additions & 106 deletions
Original file line numberDiff line numberDiff line change
@@ -1,31 +1,32 @@
11
---
22
layout: default
3-
title: "Puppet Server: Running From Source"
4-
canonical: "/puppetserver/latest/dev_running_from_source.html"
3+
title: "Running OpenVox Server from Source"
54
---
65

7-
## So you'd like to run Puppet Server from source?
6+
## So you'd like to run OpenVox Server from source?
87

9-
The following steps will help you get Puppet Server up and running from source.
8+
The following steps will help you get OpenVox Server up and running from source.
109

1110
## Step 0: Quick Start for Developers
1211

13-
# clone git repository and initialize submodules
14-
$ git clone --recursive git://github.com/puppetlabs/puppetserver
15-
$ cd puppetserver
12+
```bash
13+
# clone git repository and initialize submodules
14+
$ git clone --recursive https://github.com/OpenVoxProject/openvox-server
15+
$ cd openvox-server
1616

17-
# Remove any old config if you want to make sure you're using the latest
18-
# defaults
19-
$ rm -rf ~/.puppetserver
17+
# Remove any old config if you want to make sure you're using the latest
18+
# defaults
19+
$ rm -rf ~/.puppetserver
2020

21-
# Run the `dev-setup` script to initialize all required configuration
22-
$ ./dev-setup
21+
# Run the `dev-setup` script to initialize all required configuration
22+
$ ./dev-setup
2323

24-
# Launch the clojure REPL
25-
$ lein repl
26-
# Run Puppet Server
27-
dev-tools=> (go)
28-
dev-tools=> (help)
24+
# Launch the clojure REPL
25+
$ lein repl
26+
# Run Puppet Server
27+
dev-tools=> (go)
28+
dev-tools=> (help)
29+
```
2930

3031
You should now have a running server. All relevant paths (`$confdir`, `$codedir`, etc.) are configured by default to point to directories underneath `~/.puppetlabs`. These should all align with the default
3132
values that `puppet` uses (for non-root users).
@@ -34,34 +35,40 @@ You can find the specific paths in the `dev/puppetserver.conf` file.
3435

3536
In another shell, you can run the agent:
3637

37-
# Go to the directory where you checked out puppetserver
38-
$ cd puppetserver
39-
# Set ruby and bin paths
40-
$ export RUBYLIB=./ruby/puppet/lib:./ruby/facter/lib
41-
$ export PATH=./ruby/puppet/bin:./ruby/facter/bin:$PATH
42-
# Run the agent
43-
$ puppet agent -t
38+
```bash
39+
# Go to the directory where you checked out openvox-server
40+
$ cd openvox-server
41+
# Set ruby and bin paths
42+
$ export RUBYLIB=./ruby/puppet/lib:./ruby/facter/lib
43+
$ export PATH=./ruby/puppet/bin:./ruby/facter/bin:$PATH
44+
# Run the agent
45+
$ puppet agent -t
46+
```
4447

4548
More detailed instructions follow.
4649

4750
## Step 1: Install Prerequisites
4851

4952
Use your system's package tools to ensure that the following prerequisites are installed:
5053

51-
- JDK 1.8 or Java 11
54+
- Java 17 or 21
5255
- [Leiningen 2.9.1 (latest)](http://leiningen.org/)
5356
- Git (for checking out the source code)
5457

5558
## Step 2: Clone Git Repo and Set Up Working Tree
5659

57-
git clone --recursive git://github.com/puppetlabs/puppetserver
58-
cd puppetserver
60+
```bash
61+
git clone --recursive https://github.com/OpenVoxProject/openvox-server
62+
cd openvox-server
63+
```
5964

6065
## Step 3: Set up Config Files
6166

6267
The easiest way to do this is to just run:
6368

64-
./dev-setup
69+
```bash
70+
./dev-setup
71+
```
6572

6673
This will set up all of the necessary configuration files and directories inside of your `~/.puppetlabs` directory. If you are interested in seeing what all of the default file paths are, you can find them in
6774
`./dev/puppetserver.conf`.
@@ -75,7 +82,7 @@ If you'd like to customize your environment, here are a few things you can do:
7582
- Create a file called `dev/user.clj`. This file will be automatically loaded when you run Puppet Server from the REPL. In it, you can define a function called `get-config`, and use it to override the default
7683
values of various settings from `dev/puppetserver.conf`. For an example of what this file should look like, see `./dev/user.clj.sample`.
7784

78-
You don't need to create a `user.clj` in most cases; the settings that I change the most frequently that would warrant the creation of this file, though, are:
85+
You don't need to create a `user.clj` in most cases; settings most likely to warrant it are:
7986

8087
- `jruby-puppet.max-active-instances`: the number of JRuby instances to put into the pool. This can usually be set to 1 for dev purposes, unless you're working on something that involves concurrency.
8188
- `jruby-puppet.splay-instance-flush`: Do not attempt to splay JRuby flushing, set when testing if using multiple JRuby instances and you need to control when they are flushed from the pool
@@ -101,91 +108,109 @@ contains an integrated REPL that can be used in place of the `lein repl` command
101108

102109
To start the server from the REPL, run the following:
103110

104-
$ lein repl
105-
nREPL server started on port 47631 on host 127.0.0.1
106-
dev-tools=> (go)
107-
dev-tools=> (help)
111+
```clojure
112+
$ lein repl
113+
nREPL server started on port 47631 on host 127.0.0.1
114+
dev-tools=> (go)
115+
dev-tools=> (help)
116+
```
108117

109118
Then, if you make changes to the source code, all you need to do in order to restart the server with the latest changes is:
110119

111-
dev-tools=> (reset)
120+
```clojure
121+
dev-tools=> (reset)
122+
```
112123

113124
Restarting the server this way should be significantly faster than restarting the entire JVM process.
114125

115126
You can also run the utility functions to inspect the state of the server, e.g.:
116127

117-
dev-tools=> (print-puppet-environment-states)
128+
```clojure
129+
dev-tools=> (print-puppet-environment-states)
130+
```
118131

119132
Have a look at `dev-tools.clj` if you're interested in seeing what other utility functions are available.
120133

121134
## Step 4b: Run the server from the command line
122135

123-
If you prefer not to run the server interactively in the REPL, you can launch it as a normal process. To start the Puppet Server when running from source, simply run the following:
136+
If you prefer not to run the server interactively in the REPL, you can launch it as a normal process. To start the OpenVox Server when running from source, simply run the following:
124137

125-
lein run -c /path/to/puppetserver.conf
138+
```bash
139+
lein run -c /path/to/puppetserver.conf
140+
```
126141

127142
## Step 4c: Development environment gotchas
128143

129144
## Missing git submodules
130145

131146
If you get an error like the following:
132147

133-
Execution error (LoadError) at org.jruby.RubyKernel/require
134-
(org/jruby/RubyKernel.java:970).
135-
(LoadError) no such file to load -- puppet
148+
```text
149+
Execution error (LoadError) at org.jruby.RubyKernel/require
150+
(org/jruby/RubyKernel.java:970).
151+
(LoadError) no such file to load -- puppet
152+
```
136153

137154
Then you've probably forgotten to fetch the git submodules.
138155

139156
## Failing tests
140157

141158
If you change the `:webserver :ssl-port` config option from the default value of `8140`, tests will fail with errors like the following:
142159

143-
lein test :only puppetlabs.general-puppet.general-puppet-int-test/test-external-command-execution
144-
145-
ERROR in (test-external-command-execution) (SocketChannelImpl.java:-2)
146-
Uncaught exception, not in assertion.
147-
expected: nil
148-
2019-02-06 14:58:50,541 WARN [async-dispatch-18] [o.e.j.s.h.ContextHandler] Empty contextPath
149-
actual: java.net.ConnectException: Connection refused
150-
at sun.nio.ch.SocketChannelImpl.checkConnect (SocketChannelImpl.java:-2)
151-
sun.nio.ch.SocketChannelImpl.finishConnect (SocketChannelImpl.java:717)
152-
org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent (DefaultConnectingIOReactor.java:171)
153-
org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents (DefaultConnectingIOReactor.java:145)
154-
org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute (AbstractMultiworkerIOReactor.java:348)
155-
org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute (PoolingNHttpClientConnectionManager.java:192)
156-
org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run (CloseableHttpAsyncClientBase.java:64)
157-
java.lang.Thread.run (Thread.java:844)
160+
```text
161+
lein test :only puppetlabs.general-puppet.general-puppet-int-test/test-external-command-execution
162+
163+
ERROR in (test-external-command-execution) (SocketChannelImpl.java:-2)
164+
Uncaught exception, not in assertion.
165+
expected: nil
166+
2019-02-06 14:58:50,541 WARN [async-dispatch-18] [o.e.j.s.h.ContextHandler] Empty contextPath
167+
actual: java.net.ConnectException: Connection refused
168+
at sun.nio.ch.SocketChannelImpl.checkConnect (SocketChannelImpl.java:-2)
169+
sun.nio.ch.SocketChannelImpl.finishConnect (SocketChannelImpl.java:717)
170+
org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent (DefaultConnectingIOReactor.java:171)
171+
org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents (DefaultConnectingIOReactor.java:145)
172+
org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute (AbstractMultiworkerIOReactor.java:348)
173+
org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute (PoolingNHttpClientConnectionManager.java:192)
174+
org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run (CloseableHttpAsyncClientBase.java:64)
175+
java.lang.Thread.run (Thread.java:844)
176+
```
158177

159178
Changing the `ssl-port` variable back to `8140` makes the tests run properly.
160179

161180
## Running the Agent
162181

163-
Use a command like the one below to run an agent against your running puppetserver:
182+
Use a command like the one below to run an agent against your running OpenVox Server:
164183

165-
puppet agent --confdir ~/.puppetlabs/etc/puppet \
166-
--debug -t
184+
```bash
185+
puppet agent --confdir ~/.puppetlabs/etc/puppet \
186+
--debug -t
187+
```
167188

168-
Note that a system installed Puppet Agent is ok for use with source-based PuppetDB and Puppet Server. The `--confdir` above specifies the same confdir that Puppet Server is using. Because the Puppet Agent and
169-
Puppet Server instances are both using the same confdir, they're both using the same certificates as well. This alleviates the need to sign certificates as a separate step.
189+
Note that a system installed Puppet Agent is ok for use with source-based OpenVoxDB and OpenVox Server. The `--confdir` above specifies the same confdir that OpenVox Server is using. Because the Puppet Agent and
190+
OpenVox Server instances are both using the same confdir, they're both using the same certificates as well. This alleviates the need to sign certificates as a separate step.
170191

171192
## Running the Agent inside a Docker container
172193

173-
You can easily run a Puppet Agent inside a Docker container, either by using the `host` network profile or by accessing the Puppetserver service using the Docker host IP:
194+
You can easily run a Puppet Agent inside a Docker container, either by using the `host` network profile or by accessing the OpenVox Server service using the Docker host IP:
174195

175-
docker run -ti \
176-
--name agent1 \
177-
puppet/puppet-agent-ubuntu \
178-
agent -t --server 172.17.0.1
196+
```bash
197+
docker run -ti \
198+
--name agent1 \
199+
ghcr.io/openvoxproject/openvoxagent:8 \
200+
agent -t --server 172.17.0.1
179201

180-
docker run -ti \
181-
--name agent2 \
182-
--network host \
183-
--add-host puppet:127.0.0.1 \
184-
puppet/puppet-agent-ubuntu
202+
docker run -ti \
203+
--name agent2 \
204+
--network host \
205+
--add-host puppet:127.0.0.1 \
206+
ghcr.io/openvoxproject/openvoxagent:8
207+
```
185208

186209
To start another Puppet Agent run in a previous container you can use the `docker start` command:
187210

188-
docker start -a agent1
211+
```bash
212+
docker start -a agent1
213+
```
189214

190215
## Running tests
191216

@@ -198,73 +223,89 @@ arguments for the `:jvm-opts` `defproject` key within the `project.clj` file, it
198223

199224
1. An environment variable named `PUPPETSERVER_HEAP_SIZE`. For example, to use a heap size of 6 GiB for a `lein test` run, you could run the following:
200225

201-
$ PUPPETSERVER_HEAP_SIZE=6G lein test
226+
```bash
227+
PUPPETSERVER_HEAP_SIZE=6G lein test
228+
```
202229

203230
2. A lein `profiles.clj` setting in the `:user` profile under the `:puppetserver-heap-size` key. For example, to use a heap size of 6 GiB, you could add the following key to your `~/.lein/profiles.clj` file:
204231

205-
{:user {:puppetserver-heap-size "6G"
206-
...}}
232+
```clojure
233+
{:user {:puppetserver-heap-size "6G"
234+
...}}
235+
```
207236

208237
With the `:puppetserver-heap-size` key defined in the `profiles.clj` file, any subsequent `lein test` run would utilize the associated value for the key. If both the environment variable and the `profiles.clj`
209238
key are defined, the value from the environment variable takes precedence. When either of these settings is defined, the value is used as both the minimum and maximum JVM heap size.
210239

211-
From anecdotal testing from the puppetserver master branch as of 10/26/2016, it appeared that at least a heap size of 5 GB would provide the best performance benefit for full runs of the Clojure unit test
212-
suite. This value may change over time depending upon how the tests evolve.
240+
From anecdotal testing, at least a heap size of 5 GB provides the best performance benefit for full runs of the Clojure unit test suite. This value may change over time depending upon how the tests evolve.
213241

214242
## Installing Ruby Gems for Development
215243

216-
The gems that are vendored with the puppetserver OS packages will be automatically installed into your dev environment by the `./dev-setup` script. If you wish to install additional gems, please see the
244+
The gems that are vendored with the openvox-server OS packages will be automatically installed into your dev environment by the `./dev-setup` script. If you wish to install additional gems, please see the
217245
[Gems](./gems.html) document for detailed information.
218246

219247
## Debugging
220248

221-
For more information about debugging both Clojure and JRuby code, please see [Puppet Server: Debugging](./dev_debugging.html) documentation.
249+
For more information about debugging both Clojure and JRuby code, please see [OpenVox Server: Debugging](./dev_debugging.html) documentation.
222250

223-
## Running PuppetDB
251+
## Running OpenVoxDB
224252

225-
To run a source PuppetDB with Puppet Server, Puppet Server needs standard PuppetDB configuration and how to find the PuppetDB terminus. First copy the `dev/puppetserver.conf` file to another directory. In your
226-
copy of the config, append a new entry to the `ruby-load-path` list: `<PDB source path>/puppet/lib`. This tells PuppetServer to load the PuppetDB terminus from the specified directory.
253+
To run a source OpenVoxDB with OpenVox Server, OpenVox Server needs standard OpenVoxDB configuration and how to find the OpenVoxDB terminus. First copy the
254+
`dev/puppetserver.conf` file to another directory. In your copy of the config, append a new entry to the `ruby-load-path` list: `<PDB source path>/puppet/lib`. This tells
255+
OpenVox Server to load the OpenVoxDB terminus from the specified directory.
227256

228-
From here, the instructions are similar to installing PuppetDB manually via packages. The PuppetServer instance needs configuration for connecting to PuppetDB. Instructions on this configuration are below, but
229-
the official docs for this can be found [here](https://docs.puppet.com/puppetdb/4.3/connect_puppet_master.html).
257+
From here, the instructions are similar to installing OpenVoxDB manually via packages. The OpenVox Server instance needs configuration for connecting to OpenVoxDB. See the
258+
[OpenVoxDB documentation](../../../openvoxdb/latest/connect_puppet_server.html) for details.
230259

231260
Update `~/.puppetlabs/etc/puppet/puppet.conf` to include:
232261

233-
[master]
234-
storeconfigs = true
235-
storeconfigs_backend = puppetdb
236-
reports = store,puppetdb
262+
```ini
263+
[master]
264+
storeconfigs = true
265+
storeconfigs_backend = puppetdb
266+
reports = store,puppetdb
267+
```
237268

238269
Create a new puppetdb config file `~/.puppetlabs/etc/puppet/puppetdb.conf` that contains
239270

240-
[main]
241-
server_urls = https://<MASTERHOST>:8081
271+
```ini
272+
[main]
273+
server_urls = https://<MASTERHOST>:8081
274+
```
242275

243276
Then create a new routes file at `~/.puppetlabs/etc/puppet/routes.yaml` that contains
244277

245-
---
246-
master:
247-
facts:
248-
terminus: puppetdb
249-
cache: yaml
278+
```yaml
279+
---
280+
master:
281+
facts:
282+
terminus: puppetdb
283+
cache: yaml
284+
```
250285
251-
Assuming you have a PuppetDB instance up and running, start your Puppet Server instance with the new puppetserver.conf file that you changed:
286+
Assuming you have a OpenVoxDB instance up and running, start your OpenVox Server instance with the new puppetserver.conf file that you changed:
252287
253-
lein run -c ~/<YOUR CONFIG DIR>/puppetserver.conf
288+
```bash
289+
lein run -c ~/<YOUR CONFIG DIR>/puppetserver.conf
290+
```
254291

255-
Depending on your PuppetDB configuration, you might need to change some SSL config. PuppetDB requires that the same CA that signs it's certificate, also has signed Puppet Server's certificate. The easiest way
256-
to do this is to point PuppetDB at the same configuration directory that Puppet Server and Puppet Agent are pointing to. Typically this setting is specified in the `jetty.ini` file in the PuppetDB conf.d
292+
Depending on your OpenVoxDB configuration, you might need to change some SSL config. OpenVoxDB requires that the same CA that signs its certificate also has signed OpenVox Server's certificate. The easiest way
293+
to do this is to point OpenVoxDB at the same configuration directory that OpenVox Server and Puppet Agent are pointing to. Typically this setting is specified in the `jetty.ini` file in the OpenVoxDB conf.d
257294
directory. The update would look like:
258295

259-
[jetty]
296+
```ini
297+
[jetty]
260298

261-
#...
262-
ssl-cert = <home dir>/.puppetlabs/etc/puppet/ssl/certs/<MASTERHOST>.pem
263-
ssl-key = <home dir>/.puppetlabs/etc/puppet/ssl/private_keys/<MASTERHOST>.pem
264-
ssl-ca-cert = <home dir>/.puppetlabs/etc/puppet/ssl/certs/ca.pem
299+
#...
300+
ssl-cert = <home dir>/.puppetlabs/etc/puppet/ssl/certs/<MASTERHOST>.pem
301+
ssl-key = <home dir>/.puppetlabs/etc/puppet/ssl/private_keys/<MASTERHOST>.pem
302+
ssl-ca-cert = <home dir>/.puppetlabs/etc/puppet/ssl/certs/ca.pem
303+
```
265304

266-
After the SSL config is in place, start (or restart) PuppetDB:
305+
After the SSL config is in place, start (or restart) OpenVoxDB:
267306

268-
lein run services -c <path to PDB config>/conf.d
307+
```bash
308+
lein run services -c <path to PDB config>/conf.d
309+
```
269310

270-
Then run the Puppet Agent and you should see activity in PuppetDB and Puppet Server.
311+
Then run the Puppet Agent and you should see activity in OpenVoxDB and OpenVox Server.

0 commit comments

Comments
 (0)