Skip to content

Commit 89adeb7

Browse files
apeters1971apeters1971
committed
Document xjcd and xjc in HTML docs and add xjcd(1) man page.
Adds runtime policy and bootstrap sections to index.md/index.html and registers xjcd.1 for installation with the other man pages. Assisted-by: Cursor:Composer-2.5 CursorAI
1 parent 9148ef3 commit 89adeb7

5 files changed

Lines changed: 299 additions & 1 deletion

File tree

‎docs/man/CMakeLists.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ set(MAN1PAGES
66
xrdcp.1
77
xrdfs.1
88
xrdmapc.1
9+
xjcd.1
910
)
1011

1112
set(MAN8PAGES

‎docs/man/xjcd.1‎

Lines changed: 159 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,159 @@
1+
.TH xjcd 1 "@XRootD_VERSION_STRING@"
2+
.SH NAME
3+
xjcd \- bootstrap JournalCache forwarding proxy configuration for systemd
4+
.SH SYNOPSIS
5+
.nf
6+
\fBxjcd\fR [\fB--journal\fR \fIPATH\fR] \fIcommand\fR [\fIargs\fR...]
7+
8+
\fBxjcd\fR \fBinit\fR [\fB--journal\fR \fIPATH\fR]
9+
\fB--xroot-port\fR \fIN\fR \fB--https-port\fR \fIN\fR
10+
\fB--tls-cert\fR \fIPATH\fR \fB--tls-key\fR \fIPATH\fR
11+
[\fB--install-systemd\fR] [\fB--systemd-unit\fR \fINAME\fR]
12+
[\fB--lib-dir\fR \fIPATH\fR] [\fB--plugin-suffix\fR \fIN\fR]
13+
14+
\fBxjcd\fR \fBrender\fR
15+
16+
\fBxjcd\fR \fBshow\fR
17+
18+
\fBxjcd\fR \fBvalidate\fR
19+
.fi
20+
.SH DESCRIPTION
21+
The \fBxjcd\fR utility prepares a JournalCache HTTP forwarding proxy deployment.
22+
It writes bootstrap state, XRootD configuration, client plugin snippets, a
23+
systemd \fBEnvironmentFile\fR, and a systemd unit under \fI$journal/.xjc/\fR.
24+
Process startup is left to \fBsystemd\fR; there is no \fBxjcd run\fR command.
25+
26+
Runtime policy (allowlist, redirects, bypass) is stored in
27+
\fI$journal/.xjc/policy.conf\fR and should be edited with \fBxjc\fR(1). That
28+
file is hot-reloaded by running xrootd processes when its mtime changes.
29+
30+
Initial policy is \fIopen\fR: no \fBallow_origin\fR lines are written until
31+
rules are added with \fBxjc\fR.
32+
.SH OPTIONS
33+
.TP
34+
.B \-\-journal PATH
35+
Journal cache root directory. May also be set via
36+
.BR XRD_JOURNALCACHE_CACHE .
37+
.TP
38+
.B \-\-help
39+
Print usage and exit.
40+
.SH COMMANDS
41+
.TP
42+
.B init
43+
Create \fI$journal/.xjc/state.conf\fR, render configuration files, and seed
44+
\fI$journal/.xjc/policy.conf\fR if missing. TLS certificate and key paths are
45+
required.
46+
.RS 7
47+
Init options:
48+
.TP
49+
.B \-\-xroot-port N
50+
TCP port for the native XRootD protocol.
51+
.TP
52+
.B \-\-https-port N
53+
TLS port for HTTP/HTTPS (XrdHttp).
54+
.TP
55+
.B \-\-tls-cert PATH
56+
Path to the TLS certificate (written into \fBhttp.cert\fR in \fBxrootd.cf\fR).
57+
.TP
58+
.B \-\-tls-key PATH
59+
Path to the TLS private key (written into \fBhttp.key\fR in \fBxrootd.cf\fR).
60+
.TP
61+
.B \-\-install-systemd
62+
After rendering, install \fI$journal/.xjc/etc/xjcd.service\fR into
63+
\fI/etc/systemd/system/\fR, run \fBsystemctl daemon-reload\fR, and
64+
\fBsystemctl enable \-\-now\fR the unit. Requires root (typically \fBsudo\fR).
65+
.TP
66+
.B \-\-systemd-unit NAME
67+
Systemd unit file name to install (default: \fBxjcd.service\fR). Use a distinct
68+
name when multiple journals run on one host (for example
69+
\fBjournalcache-foo.service\fR).
70+
.TP
71+
.B \-\-lib-dir PATH
72+
Directory containing XRootD plugin shared objects (default: \fI/usr/lib64\fR or
73+
\fI/usr/local/lib\fR on macOS).
74+
.TP
75+
.B \-\-plugin-suffix N
76+
Plugin version suffix in library names (default: \fB5\fR).
77+
.RE
78+
.TP
79+
.B render
80+
Regenerate all files under \fI$journal/.xjc/etc/\fR from
81+
\fI$journal/.xjc/state.conf\fR. Run after editing bootstrap parameters, then
82+
restart the systemd service.
83+
.TP
84+
.B show
85+
Print bootstrap paths, generated config locations, and systemd install hints.
86+
.TP
87+
.B validate
88+
Check journal writability, TLS file presence, and plugin library paths.
89+
.SH GENERATED LAYOUT
90+
.TP
91+
.B $journal/.xjc/state.conf
92+
Persistent bootstrap parameters written by \fBinit\fR.
93+
.TP
94+
.B $journal/.xjc/policy.conf
95+
Hot-reloadable runtime policy; edit with \fBxjc\fR(1).
96+
.TP
97+
.B $journal/.xjc/etc/xrootd.cf
98+
XRootD server configuration for the forwarding proxy.
99+
.TP
100+
.B $journal/.xjc/etc/journalcache-http.ext.conf
101+
JournalCache HTTP ext handler configuration (\fBforwarding = 1\fR).
102+
.TP
103+
.B $journal/.xjc/etc/client.plugins.d/journalcache.conf
104+
Client plugins loaded by the xrootd/PSS process.
105+
.TP
106+
.B $journal/.xjc/etc/xjcd.env
107+
Environment variables for \fBEnvironmentFile=\fR in systemd.
108+
.TP
109+
.B $journal/.xjc/etc/xjcd.service
110+
Generated systemd unit referencing the paths above.
111+
.SH EXAMPLES
112+
Initialize a journal and install the systemd unit:
113+
.nf
114+
sudo xjcd init \-\-journal /var/tmp/journalcache \\
115+
\-\-xroot-port 1094 \-\-https-port 8443 \\
116+
\-\-tls-cert /etc/xrootd/tls.crt \-\-tls-key /etc/xrootd/tls.key \\
117+
\-\-install-systemd
118+
.fi
119+
Initialize without installing the unit:
120+
.nf
121+
xjcd init \-\-journal /var/tmp/journalcache \\
122+
\-\-xroot-port 1094 \-\-https-port 8443 \\
123+
\-\-tls-cert /etc/xrootd/tls.crt \-\-tls-key /etc/xrootd/tls.key
124+
.fi
125+
Manual systemd install:
126+
.nf
127+
sudo install \-m 0644 /var/tmp/journalcache/.xjc/etc/xjcd.service /etc/systemd/system/
128+
sudo systemctl daemon-reload
129+
sudo systemctl enable \-\-now xjcd.service
130+
.fi
131+
Re-render after editing \fIstate.conf\fR:
132+
.nf
133+
xjcd render \-\-journal /var/tmp/journalcache
134+
sudo systemctl restart xjcd.service
135+
.fi
136+
.SH ENVIRONMENT
137+
.TP
138+
.B XRD_JOURNALCACHE_CACHE
139+
Default journal root when \fB\-\-journal\fR is omitted.
140+
.SH FILES
141+
.TP
142+
.I $journal/.xjc/state.conf
143+
Bootstrap state.
144+
.TP
145+
.I $journal/.xjc/policy.conf
146+
Runtime policy file.
147+
.TP
148+
.I $journal/.xjc/etc/
149+
Generated configuration directory.
150+
.TP
151+
.I /etc/systemd/system/xjcd.service
152+
Default install target for \fB\-\-install-systemd\fR.
153+
.SH SEE ALSO
154+
.BR xjc (1),
155+
.BR xrootd (8),
156+
.BR xrdfs (1)
157+
158+
Documentation: JournalCache plugin README and \fIhtml/index.html\fR under
159+
\fIXrdClJournalCachePlugin\fR.

‎src/XrdApps/XrdClJournalCachePlugin/README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -609,6 +609,8 @@ sudo systemctl enable --now xjcd.service
609609

610610
Re-run `xjcd render` after changing `state.conf`, then `systemctl restart xjcd.service`.
611611

612+
See **xjcd(1)** for the full command reference.
613+
612614
# 8 JournalCache in a Proxy server
613615

614616
To run a proxy server with JournalCache you create a usual proxy configuration file:

‎src/XrdApps/XrdClJournalCachePlugin/html/index.html‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -234,6 +234,8 @@ <h1>JournalCache</h1>
234234
<li><a href="#journal">Journal layout</a></li>
235235
<li><a href="#filesystem">Filesystem cache</a></li>
236236
<li><a href="#http">HTTP cache</a></li>
237+
<li><a href="#xjc">Runtime policy (xjc)</a></li>
238+
<li><a href="#xjcd">xjcd bootstrap</a></li>
237239
<li><a href="#proxy">Proxy setup</a></li>
238240
<li><a href="#cleaning">Cleaning</a></li>
239241
<li><a href="#stats">Statistics</a></li>
@@ -555,6 +557,62 @@ <h3>External redirect (bypass cache for subtrees)</h3>
555557
external_redirect = /store/raw/ root://data.cern.ch:1094//store/raw/</code></pre>
556558
</section>
557559

560+
<section id="xjc">
561+
<h2>Runtime policy and <code>xjc</code></h2>
562+
<p>Hot-reloadable policy knobs live in a runtime file (default: <code>$cache/.xjc/policy.conf</code>):</p>
563+
<pre><code>bypass = 0
564+
multi_origin = 1
565+
allow_origin = ^root://([a-z0-9.-]+\.)?cern\.ch(:1094)?/
566+
external_redirect = /live/ https://stream.example.org/live/</code></pre>
567+
<table>
568+
<tr><th>Key</th><th>Meaning</th></tr>
569+
<tr><td><code>policy = /path/to/policy.conf</code></td><td>Override runtime policy path (plugin + HTTP ext)</td></tr>
570+
<tr><td><code>policy_poll = 2</code></td><td>Mtime poll interval in seconds (<code>0</code> disables watch)</td></tr>
571+
</table>
572+
<p>Environment: <code>XRD_JOURNALCACHE_POLICY</code>, <code>XRD_JOURNALCACHE_POLICY_POLL</code>.</p>
573+
<p>Both the file plugin and HTTP ext watch the policy file and reload when its mtime changes. Edit with <strong>xjc</strong>:</p>
574+
<pre><code>xjc --policy /var/tmp/journalcache/.xjc/policy.conf show
575+
xjc bypass off
576+
xjc allow-origin add '^https://.*\.example\.org/'
577+
xjc redirect add /live/ https://stream.example.org/live/</code></pre>
578+
<p>See <strong>xjc(1)</strong> for the full command reference.</p>
579+
</section>
580+
581+
<section id="xjcd">
582+
<h2><code>xjcd</code> — forwarding proxy bootstrap</h2>
583+
<p><strong>xjcd</strong> writes the static XRootD + JournalCache layout under <code>$journal/.xjc/</code> and leaves process startup to <strong>systemd</strong> (no <code>xjcd run</code>).</p>
584+
<table>
585+
<tr><th>Path</th><th>Purpose</th></tr>
586+
<tr><td><code>$journal/.xjc/state.conf</code></td><td>Bootstrap parameters (ports, TLS paths, lib dir)</td></tr>
587+
<tr><td><code>$journal/.xjc/policy.conf</code></td><td>Runtime policy — edit with <strong>xjc</strong></td></tr>
588+
<tr><td><code>$journal/.xjc/etc/xrootd.cf</code></td><td>Generated XRootD config</td></tr>
589+
<tr><td><code>$journal/.xjc/etc/journalcache-http.ext.conf</code></td><td>HTTP ext (forwarding mode)</td></tr>
590+
<tr><td><code>$journal/.xjc/etc/client.plugins.d/journalcache.conf</code></td><td>Client plugins for the xrootd/PSS process</td></tr>
591+
<tr><td><code>$journal/.xjc/etc/xjcd.env</code></td><td><code>EnvironmentFile</code> snippet for systemd</td></tr>
592+
<tr><td><code>$journal/.xjc/etc/xjcd.service</code></td><td>Generated systemd unit</td></tr>
593+
</table>
594+
<p>Initial policy is <strong>open</strong> (no <code>allow_origin</code> lines) until you add rules with <strong>xjc</strong>. TLS certificate and key paths are <strong>required</strong> at init time.</p>
595+
<pre><code>sudo xjcd init --journal /var/tmp/journalcache \
596+
--xroot-port 1094 --https-port 8443 \
597+
--tls-cert /etc/xrootd/tls.crt --tls-key /etc/xrootd/tls.key \
598+
--install-systemd</code></pre>
599+
<p>With <code>--install-systemd</code>, init copies the generated unit into <code>/etc/systemd/system/</code>, runs <code>systemctl daemon-reload</code>, and <code>systemctl enable --now xjcd.service</code> (requires root). Use <code>--systemd-unit NAME</code> for a non-default unit name when running multiple journals on one host.</p>
600+
<p>Manual install (without the flag):</p>
601+
<pre><code>xjcd init --journal /var/tmp/journalcache \
602+
--xroot-port 1094 --https-port 8443 \
603+
--tls-cert /etc/xrootd/tls.crt --tls-key /etc/xrootd/tls.key
604+
605+
xjcd show --journal /var/tmp/journalcache
606+
xjcd validate --journal /var/tmp/journalcache
607+
xjcd render --journal /var/tmp/journalcache
608+
609+
sudo install -m 0644 /var/tmp/journalcache/.xjc/etc/xjcd.service /etc/systemd/system/
610+
sudo systemctl daemon-reload
611+
sudo systemctl enable --now xjcd.service</code></pre>
612+
<p>Re-run <code>xjcd render</code> after changing <code>state.conf</code>, then <code>systemctl restart xjcd.service</code>.</p>
613+
<p>See <strong>xjcd(1)</strong> for the full command reference.</p>
614+
</section>
615+
558616
<section id="proxy">
559617
<h2>Proxy and HTTP front-end setup</h2>
560618

‎src/XrdApps/XrdClJournalCachePlugin/html/index.md‎

Lines changed: 79 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ Client-side read cache for XRootD. Files are cached as append-only journals on l
44

55
See also the [plugin README](../README.md) for journal format details, statistics, and the full to-do list.
66

7-
**Table of contents:** [Overview](#overview) · [Components](#components) · [Configuration](#configuration) · [Journal layout](#journal-on-disk-layout) · [Filesystem cache](#filesystem-cache) · [HTTP cache](#http-cache-integration) · [Proxy setup](#proxy-and-http-front-end-setup) · [Cleaning](#cache-cleaning) · [Statistics](#statistics-and-monitoring) · [Serving docs](#serving-this-documentation)
7+
**Table of contents:** [Overview](#overview) · [Components](#components) · [Configuration](#configuration) · [Journal layout](#journal-on-disk-layout) · [Filesystem cache](#filesystem-cache) · [HTTP cache](#http-cache-integration) · [Runtime policy (`xjc`)](#runtime-policy-and-xjc) · [`xjcd` bootstrap](#xjcd--forwarding-proxy-bootstrap) · [Proxy setup](#proxy-and-http-front-end-setup) · [Cleaning](#cache-cleaning) · [Statistics](#statistics-and-monitoring) · [Serving docs](#serving-this-documentation)
88

99
---
1010

@@ -359,6 +359,84 @@ external_redirect = /store/raw/ root://data.cern.ch:1094//store/raw/
359359

360360
---
361361

362+
## Runtime policy and `xjc`
363+
364+
Hot-reloadable policy knobs live in a runtime file (default: `$cache/.xjc/policy.conf`):
365+
366+
```ini
367+
bypass = 0
368+
multi_origin = 1
369+
allow_origin = ^root://([a-z0-9.-]+\.)?cern\.ch(:1094)?/
370+
external_redirect = /live/ https://stream.example.org/live/
371+
```
372+
373+
| Key | Meaning |
374+
|-----|---------|
375+
| `policy = /path/to/policy.conf` | Override runtime policy path (plugin + HTTP ext) |
376+
| `policy_poll = 2` | Mtime poll interval in seconds (`0` disables watch) |
377+
378+
Environment: `XRD_JOURNALCACHE_POLICY`, `XRD_JOURNALCACHE_POLICY_POLL`.
379+
380+
Both the file plugin and HTTP ext watch the policy file and reload when its mtime changes. Edit with **`xjc`**:
381+
382+
```bash
383+
xjc --policy /var/tmp/journalcache/.xjc/policy.conf show
384+
xjc bypass off
385+
xjc allow-origin add '^https://.*\.example\.org/'
386+
xjc redirect add /live/ https://stream.example.org/live/
387+
```
388+
389+
See **xjc(1)** for the full command reference.
390+
391+
---
392+
393+
## `xjcd` — forwarding proxy bootstrap
394+
395+
**`xjcd`** writes the static XRootD + JournalCache layout under `$journal/.xjc/` and leaves process startup to **systemd** (no `xjcd run`).
396+
397+
| Path | Purpose |
398+
|------|---------|
399+
| `$journal/.xjc/state.conf` | Bootstrap parameters (ports, TLS paths, lib dir) |
400+
| `$journal/.xjc/policy.conf` | Runtime policy — edit with **`xjc`** |
401+
| `$journal/.xjc/etc/xrootd.cf` | Generated XRootD config |
402+
| `$journal/.xjc/etc/journalcache-http.ext.conf` | HTTP ext (forwarding mode) |
403+
| `$journal/.xjc/etc/client.plugins.d/journalcache.conf` | Client plugins for the xrootd/PSS process |
404+
| `$journal/.xjc/etc/xjcd.env` | `EnvironmentFile` snippet for systemd |
405+
| `$journal/.xjc/etc/xjcd.service` | Generated systemd unit |
406+
407+
Initial policy is **open** (no `allow_origin` lines) until you add rules with **`xjc`**. TLS certificate and key paths are **required** at init time.
408+
409+
```bash
410+
sudo xjcd init --journal /var/tmp/journalcache \
411+
--xroot-port 1094 --https-port 8443 \
412+
--tls-cert /etc/xrootd/tls.crt --tls-key /etc/xrootd/tls.key \
413+
--install-systemd
414+
```
415+
416+
With **`--install-systemd`**, init copies `$journal/.xjc/etc/xjcd.service` into `/etc/systemd/system/`, runs `systemctl daemon-reload`, and `systemctl enable --now xjcd.service` (requires root). Use **`--systemd-unit NAME`** for a non-default unit name when running multiple journals on one host.
417+
418+
Manual install (without the flag):
419+
420+
```bash
421+
xjcd init --journal /var/tmp/journalcache \
422+
--xroot-port 1094 --https-port 8443 \
423+
--tls-cert /etc/xrootd/tls.crt --tls-key /etc/xrootd/tls.key
424+
425+
xjcd show --journal /var/tmp/journalcache
426+
xjcd validate --journal /var/tmp/journalcache
427+
xjcd render --journal /var/tmp/journalcache # after editing state.conf
428+
429+
sudo install -m 0644 /var/tmp/journalcache/.xjc/etc/xjcd.service /etc/systemd/system/
430+
sudo systemctl daemon-reload
431+
sudo systemctl enable --now xjcd.service
432+
```
433+
434+
Re-run **`xjcd render`** after changing `state.conf`, then `systemctl restart xjcd.service`.
435+
436+
See **xjcd(1)** for the full command reference.
437+
438+
---
439+
362440
## Proxy and HTTP front-end setup
363441

364442
### 1. PSS proxy configuration

0 commit comments

Comments
 (0)