Skip to content

Commit 824d204

Browse files
apeters1971apeters1971
committed
Document XrdHTTP architecture, wire layer, and HTTP/1 legacy paths.
Assisted-by: Cursor:Composer-2.5 CursorAI
1 parent ca697f5 commit 824d204

1 file changed

Lines changed: 231 additions & 0 deletions

File tree

‎src/XrdHttp/README.md‎

Lines changed: 231 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,231 @@
1+
# XrdHTTP
2+
3+
XrdHTTP is the HTTP and HTTPS protocol handler for XRootD. It exposes storage
4+
operations (GET, PUT, HEAD, DELETE, WebDAV, checksums, third-party copy, and
5+
more) over standard HTTP semantics and bridges authenticated requests into the
6+
native XRootD protocol via `XrdXrootdTransit`.
7+
8+
The implementation lives primarily in `XrdHttpProtocol` (connection lifecycle,
9+
TLS, configuration) and `XrdHttpReq` (per-request state, body I/O, Bridge
10+
interaction). Since the wire-layer re-platform, HTTP framing is handled by
11+
components under `wire/` while application logic remains shared across HTTP/1
12+
and HTTP/2.
13+
14+
## Architecture
15+
16+
```
17+
Client
18+
|
19+
v
20+
XrdHttpProtocol::Process()
21+
|
22+
+-- TLS handshake (HTTPS)
23+
+-- detectWireMode() (ALPN / H2 preface / SETTINGS heuristic)
24+
|
25+
+-- HTTP/1.1 +-- HTTP/2 (HTTPS + ALPN h2)
26+
| getDataOneShot() | XrdHttp2Session::drive()
27+
| header parse (llhttp or legacy) | nghttp2 stream assembly
28+
| |
29+
+------------------+------------------+
30+
v
31+
XrdHttpProtocol::processParsedRequest()
32+
|
33+
v
34+
XrdHttpReq (body, auth, handlers)
35+
|
36+
v
37+
XrdXrootdTransit / Bridge->Run()
38+
|
39+
v
40+
XrdHttp1ResponseWriter or XrdHttp2ResponseWriter
41+
```
42+
43+
Both wire protocols funnel parsed requests into the same application entry
44+
point. Auth, CGI mapping, redirects, range reads, WebDAV, and Bridge semantics
45+
are unchanged above that boundary.
46+
47+
### Wire layer (`wire/`)
48+
49+
| Component | Role |
50+
|-----------|------|
51+
| `XrdHttp1Session` | HTTP/1.1 request-line and header parsing via [llhttp](vendor/llhttp/) |
52+
| `XrdHttp1ResponseWriter` | HTTP/1.1 response headers, fixed-length bodies, and chunked encoding |
53+
| `XrdHttp2Session` | nghttp2 session management, stream state, `drive()` I/O loop |
54+
| `XrdHttp2ResponseWriter` | HTTP/2 response headers and DATA frames |
55+
| `XrdHttpConnection.hh` | `XrdHttpWireMode` enum (`kHttp1`, `kHttp2`) |
56+
57+
See [`wire/README.md`](wire/README.md) for wire-layer specifics.
58+
59+
### Application layer (largely unchanged)
60+
61+
| File | Role |
62+
|------|------|
63+
| `XrdHttpProtocol.cc` | `Process()`, TLS, configuration, `processParsedRequest()` |
64+
| `XrdHttpReq.cc` | Request state machine, body framing, file ops, Bridge calls |
65+
| `XrdHttpSecurity.cc` | TLS client auth, token validation |
66+
| `XrdHttpExtHandler.cc` | Pluggable external handlers (e.g. TPC) |
67+
| `XrdHttpReadRangeHandler.cc` | Byte-range and multipart responses |
68+
| `XrdHttpChecksum*.cc` | Checksum discovery and computation |
69+
| `XrdHttpMon.cc` | Monitoring hooks |
70+
71+
Feature-specific documentation:
72+
73+
- [Kerberos over HTTPS](README-KRB5.md)
74+
- [Checksums](README-CKSUM.md)
75+
- [HTTPS third-party copy](../XrdHttpTpc/README.md)
76+
77+
## HTTP/1.1: new wire path vs legacy parser
78+
79+
HTTP/1.1 header parsing has **two** selectable implementations. The default is
80+
llhttp; the previous line-based parser remains for regression testing.
81+
82+
| Layer | Default (llhttp) | Legacy (`http.parser legacy`) |
83+
|-------|------------------|-------------------------------|
84+
| Request headers | `XrdHttp1Session` + llhttp | `BuffgetLine()` + `XrdHttpReq::parseFirstLine()` / `parseLine()` |
85+
| Request body | Existing `XrdHttpReq` read path | Same |
86+
| Responses | `XrdHttp1ResponseWriter` | Same |
87+
88+
llhttp parses **headers only**. Content-Length, chunked upload bodies, and
89+
range-read streaming still use the long-standing read path in `XrdHttpReq`.
90+
91+
The legacy parser is the original naive line/token implementation. It is not
92+
removed; select it explicitly when comparing behaviour or running the legacy
93+
regression test suite.
94+
95+
## HTTP/2
96+
97+
HTTP/2 is available when XRootD is built with nghttp2 (`BUILD_HTTP2`).
98+
99+
- Negotiated on **HTTPS** connections via TLS ALPN (`h2`).
100+
- Plain HTTP connections remain HTTP/1.1.
101+
- `detectWireMode()` also recognises the cleartext connection preface and a
102+
SETTINGS-frame heuristic for edge cases.
103+
- Multiple requests on one connection are supported (same-connection reuse).
104+
- `Http2OutboundPending()` defers connection close while response DATA is still
105+
queued.
106+
107+
## Request lifecycle
108+
109+
1. **`Process(lp)`** — Main scheduler entry point. Reads socket data (HTTP/1),
110+
drives the nghttp2 session (HTTP/2), or re-enters after Bridge callbacks
111+
(`lp == nullptr`).
112+
113+
2. **Header parsing** — Populates `CurrentReq` (`XrdHttpReq`). On success,
114+
`CurrentReq.headerok` is set.
115+
116+
3. **`processParsedRequest(lp)`** — Shared application handler: auth checks,
117+
self-redirect, login, method dispatch (GET, PUT, PROPFIND, …).
118+
119+
4. **`Bridge->Run()`** — Async file operations via `XrdXrootdTransit`. Bridge
120+
callbacks re-invoke `Process(nullptr)`.
121+
122+
5. **Response** — `StartSimpleResp()` / `SendSimpleResp()` route to
123+
`XrdHttp1ResponseWriter` or `XrdHttp2ResponseWriter` based on
124+
`wireMode_`.
125+
126+
### Scheduler return codes
127+
128+
`Process()` return values follow XRootD scheduler semantics (see
129+
`src/Xrd/XrdLinkXeq.cc`):
130+
131+
| Return | Meaning |
132+
|--------|---------|
133+
| `0` | Continue processing (stick loop or Transit `reInvoke`) |
134+
| `1` | Idle — wait for poll I/O |
135+
| `< 0` | Close the connection |
136+
137+
HTTP/2 `drive()` must preserve this contract: return `1` when idle and waiting
138+
for bytes, but return `0` after `Bridge->Run()` so Transit can re-enter. Forcing
139+
idle `0 → 1` globally breaks Bridge integration.
140+
141+
## Configuration
142+
143+
Example server stanza: [`xrootd-http.cf`](xrootd-http.cf).
144+
145+
```cfg
146+
xrd.protocol XrdHttp /usr/lib64/libXrdHttp.so
147+
148+
http.cert /etc/grid-security/hostcert.pem
149+
http.key /etc/grid-security/hostkey.pem
150+
http.cadir /etc/grid-security/certificates
151+
152+
# HTTP/1.1 header parser (default: llhttp)
153+
http.parser llhttp
154+
# http.parser legacy
155+
156+
# Optional feature modules
157+
# http.exthandler xrdtpc libXrdHttpTPC.so
158+
# http.auth krb5
159+
```
160+
161+
| Directive | Values | Notes |
162+
|-----------|--------|-------|
163+
| `http.parser` | `llhttp`, `legacy` | HTTP/1.1 header parsing only |
164+
| `http.tlsclientauth` | `on`, `off` | TLS client certificate authentication |
165+
| `http.selfhttps2http` | `yes`, `no` | Signed redirect from HTTPS to HTTP |
166+
| `http.auth` | `krb5`, `tpc`, … | See feature READMEs |
167+
168+
HTTP/2 requires no separate config directive; it is negotiated at the TLS layer
169+
when the build includes nghttp2 and the client offers ALPN `h2`.
170+
171+
## Build
172+
173+
HTTP support is controlled by `ENABLE_HTTP`. HTTP/2 additionally requires
174+
**libnghttp2** (the development library, not tools-only packages):
175+
176+
```bash
177+
# macOS
178+
brew install libnghttp2
179+
180+
cmake .. -DENABLE_HTTP=ON
181+
# HTTP/2 is enabled automatically when libnghttp2 is found (BUILD_HTTP2)
182+
183+
make XrdHttpUtils XrdHttp
184+
```
185+
186+
Optional Kerberos HTTP auth: `-DENABLE_KRB5=ON` (see [README-KRB5.md](README-KRB5.md)).
187+
188+
Build outputs:
189+
190+
- `libXrdHttpUtils.so` — Protocol implementation (shared; plugins link against it)
191+
- `libXrdHttp.so` — Protocol plugin loaded by `xrd.protocol`
192+
193+
## Tests
194+
195+
From the build directory, after configuring with HTTP enabled:
196+
197+
```bash
198+
# TLS fixture required for httph2
199+
bash tests/tls/tls.sh setup
200+
201+
ctest -R 'XRootD::(http|httpparser|httph2)' --output-on-failure
202+
```
203+
204+
| CTest name | Script | Port | Coverage |
205+
|------------|--------|------|----------|
206+
| `XRootD::http` | `tests/XRootD/http.sh` | 7094 | Full HTTP/1.1 integration |
207+
| `XRootD::httpparser` | `tests/XRootD/httpparser.sh` | 7095 | llhttp header parsing |
208+
| `XRootD::httpparserlegacy` | `tests/XRootD/httpparserlegacy.sh` | 7096 | Legacy line parser regression |
209+
| `XRootD::httph2` | `tests/XRootD/httph2.sh` | 7097 | HTTPS + ALPN h2 (GET/PUT/HEAD/DELETE, same-connection reuse) |
210+
211+
## Source layout
212+
213+
```
214+
src/XrdHttp/
215+
README.md — this file
216+
README-KRB5.md — Kerberos / SPNEGO authentication
217+
README-CKSUM.md — Checksum support
218+
xrootd-http.cf — Example configuration
219+
XrdHttpProtocol.{cc,hh} — Protocol handler
220+
XrdHttpReq.{cc,hh} — Per-request logic
221+
XrdHttpModule.cc — Plugin entry point
222+
wire/ — HTTP/1 and HTTP/2 wire substrate
223+
vendor/llhttp/ — Bundled llhttp (MIT, v9.4.1)
224+
static/ — Embedded CSS and favicon for directory listings
225+
```
226+
227+
## Vendored dependencies
228+
229+
- **llhttp** — HTTP/1.1 request parser ([nodejs/llhttp](https://github.com/nodejs/llhttp)).
230+
See [`vendor/llhttp/README`](vendor/llhttp/README).
231+
- **nghttp2** — External dependency for HTTP/2 (not vendored).

0 commit comments

Comments
 (0)