Skip to content

Commit 3ba0ce9

Browse files
MDA2AVclaude
MDA2AV
and
claude
committed
Add 34 new HTTP/1.1 compliance, smuggling, and robustness tests (46 → 80)
New tests cover RFC 9110/9112 requirements for: comma-separated CL, chunked encoding edge cases (not-final, bare semicolon, hex prefix, underscores, leading spaces, missing CRLF, overflow, long extensions), TE in HTTP/1.0, bare CR in headers, Host validation (duplicate same, userinfo, path), asterisk-form, absolute-form, CONNECT empty port, method case sensitivity, leading CRLF, unknown TE, NUL in headers, and HTTP/2 preface detection. Includes probe STRICT dict entries and glossary documentation for all new tests. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 28aea9b commit 3ba0ce9

44 files changed

Lines changed: 1763 additions & 0 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/probe.yml‎

Lines changed: 182 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -370,6 +370,188 @@ jobs:
370370
'expected': '400/close/timeout',
371371
'reason': 'Whitespace-only request line is not valid HTTP'
372372
},
373+
# ── New Compliance tests ───────────────────────────────────
374+
'COMP-WHITESPACE-BEFORE-HEADERS': {
375+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
376+
'expected': '400 or close',
377+
'reason': 'Whitespace before first header is invalid (RFC 9112 §2.2)'
378+
},
379+
'COMP-DUPLICATE-HOST-SAME': {
380+
'accept': [400], 'close_ok': False, 'timeout_ok': False,
381+
'expected': '400',
382+
'reason': 'MUST respond with 400 for more than one Host header (RFC 9112 §3.2)'
383+
},
384+
'COMP-HOST-WITH-USERINFO': {
385+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
386+
'expected': '400 or close',
387+
'reason': 'Host header with userinfo is invalid (RFC 9112 §3.2)'
388+
},
389+
'COMP-HOST-WITH-PATH': {
390+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
391+
'expected': '400 or close',
392+
'reason': 'Host header with path component is invalid (RFC 9112 §3.2)'
393+
},
394+
'COMP-ASTERISK-WITH-GET': {
395+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
396+
'expected': '400 or close',
397+
'reason': 'Asterisk-form only valid for OPTIONS (RFC 9112 §3.2.4)'
398+
},
399+
'COMP-OPTIONS-STAR': {
400+
'accept': list(range(200, 300)),
401+
'close_ok': False, 'timeout_ok': False,
402+
'expected': '2xx',
403+
'reason': 'OPTIONS * is valid asterisk-form (RFC 9112 §3.2.4)'
404+
},
405+
'COMP-UNKNOWN-TE-501': {
406+
'accept': [400, 501], 'close_ok': True, 'timeout_ok': False,
407+
'expected': '400/501 or close',
408+
'reason': 'Unknown TE without CL should be rejected (RFC 9112 §6.1)'
409+
},
410+
'COMP-LEADING-CRLF': {
411+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
412+
'warn_on_2xx': True, 'scored': False,
413+
'expected': '400 or 2xx',
414+
'reason': 'Leading CRLF — server MAY ignore per RFC 9112 §2.2'
415+
},
416+
'COMP-ABSOLUTE-FORM': {
417+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
418+
'warn_on_2xx': True, 'scored': False,
419+
'expected': '400 or 2xx',
420+
'reason': 'Absolute-form is valid per RFC 9112 §3.2.2'
421+
},
422+
'COMP-METHOD-CASE': {
423+
'accept': [400, 405, 501], 'close_ok': True, 'timeout_ok': False,
424+
'warn_on_2xx': True, 'scored': False,
425+
'expected': '400/405/501 or 2xx',
426+
'reason': 'Methods are case-sensitive per RFC 9110 §9.1 — 2xx means case-insensitive'
427+
},
428+
'COMP-CONNECT-EMPTY-PORT': {
429+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
430+
'expected': '400 or close',
431+
'reason': 'CONNECT with empty port is invalid (RFC 9112 §3.2.3)'
432+
},
433+
# ── New Smuggling tests (scored) ───────────────────────────
434+
'SMUG-CL-COMMA-DIFFERENT': {
435+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
436+
'expected': '400 or close',
437+
'reason': 'Comma-separated different CL values must be rejected (RFC 9110 §8.6)'
438+
},
439+
'SMUG-TE-NOT-FINAL-CHUNKED': {
440+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
441+
'expected': '400 or close',
442+
'reason': 'Chunked must be the final encoding (RFC 9112 §7)'
443+
},
444+
'SMUG-TE-HTTP10': {
445+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
446+
'expected': '400 or close',
447+
'reason': 'TE in HTTP/1.0 is invalid (RFC 9112 §6.1)'
448+
},
449+
'SMUG-CHUNK-BARE-SEMICOLON': {
450+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
451+
'expected': '400 or close',
452+
'reason': 'Bare semicolon in chunk size without extension name (RFC 9112 §7.1.1)'
453+
},
454+
'SMUG-BARE-CR-HEADER-VALUE': {
455+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
456+
'expected': '400 or close',
457+
'reason': 'Bare CR must be rejected or replaced with SP (RFC 9112 §2.2)'
458+
},
459+
'SMUG-CL-OCTAL': {
460+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
461+
'expected': '400 or close',
462+
'reason': 'Octal prefix in CL is not valid DIGIT syntax (RFC 9110 §8.6)'
463+
},
464+
'SMUG-CHUNK-UNDERSCORE': {
465+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
466+
'expected': '400 or close',
467+
'reason': 'Underscores in chunk size are not valid HEXDIG (RFC 9112 §7.1)'
468+
},
469+
'SMUG-TE-EMPTY-VALUE': {
470+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
471+
'expected': '400 or close',
472+
'reason': 'Empty TE value with CL is ambiguous (RFC 9112 §6.1)'
473+
},
474+
'SMUG-TE-LEADING-COMMA': {
475+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
476+
'expected': '400 or close',
477+
'reason': 'Leading comma in TE is malformed (RFC 9112 §6.1)'
478+
},
479+
'SMUG-TE-DUPLICATE-HEADERS': {
480+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
481+
'expected': '400 or close',
482+
'reason': 'Two TE headers with CL is ambiguous framing (RFC 9112 §6.1)'
483+
},
484+
'SMUG-CHUNK-HEX-PREFIX': {
485+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
486+
'expected': '400 or close',
487+
'reason': '0x prefix in chunk size is not valid HEXDIG (RFC 9112 §7.1)'
488+
},
489+
'SMUG-CL-HEX-PREFIX': {
490+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
491+
'expected': '400 or close',
492+
'reason': 'Hex prefix in CL is not valid DIGIT syntax (RFC 9110 §8.6)'
493+
},
494+
'SMUG-CL-INTERNAL-SPACE': {
495+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
496+
'expected': '400 or close',
497+
'reason': 'Internal space in CL value is not valid DIGIT (RFC 9110 §8.6)'
498+
},
499+
'SMUG-CHUNK-LEADING-SP': {
500+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
501+
'expected': '400 or close',
502+
'reason': 'Leading space in chunk size is not valid HEXDIG (RFC 9112 §7.1)'
503+
},
504+
'SMUG-CHUNK-MISSING-TRAILING-CRLF': {
505+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
506+
'expected': '400 or close',
507+
'reason': 'Chunk data without trailing CRLF is malformed (RFC 9112 §7.1)'
508+
},
509+
# ── New Smuggling tests (unscored) ─────────────────────────
510+
'SMUG-TRANSFER_ENCODING': {
511+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
512+
'warn_on_2xx': True, 'scored': False,
513+
'expected': '400 or 2xx',
514+
'reason': 'Underscore makes it a different header — 2xx is valid'
515+
},
516+
'SMUG-CL-COMMA-SAME': {
517+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
518+
'warn_on_2xx': True, 'scored': False,
519+
'expected': '400 or 2xx',
520+
'reason': 'RFC allows merging identical CL values (RFC 9110 §8.6)'
521+
},
522+
'SMUG-CHUNKED-WITH-PARAMS': {
523+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
524+
'warn_on_2xx': True, 'scored': False,
525+
'expected': '400 or 2xx',
526+
'reason': 'Parameters on chunked encoding — some servers ignore'
527+
},
528+
'SMUG-EXPECT-100-CL': {
529+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
530+
'warn_on_2xx': True, 'scored': False,
531+
'expected': '400 or 2xx',
532+
'reason': 'Expect: 100-continue handling varies (RFC 9110 §10.1.1)'
533+
},
534+
# ── New Malformed Input tests ──────────────────────────────
535+
'MAL-NUL-IN-HEADER-VALUE': {
536+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
537+
'expected': '400 or close',
538+
'reason': 'NUL byte in header value is invalid'
539+
},
540+
'MAL-CHUNK-SIZE-OVERFLOW': {
541+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
542+
'expected': '400 or close',
543+
'reason': 'Chunk size overflow must be rejected'
544+
},
545+
'MAL-H2-PREFACE': {
546+
'accept': [400, 505], 'close_ok': True, 'timeout_ok': True,
547+
'expected': '400/505/close/timeout',
548+
'reason': 'HTTP/2 preface sent to HTTP/1.1 server is not valid HTTP/1.1'
549+
},
550+
'MAL-CHUNK-EXTENSION-LONG': {
551+
'accept': [400, 431], 'close_ok': True, 'timeout_ok': False,
552+
'expected': '400/431 or close',
553+
'reason': '100 KB chunk extension exceeds reasonable limits'
554+
},
373555
}
374556
375557
# ── Evaluate one server's results ────────────────────────────

‎docs/content/docs/headers/_index.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,4 +28,5 @@ HTTP header fields follow a strict grammar: `field-name ":" OWS field-value OWS`
2828
{{< card link="empty-header-name" title="EMPTY-HEADER-NAME" subtitle="Leading colon with no field name." >}}
2929
{{< card link="invalid-header-name" title="INVALID-HEADER-NAME" subtitle="Non-token characters in field name." >}}
3030
{{< card link="header-no-colon" title="HEADER-NO-COLON" subtitle="Header line with no colon separator." >}}
31+
{{< card link="whitespace-before-headers" title="WHITESPACE-BEFORE-HEADERS" subtitle="Whitespace before the first header line." >}}
3132
{{< /cards >}}

‎docs/content/docs/headers/whitespace-before-headers.md‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
---
2+
title: "WHITESPACE-BEFORE-HEADERS"
3+
description: "WHITESPACE-BEFORE-HEADERS test documentation"
4+
weight: 6
5+
---
6+
7+
| | |
8+
|---|---|
9+
| **Test ID** | `COMP-WHITESPACE-BEFORE-HEADERS` |
10+
| **Category** | Compliance |
11+
| **RFC** | [RFC 9112 Section 2.2](https://www.rfc-editor.org/rfc/rfc9112#section-2.2) |
12+
| **Requirement** | SHOULD reject |
13+
| **Expected** | `400` or close |
14+
15+
## What it sends
16+
17+
A request with whitespace (SP) before the first header line, between the request-line and the headers.
18+
19+
## What the RFC says
20+
21+
> "A recipient that receives whitespace between the start-line and the first header field MUST either reject the message as invalid or consume each whitespace-preceded line without further processing of it." — RFC 9112 Section 2.2
22+
23+
## Why it matters
24+
25+
Whitespace before headers can confuse parsers about where headers begin, potentially enabling smuggling.
26+
27+
## Sources
28+
29+
- [RFC 9112 Section 2.2](https://www.rfc-editor.org/rfc/rfc9112#section-2.2)

‎docs/content/docs/host-header/_index.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,4 +22,7 @@ This single sentence covers three violations:
2222
{{< cards >}}
2323
{{< card link="missing-host" title="MISSING-HOST" subtitle="No Host header present. MUST respond with 400." >}}
2424
{{< card link="duplicate-host" title="DUPLICATE-HOST" subtitle="Two Host headers with different values. MUST respond with 400." >}}
25+
{{< card link="duplicate-host-same" title="DUPLICATE-HOST-SAME" subtitle="Two Host headers with identical values. MUST respond with 400." >}}
26+
{{< card link="host-with-userinfo" title="HOST-WITH-USERINFO" subtitle="Host header with userinfo (user@host). Invalid field value." >}}
27+
{{< card link="host-with-path" title="HOST-WITH-PATH" subtitle="Host header with path component. Invalid field value." >}}
2528
{{< /cards >}}

‎docs/content/docs/host-header/duplicate-host-same.md‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
---
2+
title: "DUPLICATE-HOST-SAME"
3+
description: "DUPLICATE-HOST-SAME test documentation"
4+
weight: 3
5+
---
6+
7+
| | |
8+
|---|---|
9+
| **Test ID** | `COMP-DUPLICATE-HOST-SAME` |
10+
| **Category** | Compliance |
11+
| **RFC** | [RFC 9112 Section 3.2](https://www.rfc-editor.org/rfc/rfc9112#section-3.2) |
12+
| **Requirement** | MUST respond with 400 |
13+
| **Expected** | `400` |
14+
15+
## What it sends
16+
17+
A request with two identical Host headers.
18+
19+
## What the RFC says
20+
21+
> "A server MUST respond with a 400 (Bad Request) status code to any HTTP/1.1 request message that... contains more than one Host header field line..." — RFC 9112 Section 3.2
22+
23+
## Why it matters
24+
25+
The RFC mandates 400 for *any* duplicate Host headers, regardless of whether the values match. Some servers incorrectly allow identical duplicates.
26+
27+
## Sources
28+
29+
- [RFC 9112 Section 3.2](https://www.rfc-editor.org/rfc/rfc9112#section-3.2)

‎docs/content/docs/host-header/host-with-path.md‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
---
2+
title: "HOST-WITH-PATH"
3+
description: "HOST-WITH-PATH test documentation"
4+
weight: 5
5+
---
6+
7+
| | |
8+
|---|---|
9+
| **Test ID** | `COMP-HOST-WITH-PATH` |
10+
| **Category** | Compliance |
11+
| **RFC** | [RFC 9112 Section 3.2](https://www.rfc-editor.org/rfc/rfc9112#section-3.2) |
12+
| **Requirement** | MUST respond with 400 |
13+
| **Expected** | `400` or close |
14+
15+
## What it sends
16+
17+
A request with `Host: hostname:port/path`.
18+
19+
## What the RFC says
20+
21+
> "A server MUST respond with a 400 (Bad Request) status code to any HTTP/1.1 request message that... contains... a Host header field with an invalid field value." — RFC 9112 Section 3.2. The Host header grammar is `uri-host [ ":" port ]` with no path component allowed.
22+
23+
## Why it matters
24+
25+
A path in the Host header is a clear sign of manipulation. If a reverse proxy uses the Host to route, a path component could alter routing.
26+
27+
## Sources
28+
29+
- [RFC 9112 Section 3.2](https://www.rfc-editor.org/rfc/rfc9112#section-3.2)

‎docs/content/docs/host-header/host-with-userinfo.md‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
---
2+
title: "HOST-WITH-USERINFO"
3+
description: "HOST-WITH-USERINFO test documentation"
4+
weight: 4
5+
---
6+
7+
| | |
8+
|---|---|
9+
| **Test ID** | `COMP-HOST-WITH-USERINFO` |
10+
| **Category** | Compliance |
11+
| **RFC** | [RFC 9112 Section 3.2](https://www.rfc-editor.org/rfc/rfc9112#section-3.2) |
12+
| **Requirement** | MUST respond with 400 |
13+
| **Expected** | `400` or close |
14+
15+
## What it sends
16+
17+
A request with `Host: user@hostname:port`.
18+
19+
## What the RFC says
20+
21+
> "A server MUST respond with a 400 (Bad Request) status code to any HTTP/1.1 request message that... contains... a Host header field with an invalid field value." — RFC 9112 Section 3.2. The Host header grammar is `uri-host [ ":" port ]` — no userinfo is permitted.
22+
23+
## Why it matters
24+
25+
The userinfo component (`user@`) is not part of the Host grammar. A server that accepts it may be tricked into routing requests incorrectly.
26+
27+
## Sources
28+
29+
- [RFC 9112 Section 3.2](https://www.rfc-editor.org/rfc/rfc9112#section-3.2)
30+
- [RFC 3986 Section 3.2.1](https://www.rfc-editor.org/rfc/rfc3986#section-3.2.1)

‎docs/content/docs/line-endings/_index.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,3 +21,9 @@ RFC 9112 Section 2.2 defines that HTTP/1.1 messages use **CRLF** (`\r\n`) as the
2121
{{< card link="bare-lf-header" title="BARE-LF-HEADER" subtitle="Bare LF in a header field. Recipients MAY accept." >}}
2222
{{< card link="cr-only-line-ending" title="CR-ONLY-LINE-ENDING" subtitle="CR without LF. MUST consider invalid or replace with SP." >}}
2323
{{< /cards >}}
24+
25+
### Unscored
26+
27+
{{< cards >}}
28+
{{< card link="leading-crlf" title="LEADING-CRLF" subtitle="Leading CRLF before request-line. Server MAY ignore." >}}
29+
{{< /cards >}}

‎docs/content/docs/line-endings/leading-crlf.md‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
---
2+
title: "LEADING-CRLF"
3+
description: "LEADING-CRLF test documentation"
4+
weight: 4
5+
---
6+
7+
| | |
8+
|---|---|
9+
| **Test ID** | `COMP-LEADING-CRLF` |
10+
| **Category** | Compliance |
11+
| **RFC** | [RFC 9112 Section 2.2](https://www.rfc-editor.org/rfc/rfc9112#section-2.2) |
12+
| **Requirement** | MAY ignore (unscored) |
13+
| **Expected** | `400` or `2xx` |
14+
15+
## What it sends
16+
17+
Two leading CRLF sequences before the request-line.
18+
19+
## What the RFC says
20+
21+
> "In the interest of robustness, a server that is expecting to receive and parse a request-line SHOULD ignore at least one empty line (CRLF) received prior to the request-line." — RFC 9112 Section 2.2
22+
23+
## Why it matters
24+
25+
This is an unscored test. The RFC encourages servers to tolerate leading CRLFs. Either `400` (strict) or `2xx` (tolerant) is acceptable.
26+
27+
## Sources
28+
29+
- [RFC 9112 Section 2.2](https://www.rfc-editor.org/rfc/rfc9112#section-2.2)

‎docs/content/docs/malformed-input/_index.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,4 +32,8 @@ These tests send pathological, oversized, or completely invalid payloads. The go
3232
{{< card link="incomplete-request" title="INCOMPLETE-REQUEST" subtitle="Partial HTTP request." >}}
3333
{{< card link="empty-request" title="EMPTY-REQUEST" subtitle="Zero bytes sent." >}}
3434
{{< card link="whitespace-only-line" title="WHITESPACE-ONLY-LINE" subtitle="Only spaces/tabs, no method or URI." >}}
35+
{{< card link="nul-in-header-value" title="NUL-IN-HEADER-VALUE" subtitle="NUL byte in header value." >}}
36+
{{< card link="chunk-size-overflow" title="CHUNK-SIZE-OVERFLOW" subtitle="Chunk size integer overflow." >}}
37+
{{< card link="h2-preface" title="H2-PREFACE" subtitle="HTTP/2 preface sent to HTTP/1.1 server." >}}
38+
{{< card link="chunk-extension-long" title="CHUNK-EXTENSION-LONG" subtitle="100KB chunk extension value." >}}
3539
{{< /cards >}}

0 commit comments

Comments
 (0)