Skip to content

Commit d349f4d

Browse files
MDA2AVclaude
andcommitted
Add 10 body/content handling tests, backfill 22 missing STRICT entries
New tests verify body consumption with fixed Content-Length and chunked Transfer-Encoding (POST-CL-BODY, CHUNKED-BODY, CHUNKED-MULTI, etc.) plus edge cases (undersend, missing terminator, GET-with-body, chunk extensions). Also adds the 22 STRICT dict entries that were missing from the previous commit (upgrade, method, trailer, chunk funky tests). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent dfc3420 commit d349f4d

16 files changed

Lines changed: 821 additions & 0 deletions

File tree

.github/workflows/probe.yml

Lines changed: 186 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -554,6 +554,192 @@ jobs:
554554
'expected': '400/431 or close',
555555
'reason': '100 KB chunk extension exceeds reasonable limits'
556556
},
557+
'MAL-CL-EMPTY': {
558+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
559+
'expected': '400 or close',
560+
'reason': 'Empty Content-Length value is not valid DIGIT (RFC 9110 §8.6)'
561+
},
562+
'MAL-CL-TAB-BEFORE-VALUE': {
563+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
564+
'warn_on_2xx': True, 'scored': False,
565+
'expected': '400 or 2xx',
566+
'reason': 'Tab as OWS is valid per RFC 9110 §5.5 — 2xx is compliant'
567+
},
568+
# ── Missing Compliance tests ──────────────────────────────────
569+
'COMP-UPGRADE-POST': {
570+
'accept': [c for c in range(100, 600) if c != 101],
571+
'close_ok': True, 'timeout_ok': False,
572+
'expected': '!101',
573+
'reason': 'WebSocket upgrade via POST must not be accepted (RFC 6455 §4.1)'
574+
},
575+
'COMP-UPGRADE-MISSING-CONN': {
576+
'accept': [c for c in range(100, 600) if c != 101],
577+
'close_ok': True, 'timeout_ok': False,
578+
'expected': '!101',
579+
'reason': 'Upgrade without Connection: Upgrade must not trigger 101 (RFC 9110 §7.8)'
580+
},
581+
'COMP-UPGRADE-UNKNOWN': {
582+
'accept': [c for c in range(100, 600) if c != 101],
583+
'close_ok': True, 'timeout_ok': False,
584+
'expected': '!101',
585+
'reason': 'Upgrade to unknown protocol must not return 101 (RFC 9110 §7.8)'
586+
},
587+
'COMP-METHOD-CONNECT': {
588+
'accept': [400, 405, 501], 'close_ok': True, 'timeout_ok': False,
589+
'expected': '400/405/501 or close',
590+
'reason': 'CONNECT to an origin server must be rejected (RFC 9110 §9.3.6)'
591+
},
592+
'COMP-METHOD-CONNECT-NO-PORT': {
593+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
594+
'expected': '400 or close',
595+
'reason': 'CONNECT without port in authority-form is invalid (RFC 9112 §3.2.3)'
596+
},
597+
'COMP-EXPECT-UNKNOWN': {
598+
'accept': [417], 'close_ok': True, 'timeout_ok': False,
599+
'warn_on_2xx': True,
600+
'expected': '417 or 2xx',
601+
'reason': 'Unknown Expect value — 417 is correct, 2xx means ignored (RFC 9110 §10.1.1)'
602+
},
603+
'COMP-UPGRADE-INVALID-VER': {
604+
'accept': [c for c in range(100, 600) if c != 101],
605+
'close_ok': True, 'timeout_ok': False,
606+
'scored': False,
607+
'expected': '!101',
608+
'reason': 'Unsupported WebSocket version — 426 is correct, 101 is failure (RFC 6455 §4.4)'
609+
},
610+
'COMP-METHOD-TRACE': {
611+
'accept': [405, 501], 'close_ok': True, 'timeout_ok': False,
612+
'warn_on_2xx': True, 'scored': False,
613+
'expected': '405/501 or 2xx',
614+
'reason': 'TRACE should be disabled in production — 2xx means enabled (RFC 9110 §9.3.8)'
615+
},
616+
# ── Missing Smuggling tests (scored) ─────────────────────────
617+
'SMUG-CHUNK-EXT-LF': {
618+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
619+
'expected': '400 or close',
620+
'reason': 'Bare LF in chunk extension must be rejected (RFC 9112 §7.1.1)'
621+
},
622+
'SMUG-CHUNK-SPILL': {
623+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
624+
'expected': '400 or close',
625+
'reason': 'Oversized chunk data must be rejected (RFC 9112 §7.1)'
626+
},
627+
'SMUG-CHUNK-LF-TERM': {
628+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
629+
'expected': '400 or close',
630+
'reason': 'Bare LF as chunk data terminator must be rejected (RFC 9112 §7.1)'
631+
},
632+
'SMUG-CHUNK-EXT-CTRL': {
633+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
634+
'expected': '400 or close',
635+
'reason': 'NUL byte in chunk extension must be rejected (RFC 9112 §7.1.1)'
636+
},
637+
'SMUG-CHUNK-LF-TRAILER': {
638+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
639+
'expected': '400 or close',
640+
'reason': 'Bare LF in chunked trailer termination must be rejected (RFC 9112 §7.1)'
641+
},
642+
'SMUG-TE-IDENTITY': {
643+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
644+
'expected': '400 or close',
645+
'reason': 'TE: identity (deprecated) with CL must be rejected (RFC 9112 §7)'
646+
},
647+
'SMUG-CHUNK-NEGATIVE': {
648+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
649+
'expected': '400 or close',
650+
'reason': 'Negative chunk size must be rejected (RFC 9112 §7.1)'
651+
},
652+
# ── Missing Smuggling tests (unscored) ───────────────────────
653+
'SMUG-TRAILER-CL': {
654+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
655+
'warn_on_2xx': True, 'scored': False,
656+
'expected': '400 or 2xx',
657+
'reason': 'CL in trailers is prohibited — 2xx means server ignored it (RFC 9110 §6.5.1)'
658+
},
659+
'SMUG-TRAILER-TE': {
660+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
661+
'warn_on_2xx': True, 'scored': False,
662+
'expected': '400 or 2xx',
663+
'reason': 'TE in trailers is prohibited — 2xx means server ignored it (RFC 9110 §6.5.1)'
664+
},
665+
'SMUG-TRAILER-HOST': {
666+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
667+
'warn_on_2xx': True, 'scored': False,
668+
'expected': '400 or 2xx',
669+
'reason': 'Host in trailers must not be used for routing (RFC 9110 §6.5.2)'
670+
},
671+
'SMUG-HEAD-CL-BODY': {
672+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
673+
'warn_on_2xx': True, 'scored': False,
674+
'expected': '400 or 2xx',
675+
'reason': 'HEAD with body — server must not leave body on connection (RFC 9110 §9.3.2)'
676+
},
677+
'SMUG-OPTIONS-CL-BODY': {
678+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
679+
'warn_on_2xx': True, 'scored': False,
680+
'expected': '400 or 2xx',
681+
'reason': 'OPTIONS with body — server should consume or reject (RFC 9110 §9.3.7)'
682+
},
683+
# ── Body / Content-Length / Chunked ──────────────────────────
684+
'COMP-POST-CL-BODY': {
685+
'accept': list(range(200, 300)),
686+
'close_ok': False, 'timeout_ok': False,
687+
'expected': '2xx',
688+
'reason': 'Valid POST with CL and matching body must be accepted (RFC 9112 §6.2)'
689+
},
690+
'COMP-POST-CL-ZERO': {
691+
'accept': list(range(200, 300)),
692+
'close_ok': True, 'timeout_ok': False,
693+
'expected': '2xx or close',
694+
'reason': 'POST with CL:0 and no body must be accepted (RFC 9112 §6.2)'
695+
},
696+
'COMP-POST-NO-CL-NO-TE': {
697+
'accept': list(range(200, 300)),
698+
'close_ok': True, 'timeout_ok': False,
699+
'expected': '2xx or close',
700+
'reason': 'POST with no CL/TE implies zero-length body (RFC 9112 §6.3)'
701+
},
702+
'COMP-POST-CL-UNDERSEND': {
703+
'accept': [400], 'close_ok': True, 'timeout_ok': True,
704+
'expected': '400/close/timeout',
705+
'reason': 'Incomplete body — server blocks waiting then times out (RFC 9112 §6.2)'
706+
},
707+
'COMP-CHUNKED-BODY': {
708+
'accept': list(range(200, 300)),
709+
'close_ok': False, 'timeout_ok': False,
710+
'expected': '2xx',
711+
'reason': 'Valid single-chunk POST must be accepted (RFC 9112 §7.1)'
712+
},
713+
'COMP-CHUNKED-MULTI': {
714+
'accept': list(range(200, 300)),
715+
'close_ok': False, 'timeout_ok': False,
716+
'expected': '2xx',
717+
'reason': 'Valid multi-chunk POST must be accepted (RFC 9112 §7.1)'
718+
},
719+
'COMP-CHUNKED-EMPTY': {
720+
'accept': list(range(200, 300)),
721+
'close_ok': True, 'timeout_ok': False,
722+
'expected': '2xx or close',
723+
'reason': 'Zero-length chunked body must be accepted (RFC 9112 §7.1)'
724+
},
725+
'COMP-CHUNKED-NO-FINAL': {
726+
'accept': [400], 'close_ok': True, 'timeout_ok': True,
727+
'expected': '400/close/timeout',
728+
'reason': 'Missing zero terminator — server blocks then times out (RFC 9112 §7.1)'
729+
},
730+
'COMP-GET-WITH-CL-BODY': {
731+
'accept': [400], 'close_ok': True, 'timeout_ok': False,
732+
'warn_on_2xx': True, 'scored': False,
733+
'expected': '400 or 2xx',
734+
'reason': 'GET with body is unusual but allowed (RFC 9110 §9.3.1)'
735+
},
736+
'COMP-CHUNKED-EXTENSION': {
737+
'accept': list(range(200, 300)) + [400],
738+
'close_ok': True, 'timeout_ok': False,
739+
'scored': False,
740+
'expected': '2xx or 400',
741+
'reason': 'Chunk extensions are valid per RFC 9112 §7.1.1 — 400 means unsupported'
742+
},
557743
}
558744
559745
# ── Evaluate one server's results ────────────────────────────

docs/content/compliance/_index.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,12 @@ Each test sends a request that violates a specific **MUST** or **MUST NOT** requ
4747
{ key: 'content-length', label: 'Content-Length', testIds: [
4848
'RFC9112-6.1-CL-NON-NUMERIC','RFC9112-6.1-CL-PLUS-SIGN'
4949
]},
50+
{ key: 'body', label: 'Body Handling', testIds: [
51+
'COMP-POST-CL-BODY','COMP-POST-CL-ZERO','COMP-POST-NO-CL-NO-TE',
52+
'COMP-POST-CL-UNDERSEND','COMP-CHUNKED-BODY','COMP-CHUNKED-MULTI',
53+
'COMP-CHUNKED-EMPTY','COMP-CHUNKED-NO-FINAL',
54+
'COMP-GET-WITH-CL-BODY','COMP-CHUNKED-EXTENSION'
55+
]},
5056
{ key: 'methods', label: 'Methods & Routing', testIds: [
5157
'COMP-METHOD-CONNECT','COMP-METHOD-CONNECT-NO-PORT',
5258
'COMP-UNKNOWN-TE-501','COMP-EXPECT-UNKNOWN','COMP-METHOD-TRACE'

docs/content/docs/_index.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ Reference documentation for every test in Http11Probe, organized by topic. Each
1515
{{< card link="headers" title="Header Syntax" subtitle="Obs-fold, space before colon, empty names, invalid characters, missing colon." icon="document-text" >}}
1616
{{< card link="host-header" title="Host Header" subtitle="Missing Host, duplicate Host — the only tests where RFC explicitly mandates 400." icon="server" >}}
1717
{{< card link="content-length" title="Content-Length" subtitle="Non-numeric CL, plus sign, integer overflow, leading zeros, negative values." icon="calculator" >}}
18+
{{< card link="body" title="Body Handling" subtitle="Content-Length body consumption, chunked transfer encoding, incomplete bodies, chunk extensions." icon="document-download" >}}
1819
{{< card link="smuggling" title="Request Smuggling" subtitle="CL+TE conflicts, TE obfuscation, pipeline injection, and why ambiguous framing is dangerous." icon="shield-exclamation" >}}
1920
{{< card link="malformed-input" title="Malformed Input" subtitle="Binary garbage, oversized fields, control characters, incomplete requests." icon="lightning-bolt" >}}
2021
{{< card link="upgrade" title="Upgrade / WebSocket" subtitle="Protocol upgrade validation, WebSocket handshake method and version checks." icon="arrow-up" >}}

docs/content/docs/body/_index.md

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
---
2+
title: Body Handling
3+
description: "Body Handling — Http11Probe documentation"
4+
weight: 6
5+
sidebar:
6+
open: false
7+
---
8+
9+
HTTP/1.1 defines two mechanisms for framing a request body: **Content-Length** (fixed-size) and **Transfer-Encoding: chunked** (variable-size). RFC 9112 Sections 6 and 7 specify how servers must read, validate, and terminate body data.
10+
11+
## Key Rules
12+
13+
**Content-Length** — the sender declares the exact byte count:
14+
15+
> "If a valid Content-Length header field is present without Transfer-Encoding, its decimal value defines the expected message body length in octets." — RFC 9112 Section 6.2
16+
17+
**Chunked encoding** — body is split into self-terminating chunks:
18+
19+
> "The chunked transfer coding wraps the payload body in order to transfer it as a series of chunks, each with its own size indicator, followed by an OPTIONAL trailer section containing trailer fields." — RFC 9112 Section 7.1
20+
21+
**No CL, no TE** — with neither header, the body length is zero:
22+
23+
> "If this is a request message and none of the above are true, then the message body length is zero (no message body is present)." — RFC 9112 Section 6.3
24+
25+
## Tests
26+
27+
{{< cards >}}
28+
{{< card link="post-cl-body" title="POST-CL-BODY" subtitle="POST with Content-Length and matching body. Must return 2xx." >}}
29+
{{< card link="post-cl-zero" title="POST-CL-ZERO" subtitle="POST with Content-Length: 0, no body. Must return 2xx." >}}
30+
{{< card link="post-no-cl-no-te" title="POST-NO-CL-NO-TE" subtitle="POST with neither CL nor TE. Implicit zero-length body." >}}
31+
{{< card link="post-cl-undersend" title="POST-CL-UNDERSEND" subtitle="POST with CL:10 but only 5 bytes sent. Incomplete body." >}}
32+
{{< card link="get-with-cl-body" title="GET-WITH-CL-BODY" subtitle="GET with Content-Length and body. Unusual but allowed." >}}
33+
{{< card link="chunked-body" title="CHUNKED-BODY" subtitle="Valid single-chunk POST. Must return 2xx." >}}
34+
{{< card link="chunked-multi" title="CHUNKED-MULTI" subtitle="Valid multi-chunk POST. Must return 2xx." >}}
35+
{{< card link="chunked-empty" title="CHUNKED-EMPTY" subtitle="Zero-length chunked body. Must return 2xx." >}}
36+
{{< card link="chunked-no-final" title="CHUNKED-NO-FINAL" subtitle="Chunked body without zero terminator. Incomplete transfer." >}}
37+
{{< card link="chunked-extension" title="CHUNKED-EXTENSION" subtitle="Chunk extension (valid per RFC). Server may accept or reject." >}}
38+
{{< /cards >}}
Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
---
2+
title: "CHUNKED-BODY"
3+
description: "CHUNKED-BODY test documentation"
4+
weight: 6
5+
---
6+
7+
| | |
8+
|---|---|
9+
| **Test ID** | `COMP-CHUNKED-BODY` |
10+
| **Category** | Compliance |
11+
| **RFC** | [RFC 9112 Section 7.1](https://www.rfc-editor.org/rfc/rfc9112#section-7.1) |
12+
| **Requirement** | MUST accept |
13+
| **Expected** | `2xx` |
14+
15+
## What it sends
16+
17+
A valid chunked POST with a single 5-byte chunk followed by the zero terminator.
18+
19+
```http
20+
POST / HTTP/1.1\r\n
21+
Host: localhost\r\n
22+
Transfer-Encoding: chunked\r\n
23+
\r\n
24+
5\r\n
25+
hello\r\n
26+
0\r\n
27+
\r\n
28+
```
29+
30+
## What the RFC says
31+
32+
> "The chunked transfer coding wraps the payload body in order to transfer it as a series of chunks, each with its own size indicator, followed by an OPTIONAL trailer section containing trailer fields." — RFC 9112 Section 7.1
33+
34+
A server that supports HTTP/1.1 must be able to decode chunked transfer encoding.
35+
36+
## Why it matters
37+
38+
Chunked encoding is fundamental to HTTP/1.1 — it enables streaming, server-sent data, and requests where the body size isn't known in advance. If a server can't decode a basic chunked body, it cannot fully participate in HTTP/1.1.
39+
40+
## Sources
41+
42+
- [RFC 9112 Section 7.1](https://www.rfc-editor.org/rfc/rfc9112#section-7.1)
Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
---
2+
title: "CHUNKED-EMPTY"
3+
description: "CHUNKED-EMPTY test documentation"
4+
weight: 8
5+
---
6+
7+
| | |
8+
|---|---|
9+
| **Test ID** | `COMP-CHUNKED-EMPTY` |
10+
| **Category** | Compliance |
11+
| **RFC** | [RFC 9112 Section 7.1](https://www.rfc-editor.org/rfc/rfc9112#section-7.1) |
12+
| **Requirement** | MUST accept |
13+
| **Expected** | `2xx` or close |
14+
15+
## What it sends
16+
17+
A chunked POST with only the zero terminator — a zero-length body.
18+
19+
```http
20+
POST / HTTP/1.1\r\n
21+
Host: localhost\r\n
22+
Transfer-Encoding: chunked\r\n
23+
\r\n
24+
0\r\n
25+
\r\n
26+
```
27+
28+
## What the RFC says
29+
30+
> "The last chunk has a chunk size of zero, indicating the end of the chunk data." — RFC 9112 Section 7.1
31+
32+
A zero-size first chunk is valid and indicates an empty body. The server must recognize the terminator and not block waiting for additional data.
33+
34+
## Why it matters
35+
36+
Empty chunked bodies occur when a client starts a chunked transfer but has nothing to send, or when a proxy rewrites a zero-length CL body into chunked encoding. The server must handle this edge case cleanly.
37+
38+
## Sources
39+
40+
- [RFC 9112 Section 7.1](https://www.rfc-editor.org/rfc/rfc9112#section-7.1)
Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
---
2+
title: "CHUNKED-EXTENSION"
3+
description: "CHUNKED-EXTENSION test documentation"
4+
weight: 10
5+
---
6+
7+
| | |
8+
|---|---|
9+
| **Test ID** | `COMP-CHUNKED-EXTENSION` |
10+
| **Category** | Compliance |
11+
| **RFC** | [RFC 9112 Section 7.1.1](https://www.rfc-editor.org/rfc/rfc9112#section-7.1.1) |
12+
| **Requirement** | SHOULD accept |
13+
| **Expected** | `2xx` = Pass; `400` = Warn |
14+
15+
## What it sends
16+
17+
A chunked POST where the chunk size line includes a valid extension: `5;ext=value`.
18+
19+
```http
20+
POST / HTTP/1.1\r\n
21+
Host: localhost\r\n
22+
Transfer-Encoding: chunked\r\n
23+
\r\n
24+
5;ext=value\r\n
25+
hello\r\n
26+
0\r\n
27+
\r\n
28+
```
29+
30+
## What the RFC says
31+
32+
> "The chunked encoding allows each chunk to include zero or more chunk extensions, immediately following the chunk-size, for the sake of supplying per-chunk metadata." — RFC 9112 Section 7.1.1
33+
34+
Chunk extensions are part of the chunked encoding grammar. A compliant parser should skip unrecognized extensions and process the chunk data normally.
35+
36+
## Why it matters
37+
38+
While chunk extensions are rarely used in practice, they are syntactically valid. A server that rejects them has an overly strict chunk parser that may break with legitimate clients or proxies that add extensions for metadata (e.g., checksums, signatures).
39+
40+
## Sources
41+
42+
- [RFC 9112 Section 7.1.1](https://www.rfc-editor.org/rfc/rfc9112#section-7.1.1)

0 commit comments

Comments
 (0)