[Upload 20MB] Implementation rules

[remittor]

Rules:
https://www.http-arena.com/docs/test-profiles/h1/isolated/upload/implementation/#implementation-rules

The upload endpoint must actually read and process the request body. The returned byte count must be computed by reading the uploaded data, not inferred from request metadata.

• Do not return the value of the Content-Length header - this defeats the purpose of the test, which is to measure how efficiently the framework processes uploaded content.
• Frameworks may use small read buffers to process the upload incrementally. Holding the entire payload in memory is allowed but not required.
• The goal is to prove that the framework can efficiently compute a result over a large blob being sent to it.

HTTP engines are divided into two categories:

1. server loads the headers and immediately calls the user handler, to which it passes an object through which the user can read the body in parts, or the entire body at once up to the EOF.
2. server loads the headers and the request body completely into memory and only after that launches the user handler, to which it passes a pointer to a buffer containing the entire request body (already cleared of chunk headers, if exist TE-header).

In the first case, the rules are correct.
But in the second case, the rules force rereading data and creating unnecessary buffers. In the second case, the user handler immediately has access to any part of the request body and doesn't need to read it from beginning to end — it can work with any part of the body immediately.

Therefore, the rules need to be changed.

[MDA2AV]

If the user has access to the body as in the second case they can just return the body length directly, they are not reading it from the request headers but from the actual already received body which is in memory, naturally they still are in a disadvantage because reading in chunks is much more efficient but if a frameworks implements the second option, it is a bad option

[remittor]

they can just return the body length directly

Ok

if a frameworks implements the second option, it is a bad option

I know. The first option is quite difficult to implement correctly.
There are a lot of servers on GitHub that expect to receive the entire request with the body.

# nginx
proxy_request_buffering on;  # default

[MDA2AV]

I guess for the vast majority of cases the requests are small sized and buffering in the request parsing phase works out, but the first option to let the user control how to buffer it is a much superior API, when receiving large requests content sometimes there are 50+ ms delay between chunks in which the user can already be processing the already received data.

If a framework only implements the second option, it should be penalized

[Kaliumhexacyanoferrat]

But in the second case, the rules force rereading data and creating unnecessary buffers.

Why so? The rules only state how the framework should work, not how the user handler has to be implemented. In GenHTTP, the framework buffers large uploads into a file so the handler looks as simple as Upload(Stream body) => body.Length, but I do not see how this violates the rules as:

• The framework uses small buffers to process the data (writing it to disk)
• The handler does not analyze the Content-Lenght header