Skip to content

Commit 7c7b14c

Browse files
MDA2AVMDA2AV
committed
add results
1 parent b3316bc commit 7c7b14c

22 files changed

Lines changed: 468 additions & 272 deletions

‎site/content/docs/add-framework.md‎

Lines changed: 0 additions & 130 deletions
This file was deleted.

‎site/content/docs/add-framework/_index.md‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
---
2+
title: Add a Framework
3+
weight: 1
4+
---
5+
6+
Adding a framework to HttpArena takes a few steps: create a Dockerfile, add metadata, implement the required endpoints, and open a PR.
7+
8+
{{< cards >}}
9+
{{< card link="directory-structure" title="Directory Structure" subtitle="How to organize your framework's files — Dockerfile, meta.json, and source code." icon="folder" >}}
10+
{{< card link="endpoints" title="Endpoints" subtitle="All HTTP endpoints your framework must implement, organized by test profile." icon="code" >}}
11+
{{< card link="meta-json" title="meta.json" subtitle="Framework metadata — display name, language, type, and which tests to participate in." icon="document-text" >}}
12+
{{< card link="testing" title="Testing & Submitting" subtitle="How to validate your implementation locally and submit a pull request." icon="check-circle" >}}
13+
{{< /cards >}}

‎site/content/docs/add-framework/directory-structure.md‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
---
2+
title: Directory Structure
3+
---
4+
5+
Create a directory under `frameworks/` with your framework's name:
6+
7+
```
8+
frameworks/
9+
your-framework/
10+
Dockerfile
11+
meta.json
12+
... (source files)
13+
```
14+
15+
## Dockerfile
16+
17+
The Dockerfile should build and run your server. Containers are started with `--network host`, so bind to:
18+
19+
- **Port 8080** — HTTP/1.1
20+
- **Port 8443** — HTTPS with HTTP/2 and HTTP/3 (if supported)
21+
22+
Example (Go):
23+
24+
```dockerfile
25+
FROM golang:1.22-alpine AS build
26+
WORKDIR /app
27+
COPY go.mod go.sum ./
28+
RUN go mod download
29+
COPY . .
30+
RUN CGO_ENABLED=0 go build -o server .
31+
32+
FROM alpine:3.19
33+
COPY --from=build /app/server /server
34+
CMD ["/server"]
35+
```
36+
37+
## Mounted volumes
38+
39+
The benchmark runner mounts these paths into your container (read-only):
40+
41+
| Path | Purpose | When |
42+
|------|---------|------|
43+
| `/data/dataset.json` | 50-item dataset for `/json` endpoint | Always |
44+
| `/data/dataset-large.json` | 6000-item dataset for `/compression` endpoint | If `compression` in tests |
45+
| `/data/static/` | 20 static files (CSS, JS, HTML, fonts, images) | If `static-h2` or `static-h3` in tests |
46+
| `/certs/server.crt` | TLS certificate | If any H2/H3 test in tests |
47+
| `/certs/server.key` | TLS private key | If any H2/H3 test in tests |

‎site/content/docs/add-framework/endpoints.md‎

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
---
2+
title: Endpoints
3+
---
4+
5+
Your framework must implement endpoints depending on which test profiles it participates in. All endpoints are served on **port 8080** (HTTP/1.1) unless noted otherwise.
6+
7+
## Core endpoints
8+
9+
These are required for the `baseline`, `limited-conn`, and `noisy` profiles.
10+
11+
### `GET/POST /baseline11?a=N&b=N`
12+
13+
Parse query parameters `a` and `b`, compute their sum, and return it as plain text.
14+
15+
For POST requests, the body contains an additional integer to add. The server must accept both `Content-Length` and `chunked Transfer-Encoding` bodies.
16+
17+
```
18+
GET /baseline11?a=13&b=42 → "55"
19+
POST /baseline11?a=13&b=42 (body: "20") → "75"
20+
```
21+
22+
### `GET /pipeline`
23+
24+
Return a fixed `ok` response (exactly 2 bytes, `text/plain`). Used for the pipelined benchmark — should be as lightweight as possible.
25+
26+
## JSON processing
27+
28+
Required for the `json` profile.
29+
30+
### `GET /json`
31+
32+
Load `/data/dataset.json` at startup (50 items). For each request, compute `total = price * quantity` (rounded to 2 decimals) for every item and return:
33+
34+
```json
35+
{
36+
"items": [
37+
{"id": 1, "name": "Alpha Widget", "category": "electronics", "price": 29.99, "quantity": 5, "active": true, "tags": ["fast", "new"], "rating": {"score": 4.2, "count": 127}, "total": 149.95}
38+
],
39+
"count": 50
40+
}
41+
```
42+
43+
Response must have `Content-Type: application/json`.
44+
45+
## Upload
46+
47+
Required for the `upload` profile.
48+
49+
### `POST /upload`
50+
51+
Read the entire request body (up to 20 MB binary) and return its CRC32 checksum as an 8-character lowercase hex string.
52+
53+
```
54+
POST /upload (body: 20MB binary) → "a1b2c3d4"
55+
```
56+
57+
The CRC32 uses the ISO 3309 polynomial (`0xEDB88320`), the same as `zlib.crc32`.
58+
59+
## Compression
60+
61+
Required for the `compression` profile.
62+
63+
### `GET /compression`
64+
65+
Load `/data/dataset-large.json` at startup (6000 items). Compute `total` for each item (same as `/json`), and return the response with `Content-Type: application/json`.
66+
67+
The server **must** support gzip compression — when the client sends `Accept-Encoding: gzip`, the response should be gzip-compressed. Only frameworks with built-in gzip support are eligible.
68+
69+
## Static files
70+
71+
Required for the `static-h2` and `static-h3` profiles. Served over **HTTPS on port 8443**.
72+
73+
### `GET /static/{filename}`
74+
75+
Serve 20 pre-loaded static files from `/data/static/`. Files should be loaded into memory at startup and served with the correct `Content-Type`:
76+
77+
| Extension | Content-Type |
78+
|-----------|-------------|
79+
| `.css` | `text/css` |
80+
| `.js` | `application/javascript` |
81+
| `.html` | `text/html` |
82+
| `.woff2` | `font/woff2` |
83+
| `.svg` | `image/svg+xml` |
84+
| `.webp` | `image/webp` |
85+
| `.json` | `application/json` |
86+
87+
Return `404` for missing files.
88+
89+
## HTTP/2 baseline
90+
91+
Required for the `baseline-h2` profile. Served over **HTTPS on port 8443**.
92+
93+
### `GET /baseline2?a=N&b=N`
94+
95+
Same logic as `/baseline11` — parse query parameters and return their sum. Served over HTTP/2 with TLS.
96+
97+
Read the TLS certificate and key from `/certs/server.crt` and `/certs/server.key` (mounted read-only by the benchmark runner).
98+
99+
## HTTP/3
100+
101+
For `baseline-h3` and `static-h3` profiles. The same endpoints (`/baseline2` and `/static/*`) are served over **HTTP/3 (QUIC) on port 8443**.
102+
103+
This requires native QUIC support in the framework. Only add H3 tests to your `meta.json` if your framework supports it.

‎site/content/docs/add-framework/meta-json.md‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
---
2+
title: meta.json
3+
---
4+
5+
Create a `meta.json` file in your framework directory:
6+
7+
```json
8+
{
9+
"display_name": "your-framework",
10+
"language": "Go",
11+
"engine": "net/http",
12+
"type": "realistic",
13+
"description": "Short description of the framework and its key features.",
14+
"repo": "https://github.com/org/repo",
15+
"enabled": true,
16+
"tests": ["baseline", "pipelined", "limited-conn", "json", "upload", "compression", "noisy", "baseline-h2", "static-h2"]
17+
}
18+
```
19+
20+
## Fields
21+
22+
| Field | Description |
23+
|-------|-------------|
24+
| `display_name` | Name shown on the leaderboard |
25+
| `language` | Programming language (e.g., `Go`, `Rust`, `C#`, `Java`) |
26+
| `engine` | HTTP server engine (e.g., `Kestrel`, `Tomcat`, `hyper`) |
27+
| `type` | `realistic` for production-ready frameworks, `stripped` for custom/bare-metal implementations |
28+
| `description` | Shown in the framework detail popup on the leaderboard |
29+
| `repo` | Link to the framework's source repository |
30+
| `enabled` | Set to `false` to skip this framework during benchmark runs |
31+
| `tests` | Array of test profiles this framework participates in |
32+
33+
## Available test profiles
34+
35+
| Profile | Protocol | Required endpoints |
36+
|---------|----------|--------------------|
37+
| `baseline` | HTTP/1.1 | `/baseline11`, `/pipeline` |
38+
| `pipelined` | HTTP/1.1 | `/pipeline` |
39+
| `limited-conn` | HTTP/1.1 | `/baseline11` |
40+
| `json` | HTTP/1.1 | `/json` |
41+
| `upload` | HTTP/1.1 | `/upload` |
42+
| `compression` | HTTP/1.1 | `/compression` |
43+
| `noisy` | HTTP/1.1 | `/baseline11` |
44+
| `baseline-h2` | HTTP/2 | `/baseline2` (TLS, port 8443) |
45+
| `static-h2` | HTTP/2 | `/static/*` (TLS, port 8443) |
46+
| `baseline-h3` | HTTP/3 | `/baseline2` (QUIC, port 8443) |
47+
| `static-h3` | HTTP/3 | `/static/*` (QUIC, port 8443) |
48+
49+
Only include profiles your framework supports. Frameworks missing a profile simply don't appear in that profile's leaderboard.

‎site/content/docs/add-framework/testing.md‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
---
2+
title: Testing & Submitting
3+
---
4+
5+
## Validate your implementation
6+
7+
Run the validation script to check all endpoints return correct responses:
8+
9+
```bash
10+
./scripts/validate.sh your-framework
11+
```
12+
13+
This builds the Docker image, starts the container, and runs checks for every test profile listed in your `meta.json`. It verifies response bodies, status codes, content types, and anti-cheat randomized inputs.
14+
15+
## Run a benchmark
16+
17+
Test a single profile locally:
18+
19+
```bash
20+
./scripts/benchmark.sh your-framework baseline
21+
```
22+
23+
Run all profiles:
24+
25+
```bash
26+
./scripts/benchmark.sh your-framework
27+
```
28+
29+
By default, results are displayed but not saved. Add `--save` to persist results:
30+
31+
```bash
32+
./scripts/benchmark.sh --save your-framework
33+
```
34+
35+
## Submit a PR
36+
37+
Once validation passes and benchmarks run successfully:
38+
39+
1. Fork [HttpArena](https://github.com/MDA2AV/HttpArena)
40+
2. Add your `frameworks/your-framework/` directory
41+
3. Open a pull request
42+
43+
The PR should include:
44+
- `Dockerfile`
45+
- `meta.json`
46+
- Source files for your server implementation

0 commit comments

Comments
 (0)