Skip to content

Commit 3f0ef23

Browse files
josecelanojosecelano
committed
docs(issue-31): record T+next-day observation snapshot
Add next-day ISSUE-31 evidence with commands, outputs, and checkpoint comparison. Update ISSUE-31 status/checklist to mark snapshot completion and note that strict sustained-24h CPU/load trigger evaluation was interrupted by host restart. Refs: #31
1 parent 13f8bfc commit 3f0ef23

2 files changed

Lines changed: 247 additions & 5 deletions

File tree

‎docs/issues/ISSUE-31-reenable-caddy-http3-and-document-rationale.md‎

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -76,7 +76,10 @@ the rollback in evidence:
7676
`docs/issues/evidence/ISSUE-31/00-immediate-post-change-snapshot.md`.
7777
- T+1h checkpoint captured in
7878
`docs/issues/evidence/ISSUE-31/01-t1h-snapshot.md`.
79-
- T+next-day checkpoint and 24h rollback-trigger evaluation are still pending.
79+
- T+next-day checkpoint captured in
80+
`docs/issues/evidence/ISSUE-31/02-next-day-snapshot.md`.
81+
- Strict sustained-24h CPU/load trigger evaluation is currently inconclusive due
82+
to host restart between checkpoints (new uptime window).
8083

8184
## Implementation Plan
8285

@@ -85,20 +88,20 @@ the rollback in evidence:
8588
- [x] Validate Caddy health and confirm host UDP 443 listener exists after deploy.
8689
- [x] Capture immediate post-change metrics: `mpstat`, `docker stats`, Prometheus HTTP1/UDP1
8790
rates, and `newtrackon.com/raw` sample.
88-
- [ ] Capture T+next-day checkpoint with the same metrics.
91+
- [x] Capture T+next-day checkpoint with the same metrics.
8992
- [ ] Evaluate rollback triggers; if triggered, revert and record evidence.
9093
- [ ] Update ISSUE-29 text to explain why the earlier disablement is being reversed now.
9194
- [ ] Ensure ISSUE-29 states backend services do not need native HTTP/3 for edge HTTP/3 support.
9295
- [ ] Run `./scripts/lint.sh` and fix any markdown/cspell issues.
9396

9497
## Acceptance Criteria
9598

96-
- [ ] Caddy HTTP/3 edge capability is re-enabled via `443:443/udp` mapping.
97-
- [ ] Immediate, T+1h, and T+next-day evidence snapshots are recorded.
99+
- [x] Caddy HTTP/3 edge capability is re-enabled via `443:443/udp` mapping.
100+
- [x] Immediate, T+1h, and T+next-day evidence snapshots are recorded.
98101
- [ ] No rollback trigger is met during the observation window, or rollback is executed and
99102
documented if a trigger is met.
100103
- [ ] ISSUE-29 explicitly states that disabling HTTP/3 did not reduce CPU in prior observations.
101104
- [ ] ISSUE-29 explicitly states why HTTP/3 was re-enabled and under which conditions it may be
102105
disabled again.
103106
- [ ] Documentation clearly states edge HTTP/3 is independent from backend native HTTP/3 support.
104-
- [ ] All changed files pass `./scripts/lint.sh`.
107+
- [x] All changed files pass `./scripts/lint.sh`.

‎docs/issues/evidence/ISSUE-31/02-next-day-snapshot.md‎

Lines changed: 239 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,239 @@
1+
# ISSUE-31 Evidence — T+next-day Snapshot
2+
3+
**Issue:** [#31 Re-enable Caddy HTTP/3 and Document ISSUE-29 Rationale][issue-31]
4+
**Scope:** Post-change next-day checkpoint (T+next-day)
5+
**Captured:** 2026-05-08 08:37:47 UTC
6+
**Author:** Jose Celano
7+
8+
[issue-31]: https://github.com/torrust/torrust-tracker-demo/issues/31
9+
10+
---
11+
12+
## 1. Context
13+
14+
This snapshot is the next-day checkpoint for ISSUE-31 after re-enabling Caddy
15+
edge HTTP/3 (`443:443/udp`).
16+
17+
Goal: confirm that the edge HTTP/3 mapping remains active and healthy, and
18+
continue rollback-trigger evaluation after the T+0 and T+1h checkpoints.
19+
20+
---
21+
22+
## 2. Commands Executed and Outputs
23+
24+
### 2.1 Full capture command
25+
26+
```bash
27+
cat <<'EOS' | ssh demotracker 'bash -s'
28+
set -euo pipefail
29+
30+
echo '=== ISSUE-31 T+next-day snapshot ==='
31+
date -u +"UTC_TIME=%Y-%m-%dT%H:%M:%SZ"
32+
33+
echo
34+
35+
echo '--- caddy ports ---'
36+
docker ps --format 'table {{.Names}}\t{{.Ports}}' | grep caddy || true
37+
38+
echo
39+
40+
echo '--- udp listeners :443 ---'
41+
ss -ulnp | grep ':443' || true
42+
43+
echo
44+
45+
echo '--- caddy health ---'
46+
docker inspect --format='{{.State.Health.Status}}' caddy
47+
48+
echo
49+
50+
echo '--- http tracker health ---'
51+
code=$(curl -s -o /tmp/http1_hc.out -w '%{http_code}' https://http1.torrust-tracker-demo.com:443/health_check || true)
52+
echo "HTTP_CODE=$code"
53+
head -c 400 /tmp/http1_hc.out || true
54+
echo
55+
56+
echo
57+
58+
echo '--- api stats endpoint status ---'
59+
code=$(curl -s -o /tmp/api_stats.out -w '%{http_code}' https://http1.torrust-tracker-demo.com/api/v1/stats || true)
60+
echo "HTTP_CODE=$code"
61+
head -c 400 /tmp/api_stats.out || true
62+
echo
63+
64+
echo
65+
66+
echo '--- host uptime/load ---'
67+
uptime
68+
69+
echo
70+
71+
echo '--- mpstat -P ALL 1 1 ---'
72+
mpstat -P ALL 1 1
73+
74+
echo
75+
76+
echo '--- docker stats --no-stream ---'
77+
docker stats --no-stream --format 'table {{.Name}}\t{{.CPUPerc}}\t{{.MemUsage}}'
78+
79+
echo
80+
81+
echo '--- prometheus HTTP1 rate (5m) ---'
82+
curl -sG 'http://localhost:9090/api/v1/query' \
83+
--data-urlencode 'query=sum(rate(http_tracker_core_requests_received_total[5m]))'
84+
85+
echo
86+
87+
echo '--- prometheus UDP1 rate (5m) ---'
88+
curl -sG 'http://localhost:9090/api/v1/query' \
89+
--data-urlencode 'query=sum(rate(udp_tracker_server_requests_received_total[5m]))'
90+
91+
echo
92+
93+
echo '--- newtrackon raw page sample ---'
94+
curl -sL 'https://newtrackon.com/raw' | head -c 800
95+
96+
echo
97+
EOS
98+
```
99+
100+
### 2.2 Key outputs captured
101+
102+
Timestamp:
103+
104+
```text
105+
UTC_TIME=2026-05-08T08:37:47Z
106+
```
107+
108+
Caddy ports:
109+
110+
```text
111+
caddy 0.0.0.0:80->80/tcp, [::]:80->80/tcp,
112+
0.0.0.0:443->443/tcp, [::]:443->443/tcp,
113+
0.0.0.0:443->443/udp, [::]:443->443/udp, 2019/tcp
114+
```
115+
116+
UDP 443 listeners:
117+
118+
```text
119+
UNCONN 0 0 0.0.0.0:443 0.0.0.0:*
120+
UNCONN 0 0 [::]:443 [::]:*
121+
```
122+
123+
Caddy health:
124+
125+
```text
126+
healthy
127+
```
128+
129+
HTTP tracker health:
130+
131+
```text
132+
HTTP_CODE=200
133+
{"status":"Ok"}
134+
```
135+
136+
API stats endpoint:
137+
138+
```text
139+
HTTP_CODE=404
140+
```
141+
142+
Host load:
143+
144+
```text
145+
load average: 7.38, 7.15, 7.42
146+
```
147+
148+
`mpstat -P ALL 1 1` (aggregate):
149+
150+
| Metric | Value |
151+
| ------ | ----- |
152+
| %usr | 31.66 |
153+
| %sys | 14.93 |
154+
| %soft | 28.70 |
155+
| %idle | 24.71 |
156+
157+
`docker stats --no-stream`:
158+
159+
| Container | CPU% | Memory |
160+
| ---------- | ------- | ------------------ |
161+
| caddy | 396.15% | 529.2MiB / 30.6GiB |
162+
| tracker | 156.71% | 601MiB / 30.6GiB |
163+
| mysql | 4.26% | 611.8MiB / 30.6GiB |
164+
| grafana | 0.53% | 300.1MiB / 30.6GiB |
165+
| prometheus | 0.09% | 98.15MiB / 30.6GiB |
166+
167+
Prometheus rates (5m):
168+
169+
```text
170+
HTTP1: 1872.2701754385962 req/s
171+
UDP1: 1924.9859649122807 req/s
172+
```
173+
174+
newtrackon raw sample:
175+
176+
```text
177+
Returns HTML document content, not a plain tracker status line.
178+
```
179+
180+
---
181+
182+
## 3. Comparison vs Prior Checkpoints
183+
184+
| Signal | T+0 (13:17) | T+1h (16:29) | T+next-day (08:37) | Notes |
185+
| ------------------ | ------------ | ------------ | ------------------ | ------------------------- |
186+
| Caddy health | healthy | healthy | healthy | Stable |
187+
| UDP 443 published | yes | yes | yes | Stable |
188+
| UDP 443 listeners | yes | yes | yes | Stable |
189+
| HTTP health code | 200 | 200 | 200 | Stable |
190+
| API stats code | 500/unauth\* | 404 | 404 | Out of scope for ISSUE-31 |
191+
| HTTP1 rate (req/s) | 2116.38 | 1902.15 | 1872.27 | Normal variation |
192+
| UDP1 rate (req/s) | 2238.84 | 2053.50 | 1924.99 | Normal variation |
193+
| Caddy CPU% | 717.12 | 420.91 | 396.15 | Lower than restart spike |
194+
| Load avg (1m) | 18.93 | 11.46 | 7.38 | Lower than restart spike |
195+
196+
\*The T+0 note recorded `500 unauthorized`; from T+1h onward, `/api/v1/stats` returned `404`.
197+
198+
---
199+
200+
## 4. Rollback Trigger Check (Next-day)
201+
202+
Configured rollback triggers:
203+
204+
1. Caddy CPU > baseline x 1.20 sustained 24h
205+
2. Host load > baseline x 1.15 sustained 24h
206+
3. External availability regression on HTTP1/UDP1
207+
208+
### Assessment
209+
210+
- Trigger 3: **not observed** in this checkpoint.
211+
- Triggers 1 and 2: **inconclusive for sustained 24h comparison** because host uptime
212+
was only ~6h at capture time and kernel changed to `6.8.0-111-generic`, indicating
213+
a host restart and new observation window.
214+
215+
Interim conclusion: no immediate rollback signal; continue observation from this
216+
new host-uptime window if strict sustained-24h criteria are required.
217+
218+
---
219+
220+
## 5. Notable Environment Change
221+
222+
This snapshot shows:
223+
224+
- `uptime`: `up 6:37`
225+
- `uname` (from `mpstat` header): `Linux 6.8.0-111-generic`
226+
227+
Compared to earlier snapshots (`6.8.0-110-generic`), this indicates host reboot
228+
or kernel update occurred between checkpoints. This interrupts strict 24h
229+
continuity for baseline comparisons.
230+
231+
---
232+
233+
## 6. Summary
234+
235+
- Caddy edge HTTP/3 mapping (`443:443/udp`) remains active and healthy on IPv4 and IPv6.
236+
- HTTP and UDP tracker traffic rates remain in expected operating range.
237+
- No availability regression observed.
238+
- Sustained-24h rollback-trigger evaluation for CPU/load is interrupted by host reboot;
239+
results are operationally healthy but not a strict continuous 24h baseline window.

0 commit comments

Comments
 (0)