Skip to content

Commit 62fbef5

Browse files
josecelanojosecelano
committed
docs(issue-21): record D+2 live UDP verification state
1 parent 1e8b0a1 commit 62fbef5

3 files changed

Lines changed: 42 additions & 12 deletions

File tree

‎docs/issues/ISSUE-21-scale-up-server-for-udp-uptime.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,17 @@ Current public uptime observed in newTrackon for UDP is below target:
2121
This issue tracks a controlled resize experiment to determine whether capacity
2222
is the main bottleneck and to restore/maintain UDP uptime at or above 99%.
2323

24+
## Current State (2026-04-21)
25+
26+
- Resize (CCX23 -> CCX33) is complete and stable.
27+
- Conntrack overflow root cause was identified and fixed on 2026-04-20.
28+
- newTrackon rolling UDP uptime is still **85.70%**.
29+
- Recent probe samples in [newTrackon raw](https://newtrackon.com/raw) are
30+
currently reporting successful announces.
31+
32+
Interpretation: the rolling uptime percentage is still catching up after prior
33+
timeouts. Continue D+2 through D+7 monitoring before final pass/fail.
34+
2435
## Goal
2536

2637
Increase UDP tracker uptime to at least 99.0% over a rolling 7-day window while

‎docs/issues/evidence/ISSUE-21/02-post-resize-daily-checks.md‎

Lines changed: 26 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,30 @@
11
# Post-Resize Daily Checks (7 Days)
22

3+
<!-- cspell:ignore Rcvbuf -->
4+
35
## Daily Log Template
46

5-
| Day | Date (UTC) | HTTP1 req/s | UDP1 req/s | Total req/s | Req/s per vCPU | UDP uptime (%) | UDP errors trend | UDP aborted trend | Host load trend | Notes |
6-
| --- | ---------- | ----------- | ---------- | ----------- | -------------- | -------------- | ---------------- | ----------------- | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
7-
| D+1 | 2026-04-20 | ~1564 | ~1015 | ~2579 | ~322 | 83.9% | ~37k/h (pre-fix) | 0 | 6.05/5.49/4.80 | conntrack table full (262144/262144); fixed: nf_conntrack_max→1048576, UDP timeouts reduced; also includes planned resize downtime on 2026-04-14 |
8-
| D+2 | | | | | | | | | | |
9-
| D+3 | | | | | | | | | | |
10-
| D+4 | | | | | | | | | | |
11-
| D+5 | | | | | | | | | | |
12-
| D+6 | | | | | | | | | | |
13-
| D+7 | | | | | | | | | | |
7+
| Day | Date (UTC) | HTTP1 req/s | UDP1 req/s | Total req/s | Req/s per vCPU | UDP uptime (%) | UDP errors trend | UDP aborted trend | Host load trend | Notes |
8+
| --- | ---------- | ----------- | ---------- | ----------- | -------------- | -------------- | ---------------- | ----------------- | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- |
9+
| D+1 | 2026-04-20 | ~1564 | ~1015 | ~2579 | ~322 | 83.9% | ~37k/h (pre-fix) | 0 | 6.05/5.49/4.80 | conntrack table full (262144/262144); fixed: nf_conntrack_max→1048576, UDP timeouts reduced; also includes planned resize downtime on 2026-04-14 |
10+
| D+2 | 2026-04-21 | | | | | 85.70% | | | | Rolling uptime still low, but recent [newTrackon raw](https://newtrackon.com/raw) probes are currently successful; likely lag from prior failures |
11+
| D+3 | | | | | | | | | | |
12+
| D+4 | | | | | | | | | | |
13+
| D+5 | | | | | | | | | | |
14+
| D+6 | | | | | | | | | | |
15+
| D+7 | | | | | | | | | | |
16+
17+
## D+2 Live Verification Snapshot (2026-04-21T07:23:08Z)
18+
19+
- Host check command source: `ssh demotracker` runtime validation
20+
- `nf_conntrack_max`: `1048576`
21+
- `nf_conntrack_count`: `331258` (`31.59%` of max)
22+
- `nf_conntrack_udp_timeout_stream`: `15`
23+
- `nf_conntrack_udp_timeout`: `10`
24+
- `UdpRcvbufErrors`: `0`
25+
- `Udp6RcvbufErrors`: `0`
26+
- `dmesg` check (`sudo -n dmesg -T | grep -i "nf_conntrack: table full" | tail -10`): no recent matches
27+
28+
Interpretation: the configured conntrack sizing and UDP timeouts remain active
29+
on the live host, and there is no current evidence of UDP packet drops caused
30+
by conntrack table saturation.

‎docs/issues/evidence/ISSUE-21/03-pre-post-comparison.md‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -24,9 +24,11 @@ and reduced sustained reliability pressure.
2424
- [x] Partial: improved but below target — resize alone was insufficient; conntrack overflow was the actual bottleneck
2525
- [ ] No improvement: continue with next bottleneck path
2626

27-
**Status (2026-04-20):** Conntrack fix applied on D+1. UDP uptime recovery expected once newTrackon
28-
records post-fix probe results over the following 24–48 hours. 7-day window must
29-
complete before a final pass/fail decision can be made.
27+
**Status (2026-04-21):** Conntrack fix applied on D+1 and appears active. Rolling UDP
28+
uptime on newTrackon is still 85.70% on D+2, while recent probes in
29+
[newTrackon raw](https://newtrackon.com/raw) are currently successful. This
30+
supports a lagging rolling-window effect; 7-day monitoring must complete before
31+
a final pass/fail decision.
3032

3133
## Follow-up Actions
3234

0 commit comments

Comments
 (0)