Skip to content

Commit adb14ee

Browse files
committed
Soak: enforce RSS slope only for 30+ min runs (10-min too noisy)
Investigation: macOS arm64 grew 1MB (43→44MB) but slope was 1123 KB/hr. Linux amd64 grew 3MB (25→28MB) but slope was only 424 KB/hr. The linear regression fits noise on short runs — small timing fluctuations produce wildly different slopes on nearly flat data. For 10-min quick soak: use RSS ceiling (200MB) + ratio (3.0x) + FD drift + idle CPU. Slope is still reported but not enforced. For 30+ min runs: slope enforcement at 500 KB/hr (enough samples for reliable regression).
1 parent 0d4aa3f commit adb14ee

File tree

1 file changed

+23
-24
lines changed

1 file changed

+23
-24
lines changed

scripts/soak-test.sh

Lines changed: 23 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -381,30 +381,29 @@ FIRST_RSS=$(awk -F, 'NR==2 && $3>0 { printf "%.0f", $3/1024/1024 }' "$METRICS_CS
381381
LAST_RSS=$(awk -F, '$3>0 { last=$3 } END { printf "%.0f", last/1024/1024 }' "$METRICS_CSV")
382382
echo "RSS: first=${FIRST_RSS}MB last=${LAST_RSS}MB max=${MAX_RSS}MB (${TOTAL_SAMPLES} samples)" | tee -a "$SUMMARY"
383383

384-
if [ "$DURATION_MIN" -lt 10 ]; then
385-
# Short run: absolute ceiling (warmup noise dominates slope)
386-
if [ "${MAX_RSS:-0}" -gt 200 ] 2>/dev/null; then
387-
echo "FAIL: RSS ${MAX_RSS}MB > 200MB ceiling" | tee -a "$SUMMARY"
388-
PASS=false
389-
fi
390-
else
391-
# Long run: linear regression on post-warmup samples (skip first 20%)
392-
RSS_SLOPE=$(awk -F, -v skip="$((TOTAL_SAMPLES / 5))" '
393-
NR>1 && $3>0 {
394-
row++
395-
if (row <= skip) next
396-
n++; x=$1; y=$3; sx+=x; sy+=y; sxx+=x*x; sxy+=x*y
397-
}
398-
END {
399-
if (n<5) { print 0; exit }
400-
slope = (n*sxy - sx*sy) / (n*sxx - sx*sx)
401-
printf "%.0f", slope * 3600 / 1024
402-
}' "$METRICS_CSV")
403-
echo "RSS slope (post-warmup): ${RSS_SLOPE} KB/hr" | tee -a "$SUMMARY"
404-
if [ "${RSS_SLOPE:-0}" -gt 500 ] 2>/dev/null; then
405-
echo "FAIL: RSS slope ${RSS_SLOPE} KB/hr > 500 KB/hr" | tee -a "$SUMMARY"
406-
PASS=false
407-
fi
384+
# Absolute ceiling — catches catastrophic leaks on any run length
385+
if [ "${MAX_RSS:-0}" -gt 200 ] 2>/dev/null; then
386+
echo "FAIL: RSS ${MAX_RSS}MB > 200MB ceiling" | tee -a "$SUMMARY"
387+
PASS=false
388+
fi
389+
390+
# Slope — informational for short runs, enforced only for runs >= 30 min
391+
# (10-min runs have too few post-warmup samples for reliable regression)
392+
RSS_SLOPE=$(awk -F, -v skip="$((TOTAL_SAMPLES / 5))" '
393+
NR>1 && $3>0 {
394+
row++
395+
if (row <= skip) next
396+
n++; x=$1; y=$3; sx+=x; sy+=y; sxx+=x*x; sxy+=x*y
397+
}
398+
END {
399+
if (n<5) { print 0; exit }
400+
slope = (n*sxy - sx*sy) / (n*sxx - sx*sx)
401+
printf "%.0f", slope * 3600 / 1024
402+
}' "$METRICS_CSV")
403+
echo "RSS slope (post-warmup): ${RSS_SLOPE} KB/hr" | tee -a "$SUMMARY"
404+
if [ "$DURATION_MIN" -ge 30 ] && [ "${RSS_SLOPE:-0}" -gt 500 ] 2>/dev/null; then
405+
echo "FAIL: RSS slope ${RSS_SLOPE} KB/hr > 500 KB/hr" | tee -a "$SUMMARY"
406+
PASS=false
408407
fi
409408

410409
# Check 1b: RSS ratio (last / first) — catches step-function leaks

0 commit comments

Comments
 (0)