Skip to content

Commit fbe6983

Browse files
jason-p-pickeringclaude
authored andcommitted
test: calibrate UserRolesPerformanceTest p95/max thresholds
Calibrated from the baseline/candidate A/B run on the platform-perf DB used to recalibrate PR #24489 after the rebase onto the JsonPatchFilterMixin refactor. PATCH scenarios share one threshold since the invariant under test is that PATCH latency must not depend on role membership size. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
1 parent 04d4ee5 commit fbe6983

1 file changed

Lines changed: 23 additions & 4 deletions

File tree

dhis-2/dhis-test-performance/src/test/java/org/hisp/dhis/test/platform/UserRolesPerformanceTest.java

Lines changed: 23 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -77,10 +77,15 @@
7777
* <li>{@code iterations} (default: {@code 10})
7878
* </ul>
7979
*
80-
* <p>No p95/max threshold assertions yet, only 100% success. Thresholds should be calibrated from
81-
* nightly baselines once this simulation has history (see {@link UsersPerformanceTest} for the
82-
* calibration workflow). On an unfixed server, the large-role PATCH may exceed Gatling's default
83-
* 60s request timeout; raise it with {@code -Dgatling.http.requestTimeout=600000}.
80+
* <p>Thresholds calibrated 2026-07-26 from a baseline/candidate A/B run on the platform-perf DB
81+
* (see PR #24489): fixed-code p95/max were empty-role PATCH 37/39ms, large-role (83,334 members)
82+
* PATCH 33/35ms, GET 6/6ms &mdash; consistent with the invariant this simulation checks, that PATCH
83+
* latency is independent of membership size. Thresholds are set well above that noise floor but far
84+
* below the pre-fix regression (large-role PATCH p95 was 17,304ms), so a reintroduced O(members)
85+
* hydration fails loudly while ordinary CI-runner jitter does not. Recalibrate the same way if
86+
* thresholds start flapping (see {@link UsersPerformanceTest} for the general workflow). On an
87+
* unfixed server, the large-role PATCH may exceed Gatling's default 60s request timeout; raise it
88+
* with {@code -Dgatling.http.requestTimeout=600000}.
8489
*
8590
* @author Morten Svanæs
8691
*/
@@ -126,6 +131,14 @@ private static String prop(String key, String defaultValue) {
126131
private static final String PATCH_LARGE_REQUEST = "PATCH UserRole - scalar (large role)";
127132
private static final String GET_LARGE_REQUEST = "GET UserRole - narrow fields (large role)";
128133

134+
private record Thresholds(int p95, int max) {}
135+
136+
// Thresholds (p95, max) in ms, calibrated 2026-07-26 (see class javadoc). Both PATCH scenarios
137+
// share one threshold: the point of this simulation is that PATCH latency must not depend on
138+
// membership size, so empty-role and large-role PATCH are held to the same bar.
139+
private static final Thresholds PATCH_THRESH = new Thresholds(100, 150);
140+
private static final Thresholds GET_THRESH = new Thresholds(30, 50);
141+
129142
private static final String PATCH_BODY_TEMPLATE =
130143
"""
131144
[{"op":"replace","path":"/description","value":"perf-patched %s"}]\
@@ -270,8 +283,14 @@ public UserRolesPerformanceTest() {
270283
.andThen(getLargeScenario.injectClosed(singleUser)))
271284
.protocols(httpProtocol)
272285
.assertions(
286+
details(PATCH_EMPTY_REQUEST).responseTime().percentile(95).lt(PATCH_THRESH.p95()),
287+
details(PATCH_EMPTY_REQUEST).responseTime().max().lt(PATCH_THRESH.max()),
273288
details(PATCH_EMPTY_REQUEST).successfulRequests().percent().is(100D),
289+
details(PATCH_LARGE_REQUEST).responseTime().percentile(95).lt(PATCH_THRESH.p95()),
290+
details(PATCH_LARGE_REQUEST).responseTime().max().lt(PATCH_THRESH.max()),
274291
details(PATCH_LARGE_REQUEST).successfulRequests().percent().is(100D),
292+
details(GET_LARGE_REQUEST).responseTime().percentile(95).lt(GET_THRESH.p95()),
293+
details(GET_LARGE_REQUEST).responseTime().max().lt(GET_THRESH.max()),
275294
details(GET_LARGE_REQUEST).successfulRequests().percent().is(100D));
276295
}
277296
}

0 commit comments

Comments
 (0)