Fixed flaky test for play-2.8: Source directory '.../routes' is not a directory.#9328
Merged
AlexeyKuznetsov-DD merged 9 commits intomasterfrom Aug 11, 2025
Merged
Conversation
…/src/play/routes' is not a directory.
AlexeyKuznetsov-DD
added
comp: testing
AlexeyKuznetsov-DD
requested review from
PerfectSlayer,
amarziali,
bric3,
mcculls and
sarahchen6
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 48 metrics, 11 unstable metrics. Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.53.0-SNAPSHOT~6556191861, baseline=1.53.0-SNAPSHOT~5ed2f5961b
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.043 s) : 0, 1042849
Total [baseline] (8.577 s) : 0, 8576771
Agent [candidate] (1.046 s) : 0, 1045783
Total [candidate] (8.653 s) : 0, 8652603
section iast
Agent [baseline] (1.18 s) : 0, 1180357
Total [baseline] (9.333 s) : 0, 9333124
Agent [candidate] (1.193 s) : 0, 1192998
Total [candidate] (9.308 s) : 0, 9307731
gantt
title insecure-bank - break down per module: candidate=1.53.0-SNAPSHOT~6556191861, baseline=1.53.0-SNAPSHOT~5ed2f5961b
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.43 ms) : 0, 1430
crashtracking [candidate] (1.423 ms) : 0, 1423
BytebuddyAgent [baseline] (730.501 ms) : 0, 730501
BytebuddyAgent [candidate] (730.916 ms) : 0, 730916
GlobalTracer [baseline] (241.726 ms) : 0, 241726
GlobalTracer [candidate] (242.946 ms) : 0, 242946
AppSec [baseline] (30.004 ms) : 0, 30004
AppSec [candidate] (30.27 ms) : 0, 30270
Debugger [baseline] (6.11 ms) : 0, 6110
Debugger [candidate] (6.047 ms) : 0, 6047
Remote Config [baseline] (654.228 µs) : 0, 654
Remote Config [candidate] (652.44 µs) : 0, 652
Telemetry [baseline] (11.416 ms) : 0, 11416
Telemetry [candidate] (12.6 ms) : 0, 12600
section iast
crashtracking [baseline] (1.423 ms) : 0, 1423
crashtracking [candidate] (1.446 ms) : 0, 1446
BytebuddyAgent [baseline] (851.47 ms) : 0, 851470
BytebuddyAgent [candidate] (863.011 ms) : 0, 863011
GlobalTracer [baseline] (233.774 ms) : 0, 233774
GlobalTracer [candidate] (234.483 ms) : 0, 234483
AppSec [baseline] (28.509 ms) : 0, 28509
AppSec [candidate] (29.234 ms) : 0, 29234
Debugger [baseline] (5.82 ms) : 0, 5820
Debugger [candidate] (5.868 ms) : 0, 5868
Remote Config [baseline] (580.497 µs) : 0, 580
Remote Config [candidate] (594.722 µs) : 0, 595
Telemetry [baseline] (8.505 ms) : 0, 8505
Telemetry [candidate] (8.491 ms) : 0, 8491
IAST [baseline] (29.261 ms) : 0, 29261
IAST [candidate] (28.659 ms) : 0, 28659
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.53.0-SNAPSHOT~6556191861, baseline=1.53.0-SNAPSHOT~5ed2f5961b
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.037 s) : 0, 1036815
Total [baseline] (10.642 s) : 0, 10642349
Agent [candidate] (1.043 s) : 0, 1042603
Total [candidate] (10.696 s) : 0, 10695518
section appsec
Agent [baseline] (1.219 s) : 0, 1218776
Total [baseline] (10.738 s) : 0, 10738333
Agent [candidate] (1.217 s) : 0, 1216655
Total [candidate] (10.803 s) : 0, 10803305
section iast
Agent [baseline] (1.176 s) : 0, 1175744
Total [baseline] (10.897 s) : 0, 10896843
Agent [candidate] (1.184 s) : 0, 1184483
Total [candidate] (10.956 s) : 0, 10955756
section profiling
Agent [baseline] (1.191 s) : 0, 1190668
Total [baseline] (10.842 s) : 0, 10842174
Agent [candidate] (1.191 s) : 0, 1191443
Total [candidate] (10.969 s) : 0, 10968813
gantt
title petclinic - break down per module: candidate=1.53.0-SNAPSHOT~6556191861, baseline=1.53.0-SNAPSHOT~5ed2f5961b
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.415 ms) : 0, 1415
crashtracking [candidate] (1.421 ms) : 0, 1421
BytebuddyAgent [baseline] (726.691 ms) : 0, 726691
BytebuddyAgent [candidate] (729.593 ms) : 0, 729593
GlobalTracer [baseline] (240.686 ms) : 0, 240686
GlobalTracer [candidate] (241.377 ms) : 0, 241377
AppSec [baseline] (29.769 ms) : 0, 29769
AppSec [candidate] (29.827 ms) : 0, 29827
Debugger [baseline] (5.989 ms) : 0, 5989
Debugger [candidate] (6.035 ms) : 0, 6035
Remote Config [baseline] (641.387 µs) : 0, 641
Remote Config [candidate] (643.144 µs) : 0, 643
Telemetry [baseline] (10.68 ms) : 0, 10680
Telemetry [candidate] (12.727 ms) : 0, 12727
section appsec
crashtracking [baseline] (1.428 ms) : 0, 1428
crashtracking [candidate] (1.425 ms) : 0, 1425
BytebuddyAgent [baseline] (752.346 ms) : 0, 752346
BytebuddyAgent [candidate] (751.277 ms) : 0, 751277
GlobalTracer [baseline] (234.245 ms) : 0, 234245
GlobalTracer [candidate] (233.858 ms) : 0, 233858
AppSec [baseline] (169.275 ms) : 0, 169275
AppSec [candidate] (168.118 ms) : 0, 168118
Debugger [baseline] (6.43 ms) : 0, 6430
Debugger [candidate] (8.623 ms) : 0, 8623
Remote Config [baseline] (628.014 µs) : 0, 628
Remote Config [candidate] (604.34 µs) : 0, 604
Telemetry [baseline] (9.832 ms) : 0, 9832
Telemetry [candidate] (8.339 ms) : 0, 8339
IAST [baseline] (23.59 ms) : 0, 23590
IAST [candidate] (23.449 ms) : 0, 23449
section iast
crashtracking [baseline] (1.424 ms) : 0, 1424
crashtracking [candidate] (1.434 ms) : 0, 1434
BytebuddyAgent [baseline] (849.07 ms) : 0, 849070
BytebuddyAgent [candidate] (856.084 ms) : 0, 856084
GlobalTracer [baseline] (232.511 ms) : 0, 232511
GlobalTracer [candidate] (233.497 ms) : 0, 233497
AppSec [baseline] (26.546 ms) : 0, 26546
AppSec [candidate] (28.401 ms) : 0, 28401
Debugger [baseline] (5.727 ms) : 0, 5727
Debugger [candidate] (7.413 ms) : 0, 7413
Remote Config [baseline] (573.788 µs) : 0, 574
Remote Config [candidate] (584.379 µs) : 0, 584
Telemetry [baseline] (8.297 ms) : 0, 8297
Telemetry [candidate] (8.444 ms) : 0, 8444
IAST [baseline] (30.553 ms) : 0, 30553
IAST [candidate] (27.505 ms) : 0, 27505
section profiling
crashtracking [baseline] (1.388 ms) : 0, 1388
crashtracking [candidate] (1.396 ms) : 0, 1396
BytebuddyAgent [baseline] (759.034 ms) : 0, 759034
BytebuddyAgent [candidate] (757.858 ms) : 0, 757858
GlobalTracer [baseline] (221.248 ms) : 0, 221248
GlobalTracer [candidate] (221.543 ms) : 0, 221543
AppSec [baseline] (29.707 ms) : 0, 29707
AppSec [candidate] (29.707 ms) : 0, 29707
Debugger [baseline] (6.24 ms) : 0, 6240
Debugger [candidate] (6.331 ms) : 0, 6331
Remote Config [baseline] (668.823 µs) : 0, 669
Remote Config [candidate] (671.056 µs) : 0, 671
Telemetry [baseline] (14.869 ms) : 0, 14869
Telemetry [candidate] (16.454 ms) : 0, 16454
ProfilingAgent [baseline] (108.098 ms) : 0, 108098
ProfilingAgent [candidate] (108.091 ms) : 0, 108091
Profiling [baseline] (108.748 ms) : 0, 108748
Profiling [candidate] (108.748 ms) : 0, 108748
LoadParameters
See matching parameters
SummaryFound 1 performance improvements and 2 performance regressions! Performance is the same for 9 metrics, 12 unstable metrics.
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~6556191861, baseline=1.53.0-SNAPSHOT~5ed2f5961b
dateFormat X
axisFormat %s
section baseline
no_agent (4.364 ms) : 4308, 4419
. : milestone, 4364,
iast (9.222 ms) : 9070, 9374
. : milestone, 9222,
iast_FULL (13.891 ms) : 13611, 14170
. : milestone, 13891,
iast_GLOBAL (9.83 ms) : 9660, 10000
. : milestone, 9830,
profiling (8.554 ms) : 8422, 8685
. : milestone, 8554,
tracing (7.546 ms) : 7438, 7653
. : milestone, 7546,
section candidate
no_agent (4.366 ms) : 4313, 4418
. : milestone, 4366,
iast (9.226 ms) : 9071, 9380
. : milestone, 9226,
iast_FULL (13.669 ms) : 13400, 13938
. : milestone, 13669,
iast_GLOBAL (10.093 ms) : 9919, 10267
. : milestone, 10093,
profiling (9.106 ms) : 8956, 9256
. : milestone, 9106,
tracing (7.57 ms) : 7464, 7676
. : milestone, 7570,
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~6556191861, baseline=1.53.0-SNAPSHOT~5ed2f5961b
dateFormat X
axisFormat %s
section baseline
no_agent (37.088 ms) : 36793, 37383
. : milestone, 37088,
appsec (45.318 ms) : 44923, 45713
. : milestone, 45318,
code_origins (44.15 ms) : 43783, 44518
. : milestone, 44150,
iast (46.926 ms) : 46525, 47327
. : milestone, 46926,
profiling (52.487 ms) : 51959, 53015
. : milestone, 52487,
tracing (44.141 ms) : 43765, 44517
. : milestone, 44141,
section candidate
no_agent (36.684 ms) : 36388, 36980
. : milestone, 36684,
appsec (46.446 ms) : 46021, 46872
. : milestone, 46446,
code_origins (45.622 ms) : 45217, 46026
. : milestone, 45622,
iast (44.939 ms) : 44555, 45323
. : milestone, 44939,
profiling (51.743 ms) : 51220, 52266
. : milestone, 51743,
tracing (43.389 ms) : 43025, 43754
. : milestone, 43389,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics. Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~6556191861, baseline=1.53.0-SNAPSHOT~5ed2f5961b
dateFormat X
axisFormat %s
section baseline
no_agent (15.538 s) : 15538000, 15538000
. : milestone, 15538000,
appsec (14.893 s) : 14893000, 14893000
. : milestone, 14893000,
iast (18.441 s) : 18441000, 18441000
. : milestone, 18441000,
iast_GLOBAL (17.9 s) : 17900000, 17900000
. : milestone, 17900000,
profiling (15.224 s) : 15224000, 15224000
. : milestone, 15224000,
tracing (14.906 s) : 14906000, 14906000
. : milestone, 14906000,
section candidate
no_agent (15.335 s) : 15335000, 15335000
. : milestone, 15335000,
appsec (14.876 s) : 14876000, 14876000
. : milestone, 14876000,
iast (18.596 s) : 18596000, 18596000
. : milestone, 18596000,
iast_GLOBAL (18.23 s) : 18230000, 18230000
. : milestone, 18230000,
profiling (15.728 s) : 15728000, 15728000
. : milestone, 15728000,
tracing (15.096 s) : 15096000, 15096000
. : milestone, 15096000,
Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~6556191861, baseline=1.53.0-SNAPSHOT~5ed2f5961b
dateFormat X
axisFormat %s
section baseline
no_agent (1.483 ms) : 1471, 1494
. : milestone, 1483,
appsec (2.495 ms) : 2441, 2549
. : milestone, 2495,
iast (2.213 ms) : 2150, 2275
. : milestone, 2213,
iast_GLOBAL (2.254 ms) : 2190, 2317
. : milestone, 2254,
profiling (2.052 ms) : 2002, 2103
. : milestone, 2052,
tracing (2.023 ms) : 1974, 2071
. : milestone, 2023,
section candidate
no_agent (1.484 ms) : 1472, 1496
. : milestone, 1484,
appsec (3.678 ms) : 3462, 3894
. : milestone, 3678,
iast (2.21 ms) : 2147, 2273
. : milestone, 2210,
iast_GLOBAL (2.241 ms) : 2178, 2304
. : milestone, 2241,
profiling (2.054 ms) : 2003, 2104
. : milestone, 2054,
tracing (2.022 ms) : 1974, 2071
. : milestone, 2022,
|
sarahchen6
approved these changes
Aug 6, 2025
Contributor
sarahchen6
left a comment
sarahchen6
left a comment
There was a problem hiding this comment.
Strange.. I wonder if anything depends on the "directory" existing
mcculls
approved these changes
Aug 6, 2025
Contributor
mcculls
left a comment
mcculls
left a comment
There was a problem hiding this comment.
is it only dd-smoke-tests/play-2.8 that's affected? should we proactively apply this to all the play smoke-tests?
Contributor
Author
@mcculls For now I see only play-2.8 affected. In case I will see other modules failed, I will apply same fix. |
AlexeyKuznetsov-DD
requested review from
TonyCTHsu
and removed request for
a team
Contributor
AlexeyKuznetsov-DD
changed the title
Contributor
@AlexeyKuznetsov-DD Could that be caused by the Also, I believe this plugin can be updated. |
bric3
approved these changes
Aug 7, 2025
Contributor
bric3
left a comment
bric3
left a comment
There was a problem hiding this comment.
LGTM, I left a few other suggestions, feel free to apply them.
Comment on lines
+80
to
+91
| tasks.register('fixPlayRoutesDirectory') { | ||
| description = 'Deletes routes path if it is a file instead of directory' | ||
| group = 'build cleanup' | ||
|
|
||
| doFirst { | ||
| def routesPath = layout.buildDirectory.dir('src/play/routes').get().asFile | ||
| if (routesPath.exists() && !routesPath.isDirectory()) { | ||
| logger.lifecycle("Removing file that blocks routes directory: ${routesPath}") | ||
| routesPath.delete() | ||
| } | ||
| } | ||
| } |
Contributor
There was a problem hiding this comment.
suggestion: I suggest using project.delete(routesPath)
Also, declaring the input outputs.
Suggested change
| tasks.register('fixPlayRoutesDirectory') { | |
| description = 'Deletes routes path if it is a file instead of directory' | |
| group = 'build cleanup' | |
| doFirst { | |
| def routesPath = layout.buildDirectory.dir('src/play/routes').get().asFile | |
| if (routesPath.exists() && !routesPath.isDirectory()) { | |
| logger.lifecycle("Removing file that blocks routes directory: ${routesPath}") | |
| routesPath.delete() | |
| } | |
| } | |
| } | |
| tasks.register('fixPlayRoutesDirectory') { | |
| description = 'Deletes routes path if it is a file instead of directory' | |
| group = 'build cleanup' | |
| def routesPath = layout.buildDirectory.dir('src/play/routes') | |
| input.dir(routesPath) | |
| destroyables.register(routesPath) | |
| doFirst { | |
| def routesPathFile = routesPath.get().asFile | |
| if (routesPathFile.exists() && !routesPathFile.isDirectory()) { | |
| logger.lifecycle("Removing file that blocks routes directory: ${routesPathFile}") | |
| project.delete(routesPathFile) | |
| } | |
| } | |
| } |
Contributor
Author
There was a problem hiding this comment.
Applied, but Gradle complains Task :dd-smoke-tests:play-2.8:fixPlayRoutesDirectory has both inputs and destroyables defined. A task can define either inputs or destroyables, but not both.
So I keep destroyables only.
Comment on lines
+93
to
+94
| tasks.named('compileScala') { | ||
| dependsOn tasks.named('fixPlayRoutesDirectory') |
Contributor
There was a problem hiding this comment.
suggestion: I suggest instead to declare the routes path as input rather than using dependsOn. That said I don;t know the behavior of this when using destroyables, I believe this work the same way.
FYI using dependsOn is considered bad practice, instead wiring should use task input / output declarations with providers, Gradle identifies the needed task to run.
That's also why tasks with overlapping outputs are very bad. All of that contributes to the ability of gradle to be incremental, and have reproducible builds.
I haven't looked at the failure, but I saw there's a compileRoutes, maybe this should be this task rather than compileScala ? But unsure at this point.
Contributor
Author
There was a problem hiding this comment.
I removed dependsOn. Do you think current script with destroyables declared would work properly?
Contributor
There was a problem hiding this comment.
I think so. I believe this can be checked by looking at task status with verbose console.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What Does This Do
Fix for flaky test:
Source directory '/dd-smoke-tests/play-2.8/build/src/play/routes' is not a directory.Motivation
Smoke test for play 2.8 is failing very often on GitLab.
Additional Notes
After investigation and reading Gradle sources I was not able to find a real root cause, but it looks like somehow a file created instead of folder somehow (maybe GitLab CI is doing that?).
But from Gradle sources it is clear,
that exception will be thrown only if file exists: