chore: tests speed improvements#1636
Conversation
|
This is a great reduction in time for the tests to run with a minor change, good spotting |
|
Though just to point out that the resources to run the tests is the same before. Resources being the number of shards running multiplied by time. So before one pipeline run would take 40 min, but for 25 min you had most of the shards free to run on other things. But now the tests take 24 min, but 99% of that time its consuming all of the shards. Which means it takes the next pipeline longer to start running. Like we aren't getting performance for free. Still a good change but fundamentally we need faster tests or more resources. |
356efc4 to
bda3587
Compare
|
For this PR to complete, the branch protection settings for develop will need to be updated by someone who has access. At the moment, the PR is waiting on shards that no longer exist and so it wont be able to be merged. |
mutambaraf
left a comment
mutambaraf
left a comment
There was a problem hiding this comment.
Looks good just a comment and also we may also need to add (letter q) to match questions.cy.ts
Well spotted! Looks like that's always been missing. I've widened the "rs" start to include files beginning with q and I've renamed it to "qs". |
5 participants
Description
The test shards are currently split alphabetically to take advantage of the ability to run shards in parallel. However, further splitting can be made based on test suite size.
The changes in this PR separate out some long running suites into their own shard, whilst still keeping the generic alphabet based shards so that any new tests are picked up.
Motivation and Context
The tests previously took about 40+ minutes to run. This change typically gets them under 30 minutes.