Problem

The majority of the time, I like running my tests with randomize? true. Rarely, I will have a case where running tests for a specific test suite or namespace would be beneficial. Typically, this is because an earlier test(s) do an expensive, mutative operation that I do not want to redo. This might feel close to a :once fixture. However, there can be an important difference -- the expensive operation is the thing we actually want to test. As such, a :once fixture does not feel like a good fit.

Proposal

I propose adding support for setting :kaocha.plugin.randomize/randomize? on a test suite map and namespace.

Suggested implementation path

See relevant code. If the test-plan has :kaocha.plugin.randomize/randomize? set as a top-level key or in the [:kaocha.testable/meta :kaocha.plugin.randomize/randomize?] path, do not sort the :kaocha.test-plan/tests, but still continue the recursion, allowing the toggle to be on/off at any level.

If the above sounds good, I can provide a PR.

Alternatives

1. A harsh alternative is to set :kaocha.plugin.randomize/randomize? false directly in tests.edn for all tests that use that config. This is not ideal because most often you want randomize? true.
2. Always pass --no-randomize to the CLI when running the tests. This is not ideal because it requires the caller to know they must pass that flag for tests to be idempotent.
3. Execute the expensive operation multiple times. This is, perhaps, the cleanest path, but costs us (potentially significantly) extra time when running tests locally and on CI.