Randomize tests and run them in parallel

Document optional test suites

Author: kdeldycke

CHANGES.rst

@@ -13,6 +13,10 @@ Unreleased
     ``semver.Version``. :issue:`3298` :pr:`3299`
 -   Fix pager test pollution under parallel execution by using pytest's
     ``tmp_path`` fixture instead of a shared temporary file path. :pr:`3238`
+-   Add optional randomized parallel test execution using ``pytest-randomly`` and
+    ``pytest-xdist`` to detect test pollution and race conditions. :pr:`3151`
+-   Add contributor documentation for running stress tests, randomized
+    parallel tests, and Flask smoke tests. :pr:`3151` :pr:`3177`
 
 Version 8.3.2
 -------------

docs/contributing.md

@@ -0,0 +1,21 @@
+# Contributing
+
+This is a quick reference for Click-specific development tasks. For setting up the development environment and the general contribution workflow, see the Pallets [quick reference](https://palletsprojects.com/contributing/quick/) and the [detailed contributing guide](https://palletsprojects.com/contributing/).
+
+## Extra Test Environments
+
+Click includes some extra test environments:
+
+-   `tox r -e stress` runs stress tests for race conditions in Click's test runner.
+
+    ```shell-session
+    $ tox r -e stress
+    ```
+
+-   `tox r -e random` runs tests in parallel in a random order to detect test pollution.
+
+    ```shell-session
+    $ tox r -e random
+    ```
+
+-   A CI workflow (`.github/workflows/test-flask.yaml`) runs Flask's test suite to catch downstream regressions.

docs/index.rst

@@ -133,13 +133,12 @@ About Project
 
 * `Version Policy <https://palletsprojects.com/versions>`_
 
-* `Contributing <https://palletsprojects.com/contributing/>`_
-
 * `Donate <https://palletsprojects.com/donate>`_
 
 .. toctree::
    :maxdepth: 1
 
+   contributing
    contrib
    license
    changes

pyproject.toml

@@ -51,6 +51,11 @@ pre-commit = [
 tests = [
     "pytest",
 ]
+tests-random = [
+    "pytest",
+    "pytest-randomly",
+    "pytest-xdist",
+]
 typing = [
     "mypy",
     "pyright",
@@ -165,6 +170,15 @@ commands = [[
     {replace = "posargs", default = [], extend = true},
 ]]
 
+[tool.tox.env.random]
+description = "randomized parallel tests to detect test pollution and race conditions"
+dependency_groups = ["tests-random"]
+commands = [[
+    "pytest", "-v", "--tb=short", "--numprocesses=auto",
+    "--basetemp={env_tmp_dir}",
+    {replace = "posargs", default = [], extend = true},
+]]
+
 [tool.tox.env.stress]
 description = "stress tests for stream lifecycle race conditions"
 commands = [[