DOC: Improve testing documentation

Rordered topics and made descriptions more concise.

Author: timhoffm

doc/devel/testing.rst

@@ -100,34 +100,45 @@ to the folder where the baseline test images are stored. The triage tool require
 :ref:`QT <backend_dependencies>` is installed.
 
 
-Writing a simple test
----------------------
+Writing tests
+-------------
+Tests are located in :file:`lib/matplotlib/tests`. They are organized to mirror
+the structure of the main code in :file:`lib/matplotlib`. For example, tests for
+the ``mathtext.py`` module are in :file:`lib/matplotlib/tests/test_mathtext.py`.
 
-Many elements of Matplotlib can be tested using standard tests. For
-example, here is a test from :file:`matplotlib/tests/test_basic.py`::
+Naming follows standard pytest conventions:
 
-  def test_simple():
-      """
-      very simple example test
-      """
-      assert 1 + 1 == 2
+- files begin with ``"test_"``
+- test functions begin with ``"test_"``
+- test class begin ``"Test"``.
 
-Pytest determines which functions are tests by searching for files whose names
-begin with ``"test_"`` and then within those files for functions beginning with
-``"test"`` or classes beginning with ``"Test"``.
+We prefer simple test functions, but test classes are also acceptable.
+Test function names should be descriptive of what they are testing, and long names
+like ``test_to_rgba_array_accepts_color_alpha_tuple_with_multiple_colors()`` are
+perfectly fine.
 
-Some tests have internal side effects that need to be cleaned up after their
-execution (such as created figures or modified `.rcParams`). The pytest fixture
-``matplotlib.testing.conftest.mpl_test_settings`` will automatically clean
-these up; there is no need to do anything further.
+Simple unit tests
+^^^^^^^^^^^^^^^^^
 
-Random data in tests
---------------------
+Many elements of Matplotlib can be tested using simple unit tests, e.g. ::
 
-Random data is a very convenient way to generate data for examples,
-however the randomness is problematic for testing (as the tests
-must be deterministic!).  To work around this set the seed in each test.
-For numpy's default random number generator use::
+  def test_to_rgba_explicit_alpha_overrides_tuple_alpha():
+      assert mcolors.to_rgba(('red', 0.1), alpha=0.9) == (1, 0, 0, 0.9)
+
+Data in tests
+^^^^^^^^^^^^^
+Try to use minimal explicit data. Often you can use explicit values like
+``[1, 2, 3]``, ``range(5)`` or ``np.arange(5)``. This is cheap and
+makes the test more readable.
+
+When you need more and non-trivial data, generate it programmatically, e.g. ::
+
+   x = np.linspace(0, 2*np.pi, 101)
+   y = 2 * np.sin(x) + 1
+
+Use random numbers only, if an algorihmical way to generate the data is not
+possible or too cumbersome. In this case, set the seed to a fixed value to make
+the test deterministic. For numpy's default random number generator use ::
 
   import numpy as np
   rng = np.random.default_rng(19680801)
@@ -136,10 +147,52 @@ and then use ``rng`` when generating the random numbers.
 
 The seed is :ref:`John Hunter's <project_history>` birthday.
 
+Test cleanup
+^^^^^^^^^^^^
+We often need figures or modify `.rcParams` to test some functionality. Cleanup
+of such side effects is handled automatically through the the pytest fixture
+``matplotlib.testing.conftest.mpl_test_settings``.
+
+In particular, you don't need to call ``plt.close()``.
+
+Testing with figures and Axes
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+When you need figures and/or Axes, create them through the standard methods
+(``plt.figure()``, ``plt.subplots()``, etc.).
+
+Creating figures and Axes is rather expensive (>100ms). Only create as many as you need for
+the test, and reuse them if possible. It is perfectly fine to do multiple operations
+and asserts in one test, e.g.::
+
+  def test_stackplot_facecolor():
+      # Test that facecolors are properly passed and take precedence over colors parameter
+      x = np.linspace(0, 10, 10)
+      y1 = 1.0 * x
+      y2 = 2.0 * x + 1
+
+      fig, ax = plt.subplots()
+
+      colls = ax.stackplot(x, y1, y2, facecolor=['r', 'b'], colors=['c', 'm'])
+      for coll, fcolor in zip(colls, facecolors):
+          assert mcolors.same_color(coll.get_facecolor(), fcolor)
+
+     # Plural alias should also work
+      colls = ax.stackplot(x, y1, y2, facecolors=['r', 'b'], colors=['c', 'm'])
+      for coll, fcolor in zip(colls, facecolors):
+          assert mcolors.same_color(coll.get_facecolor(), fcolor)
+
+Assert values rather than visual results when feasible. This is clearer,
+less expensive and less fragile than comparing images, e.g. ::
+
+  def test_savefig_preserve_layout_engine():
+      fig = plt.figure(layout='compressed')
+      fig.savefig(io.BytesIO(), bbox_inches='tight')
+      assert fig.get_layout_engine()._compress
+
 .. _image-comparison:
 
-Writing an image comparison test
---------------------------------
+Testing with reference images
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Writing an image-based test is only slightly more difficult than a simple
 test. The main consideration is that you must specify the "baseline", or
@@ -180,9 +233,8 @@ texts (labels, tick labels, etc) are not really part of what is tested, use the
 will lead to smaller figures and reduce possible issues with font mismatch on
 different platforms.
 
-
-Compare two methods of creating an image
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Testing by comparing two methods to create an image
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Baseline images take a lot of space in the Matplotlib repository.
 An alternative approach for image comparison tests is to use the
@@ -228,13 +280,6 @@ See the documentation of `~matplotlib.testing.decorators.image_comparison` and
 `~matplotlib.testing.decorators.check_figures_equal` for additional information
 about their use.
 
-Creating a new module in matplotlib.tests
------------------------------------------
-
-We try to keep the tests categorized by the primary module they are
-testing.  For example, the tests related to the ``mathtext.py`` module
-are in ``test_mathtext.py``.
-
 Using GitHub Actions for CI
 ---------------------------