Rename example to test case in many places

README.md

@@ -21,7 +21,7 @@ def test_matches_builtin(ls):
     assert sorted(ls) == my_sort(ls)
 ```
 
-This randomized testing can catch bugs and edge cases that you didn't think of and wouldn't have found. In addition, when Hypothesis does find a bug, it doesn't just report any failing example — it reports the simplest possible one. This makes property-based tests a powerful tool for debugging, as well as testing.
+This randomized testing can catch bugs and edge cases that you didn't think of and wouldn't have found. In addition, when Hypothesis does find a bug, it doesn't just report any failing test case — it reports the simplest possible one. This makes property-based tests a powerful tool for debugging, as well as testing.
 
 For instance,
 
@@ -30,10 +30,10 @@ def my_sort(ls):
     return sorted(set(ls))
 ```
 
-fails with the simplest possible failing example:
+fails with the simplest possible failing test case:
 
 ```
-Falsifying example: test_matches_builtin(ls=[0, 0])
+Failing test case: test_matches_builtin(ls=[0, 0])
 ```
 
 ### Installation

hypothesis-python/RELEASE.rst

@@ -0,0 +1,7 @@
+RELEASE_TYPE: minor
+
+Hypothesis has historically referred to the single execution of a test function as an "example". However, we have in recent years tended to call a single execution a "test case" instead, which we think is a more precise and less overloaded term.
+
+This release updates user-facing documentation, log messages, and error messages to use the "test case" terminology.
+
+This is not a breaking change, as we have intentionally not changed any code APIs. For example, |settings.max_examples| has not been changed.

hypothesis-python/docs/quickstart.rst

@@ -70,7 +70,7 @@ This test will clearly fail, which can be confirmed by running ``pytest example.
         def test_integers(n):
     >       assert n < 50
     E       assert 50 < 50
-    E       Falsifying example: test_integers(
+    E       Failing test case: test_integers(
     E           n=50,
     E       )
 

hypothesis-python/docs/reference/api.rst

@@ -121,7 +121,7 @@ For instance, in the latter case, you would see output like:
 
     - during generate phase (0.09 seconds):
         - Typical runtimes: < 1ms, ~ 59% in data generation
-        - 100 passing examples, 0 failing examples, 32 invalid examples
+        - 100 passing test cases, 0 failing test cases, 32 invalid test cases
         - Events:
           * 54.55%, Retried draw from integers().filter(lambda x: x % 2 == 0) to satisfy filter
           * 31.06%, i mod 3 = 2

hypothesis-python/docs/tutorial/adding-notes.rst

@@ -5,7 +5,7 @@ When a test fails, Hypothesis will normally print output that looks like this:
 
 .. code::
 
-    Falsifying example: test_a_thing(x=1, y="foo")
+    Failing test case: test_a_thing(x=1, y="foo")
 
 Sometimes you want to add some additional information to a failure, such as the output of some intermediate step in your test. The |note| function lets you do this:
 
@@ -24,11 +24,11 @@ Sometimes you want to add some additional information to a failure, such as the
     ... except AssertionError:
     ...     print("ls != ls2")
     ...
-    Falsifying example: test_shuffle_is_noop(ls=[0, 1], r=RandomWithSeed(1))
+    Failing test case: test_shuffle_is_noop(ls=[0, 1], r=RandomWithSeed(1))
     Shuffle: [1, 0]
     ls != ls2
 
-|note| is like a print statement that gets attached to the falsifying example reported by Hypothesis. It's also reported by :ref:`observability <observability>`, and shown for all examples (if |settings.verbosity| is set to |Verbosity.verbose| or higher).
+|note| is like a print statement that gets attached to the failing test case reported by Hypothesis. It's also reported by :ref:`observability <observability>`, and shown for all examples (if |settings.verbosity| is set to |Verbosity.verbose| or higher).
 
 .. note::
 

hypothesis-python/docs/tutorial/flaky.rst

@@ -61,7 +61,7 @@ As a result, running ``test_fails_flakily()`` will raise |FlakyFailure|. |FlakyF
 
   + Exception Group Traceback (most recent call last):
   | hypothesis.errors.FlakyFailure: Hypothesis test_fails_flakily(n=0) produces unreliable results: Falsified on the first call but did not on a subsequent one (1 sub-exception)
-  | Falsifying example: test_fails_flakily(
+  | Failing test case: test_fails_flakily(
   |     n=0,
   | )
   | Failed to reproduce exception. Expected:

hypothesis-python/docs/tutorial/replaying-failures.rst

@@ -67,7 +67,7 @@ Note that unlike examples generated by Hypothesis, examples provided using |@exa
     def test_something_with_integers(n):
         assert n < 100
 
-Hypothesis will print ``Falsifying explicit example: test_something_with_integers(n=131071)``, instead of shrinking to ``n=100``.
+Hypothesis will print ``Failing explicit example: test_something_with_integers(n=131071)``, instead of shrinking to ``n=100``.
 
 Prefer |@example| over the database for correctness
 ---------------------------------------------------
@@ -93,10 +93,10 @@ If |settings.print_blob| is set to ``True`` (the default in the ``ci`` settings
     >>> test()
 
     ...
-    Falsifying example: test(
+    Failing test case: test(
         f=nan,
     )
-    You can reproduce this example by temporarily adding @reproduce_failure('6.131.23', b'ACh/+AAAAAAAAA==') as a decorator on your test case
+    You can reproduce this test case by temporarily adding @reproduce_failure('6.131.23', b'ACh/+AAAAAAAAA==') as a decorator on your test function
 
 
 You can add this decorator to your test to reproduce the failure. This can be useful for locally replaying failures found by CI. Note that the binary blob is not stable across Hypothesis versions, so you should not leave this decorator on your tests permanently. Use |@example| with an explicit input instead.

hypothesis-python/src/_hypothesis_pytestplugin.py

@@ -405,7 +405,7 @@ def pytest_terminal_summary(terminalreporter):
                 terminalreporter.write_line(f"observations written to {fname}")
 
         if failing_examples:
-            # This must have been imported already to write the failing examples
+            # This must have been imported already to write the failing test cases
             from hypothesis.extra._patching import gc_patches, make_patch, save_patch
 
             patch = make_patch(failing_examples)
@@ -418,7 +418,7 @@ def pytest_terminal_summary(terminalreporter):
             if not _WROTE_TO:
                 terminalreporter.section("Hypothesis")
             terminalreporter.write_line(
-                f"`git apply {fname}` to add failing examples to your code."
+                f"`git apply {fname}` to add failing test cases to your code."
             )
 
     def pytest_collection_modifyitems(items):

hypothesis-python/src/hypothesis/__init__.py

@@ -12,7 +12,7 @@
 some source of data.
 
 It verifies your code against a wide range of input and minimizes any
-failing examples it finds.
+failing test cases it finds.
 """
 
 import _hypothesis_globals

hypothesis-python/src/hypothesis/_settings.py

@@ -68,12 +68,12 @@ class Verbosity(Enum):
 
     quiet = "quiet"
     """
-    Hypothesis will not print any output, not even the final falsifying example.
+    Hypothesis will not print any output, not even the final failing test case.
     """
 
     normal = "normal"
     """
-    Standard verbosity. Hypothesis will print the falsifying example, alongside
+    Standard verbosity. Hypothesis will print the failing test case, alongside
     any notes made with |note| (only for the falsfying example).
     """
 
@@ -173,7 +173,7 @@ class Phase(Enum):
     Controls whether Hypothesis attempts to explain test failures.
 
     The explain phase has two parts, each of which is best-effort - if Hypothesis
-    can't find a useful explanation, we'll just print the minimal failing example.
+    can't find a useful explanation, we'll just print the minimal failing test case.
     """
 
     @classmethod
@@ -851,18 +851,18 @@ def verbosity(self):
             ... def f(x):
             ...     assert not any(x)
             ... f()
-            Trying example: []
-            Falsifying example: [-1198601713, -67, 116, -29578]
-            Shrunk example to [-1198601713]
-            Shrunk example to [-128]
-            Shrunk example to [32]
-            Shrunk example to [1]
+            Test case: []
+            Failing test case: [-1198601713, -67, 116, -29578]
+            Shrunk test case to [-1198601713]
+            Shrunk test case to [-128]
+            Shrunk test case to [32]
+            Shrunk test case to [1]
             [1]
 
         The four levels are |Verbosity.quiet|, |Verbosity.normal|,
         |Verbosity.verbose|, and |Verbosity.debug|. |Verbosity.normal| is the
         default. For |Verbosity.quiet|, Hypothesis will not print anything out,
-        not even the final falsifying example. |Verbosity.debug| is basically
+        not even the final failing test case. |Verbosity.debug| is basically
         |Verbosity.verbose| but a bit more so. You probably don't want it.
 
         Verbosity can be passed either as a |Verbosity| enum value, or as the
@@ -889,16 +889,16 @@ def phases(self):
         Hypothesis divides tests into logically distinct phases.
 
         - |Phase.explicit|: Running explicit examples from |@example|.
-        - |Phase.reuse|: Running examples from the database which previously failed.
-        - |Phase.generate|: Generating new random examples.
-        - |Phase.target|: Mutating examples for :ref:`targeted property-based
+        - |Phase.reuse|: Running test cases from the database which previously failed.
+        - |Phase.generate|: Generating new random test cases.
+        - |Phase.target|: Mutating test cases for :ref:`targeted property-based
           testing <targeted>`. Requires |Phase.generate|.
-        - |Phase.shrink|: Shrinking failing examples.
+        - |Phase.shrink|: Shrinking failing test cases.
         - |Phase.explain|: Attempting to explain why a failure occurred.
           Requires |Phase.shrink|.
 
         The phases argument accepts a collection with any subset of these. E.g.
-        ``settings(phases=[Phase.generate, Phase.shrink])`` will generate new examples
+        ``settings(phases=[Phase.generate, Phase.shrink])`` will generate new test cases
         and shrink them, but will not run explicit examples or reuse previous failures,
         while ``settings(phases=[Phase.explicit])`` will only run explicit examples
         from |@example|.
@@ -921,10 +921,10 @@ def phases(self):
         there are no clearly suspicious lines of code, :pep:`we refuse the
         temptation to guess <20>`.
 
-        After shrinking to a minimal failing example, Hypothesis will try to find
-        parts of the example -- e.g. separate args to |@given|
+        After shrinking to a minimal failing test case, Hypothesis will try to find
+        parts of the test case -- e.g. separate args to |@given|
         -- which can vary freely without changing the result
-        of that minimal failing example. If the automated experiments run without
+        of that minimal failing test case. If the automated experiments run without
         finding a passing variation, we leave a comment in the final report:
 
         .. code-block:: python
@@ -936,7 +936,7 @@ def phases(self):
 
         Just remember that the *lack* of an explanation sometimes just means that
         Hypothesis couldn't efficiently find one, not that no explanation (or
-        simpler failing example) exists.
+        simpler failing test case) exists.
         """
 
         return self._phases
@@ -1011,8 +1011,8 @@ def deadline(self):
     @property
     def print_blob(self):
         """
-        If set to ``True``, Hypothesis will print code for failing examples that
-        can be used with |@reproduce_failure| to reproduce the failing example.
+        If set to ``True``, Hypothesis will print code for failing test cases that
+        can be used with |@reproduce_failure| to reproduce the failing test case.
 
         The default value is ``False``. If running on CI, the default is ``True`` instead.
         """

hypothesis-python/src/hypothesis/control.py

@@ -237,7 +237,7 @@ def should_note():
 
 
 def note(value: object) -> None:
-    """Report this value for the minimal failing example."""
+    """Report this value for the minimal failing test case."""
     if should_note():
         if not isinstance(value, str):
             value = pretty(value)

hypothesis-python/src/hypothesis/core.py

@@ -647,7 +647,7 @@ def execute_explicit_examples(state, wrapped_test, arguments, kwargs, original_s
                 with contextlib.suppress(StopTest):
                     empty_data.conclude_test(Status.INVALID)
             except BaseException as err:
-                # In order to support reporting of multiple failing examples, we yield
+                # In order to support reporting of multiple failing test cases, we yield
                 # each of the (report text, error) pairs we find back to the top-level
                 # runner.  This also ensures that user-facing stack traces have as few
                 # frames of Hypothesis internals as possible.
@@ -682,9 +682,9 @@ def execute_explicit_examples(state, wrapped_test, arguments, kwargs, original_s
                 break
             finally:
                 if fragments_reported:
-                    assert fragments_reported[0].startswith("Falsifying example")
+                    assert fragments_reported[0].startswith("Failing test case")
                     fragments_reported[0] = fragments_reported[0].replace(
-                        "Falsifying example", "Falsifying explicit example", 1
+                        "Failing test case", "Failing explicit example", 1
                     )
 
                 empty_data.freeze()
@@ -700,7 +700,7 @@ def execute_explicit_examples(state, wrapped_test, arguments, kwargs, original_s
                     deliver_observation(tc)
 
             if fragments_reported:
-                verbose_report(fragments_reported[0].replace("Falsifying", "Trying", 1))
+                verbose_report(fragments_reported[0].replace("Failing", "Trying", 1))
                 for f in fragments_reported[1:]:
                     verbose_report(f)
 
@@ -1055,9 +1055,9 @@ def run(data: ConjectureData) -> None:
             if print_example or current_verbosity() >= Verbosity.verbose:
                 printer = RepresentationPrinter(context=context)
                 if print_example:
-                    printer.text("Falsifying example:")
+                    printer.text("Failing test case:")
                 else:
-                    printer.text("Trying example:")
+                    printer.text("Test case:")
 
                 if self.print_given_args:
                     printer.text(" ")
@@ -1533,12 +1533,12 @@ def run_engine(self):
             finally:
                 ran_example.freeze()
                 if observability_enabled():
-                    # log our observability line for the final failing example
+                    # log our observability line for the final failing test case
                     tc = make_testcase(
                         run_start=self._start_timestamp,
                         property=self.test_identifier,
                         data=ran_example,
-                        how_generated="minimal failing example",
+                        how_generated="minimal failing test case",
                         representation=self._string_repr,
                         arguments=ran_example._observability_args,
                         timing=self._timing_features,
@@ -1550,12 +1550,12 @@ def run_engine(self):
                     deliver_observation(tc)
 
                 # Whether or not replay actually raised the exception again, we want
-                # to print the reproduce_failure decorator for the failing example.
+                # to print the reproduce_failure decorator for the failing test case.
                 if self.settings.print_blob:
                     fragments.append(
-                        "\nYou can reproduce this example by temporarily adding "
+                        "\nYou can reproduce this test case by temporarily adding "
                         f"{reproduction_decorator(falsifying_example.choices)} "
-                        "as a decorator on your test case"
+                        "as a decorator on your test function"
                     )
 
         _raise_to_user(
@@ -1611,7 +1611,7 @@ def _raise_to_user(
     errors_to_report, settings, target_lines, trailer="", *, unsound_backend=None
 ):
     """Helper function for attaching notes and grouping multiple errors."""
-    failing_prefix = "Falsifying example: "
+    failing_prefix = "Failing test case: "
     ls = []
     for error in errors_to_report:
         for note in error.fragments:
@@ -1648,14 +1648,14 @@ def _raise_to_user(
 def fake_subTest(self, msg=None, **__):
     """Monkeypatch for `unittest.TestCase.subTest` during `@given`.
 
-    If we don't patch this out, each failing example is reported as a
+    If we don't patch this out, each failing test case is reported as a
     separate failing test by the unittest test runner, which is
     obviously incorrect. We therefore replace it for the duration with
     this version.
     """
     warnings.warn(
-        "subTest per-example reporting interacts badly with Hypothesis "
-        "trying hundreds of examples, so we disable it for the duration of "
+        "subTest per-test-case reporting interacts badly with Hypothesis "
+        "trying hundreds of test cases, so we disable it for the duration of "
         "any test that uses `@given`.",
         HypothesisWarning,
         stacklevel=2,
@@ -2179,7 +2179,7 @@ def wrapped_test(*arguments, **kwargs):
                     and getattr(wrapped_test, "hypothesis_explicit_examples", ())
                 )
                 SKIP_BECAUSE_NO_EXAMPLES = unittest.SkipTest(
-                    "Hypothesis has been told to run no examples for this test."
+                    "Hypothesis has been told to run no test cases for this test."
                 )
                 if not (
                     Phase.reuse in settings.phases or Phase.generate in settings.phases

hypothesis-python/src/hypothesis/database.py

@@ -765,7 +765,7 @@ class GitHubArtifactDatabase(ExampleDatabase):
     You can use this for sharing example databases between CI runs and developers, allowing
     the latter to get read-only access to the former. This is particularly useful for
     continuous fuzzing (i.e. with `HypoFuzz <https://hypofuzz.com/>`_),
-    where the CI system can help find new failing examples through fuzzing,
+    where the CI system can help find new failing test cases through fuzzing,
     and developers can reproduce them locally without any manual effort.
 
     .. note::

hypothesis-python/src/hypothesis/extra/_patching.py

@@ -13,8 +13,8 @@
 
 Requires `hypothesis[codemods,ghostwriter]` installed, i.e. black and libcst.
 
-This module is used by Hypothesis' builtin pytest plugin for failing examples
-discovered during testing, and by HypoFuzz for _covering_ examples discovered
+This module is used by Hypothesis' builtin pytest plugin for failing test cases
+discovered during testing, and by HypoFuzz for _covering_ test cases discovered
 during fuzzing.
 """
 
@@ -98,7 +98,7 @@ def __init__(
             [m.Arg(m.SimpleString() & value_in_strip_via)],
         )
 
-        # Codemod the failing examples to Call nodes usable as decorators
+        # Codemod the failing test cases to Call nodes usable as decorators
         self.fn_examples = {
             k: tuple(
                 d