diff --git a/doc/api/test.md b/doc/api/test.md index 40fb08d0d5b181..1fd109b087b812 100644 --- a/doc/api/test.md +++ b/doc/api/test.md @@ -275,6 +275,51 @@ it.todo('should do the thing', { expectFailure: true }, () => { }); ``` +## Flaky tests + +This flag causes a test or suite to be re-run a number of times until it +either passes or has not passed after the final re-try. + +When `flaky` is `true`, the test harness re-tries the test up to the default +number of times (20), inclusive. + +When `flaky` is a positive integer, the test harness re-tries the test up to +the specified number of times, inclusive. + +When `flaky` is falsy (the default), the test harness does not re-try the test. + +When both a suite and an included test specify the `flaky` flag, the +test's `flaky` value wins. + +```js +it.flaky('should do something', () => { + // This test will be retried up to 20 times if it fails +}); + +it('may take several times', { flaky: true }, () => { + // Also retries up to 20 times +}); + +it('may also take several times', { flaky: 5 }, () => { + // Retries up to 5 times +}); + +describe.flaky('flaky suite', () => { + it('inherits flaky from suite', () => { + // Retried up to 20 times (inherited from suite) + }); + + it('not flaky', { flaky: false }, () => { + // Not retried, overrides suite setting + }); +}); +``` + +When a test marked `flaky` passes after retries, the number of re-tries taken +is reported with that test. + +`skip` and `todo` take precedence over `flaky`. + ## `describe()` and `it()` aliases Suites and tests can also be written using the `describe()` and `it()` @@ -1649,6 +1694,16 @@ added: Shorthand for marking a suite as `only`. This is the same as [`suite([name], { only: true }[, fn])`][suite options]. +## `suite.flaky([name][, options][, fn])` + + + +Shorthand for marking a suite as flaky. This is the same as +[`suite([name], { flaky: true }[, fn])`][suite options]. + ## `test([name][, options][, fn])` + +Shorthand for marking a test as flaky, +same as [`test([name], { flaky: true }[, fn])`][it options]. + ## `describe([name][, options][, fn])` Alias for [`suite()`][]. @@ -1782,6 +1852,16 @@ added: Shorthand for marking a suite as `only`. This is the same as [`describe([name], { only: true }[, fn])`][describe options]. +## `describe.flaky([name][, options][, fn])` + + + +Shorthand for marking a suite as flaky. This is the same as +[`describe([name], { flaky: true }[, fn])`][describe options]. + ## `it([name][, options][, fn])` + +Shorthand for marking a test as flaky, +same as [`it([name], { flaky: true }[, fn])`][it options]. + ## `before([fn][, options])`