-
Notifications
You must be signed in to change notification settings - Fork 1
Specification
The universal access point for testing is the $TEST_SUITE global variable.
$TEST_SUITE = []
A test framework need only add compliant test objects to the $TEST_SUITE
global array. Ruby Test will iterate through these objects. If a test object
responds to #call, it is run as a test procedure. If it responds to #each
it is iterated over as a test case, where each entry goes though the same process.
All test objects must respond to #to_s so their description can be used in
test reports.
Any raised exception that responds to #assertion? in the affirmative is taken
to be a failed assertion rather than simply an error. Ruby Test extends the
Exception class to support this method for all exceptions.
A test framework may raise a NotImplementedError to have a test recorded
as pending --a todo item to remind the developer of tests that still
need to be written. The NotImplementedError is a standard Ruby exception
and a subclass of ScriptError.
If the NotImplmentedError responds in the affirmative to #assertion? then
the test is taken to be a purposeful omission, rather than simply pending.
If the return value of #to_s is a multi-line string, the first line is
taken to be the label or summary of the test object. The remaining
lines are taken to be additional detail. A report format might only
show the detail if the verbose option is set.
A test case that responds to #ordered? indicates if its tests must be run
in order. If false, which is the default, then the test runner can randomize
the order for more rigorous testing. But if true, then the tests will always
be run in the order given. Also, if ordered, a case's test units cannot be
selected or filtered independent of one another.
A test object may provide a #type, which can be used to characterize the
type of test case or test unit. For example, RSpec might return "describe"
as the type of a test case and "it" as the type of a test unit, where as
Cucumber would use "Feature" and "Scenario" for test cases and "Then"
for test units (depending on choices made by the implementors).
(TODO: Is "Setup" a better name for this?)
A test object can respond to #topic which can be used to provide
a description of the setup associated with a test object. The return
value of #topic is sent #to_s to ensure it is a string.
If a test object responds to #source_location it will be taken to
identify the file and line in which the the test was defined. The
return value of #source_location must be a two-element array of
[file, line].
A "unit" (as defined in the software testing literature) is the smallest piece of code software that can be tested in isolation. In Ruby software, being object-oriented, these are classes, metaclasses and their corresponding instances, methods and functions (class methods).
A test object may respond to #unit to identify the particular
component of the software being tested. The return value must be the
the full name of a class, module, or method. Methods must be represented
with fully qualified names, e.g. Foo::Bar#baz and Foo::Bar.baz for an
instance method or class method, respectively.
A test object can provide #tags which must return a one-word string or
array of one-word strings. The test runner can use the tags to limit the
particular tests to be run.
If any test object responds to #skip? it indicates that the test unit or
test case is to be skipped and not tested. If the return value is a string
the string is take to be a description of the reason it is being skipped.
It may or may not be mentioned in the test output depending on the reporter
used and the verbosity mode at run-time.
Skipping test differs from omission in that omission test are still run. They simply raise an error at the point of omission. Unlike skipped tests, this allows omitted tests to have active partial implementations.