Skip to content

Commit b9cfbe8

Browse files
committed
update readme
1 parent 1b7a762 commit b9cfbe8

1 file changed

Lines changed: 22 additions & 33 deletions

File tree

README.md

Lines changed: 22 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -9,28 +9,26 @@ One major difference between unit tests in programming and
99
network tests is the definition of what a test actually is.
1010
In programming, unit tests normally focus on testing edge cases,
1111
since the amount of non-edge cases is not definable.
12-
In the network testing domain, tests are less about edge cases, but more about testing existing configurations with
12+
In the network testing domain, tests are less about edge cases, but more about testing existing network states with
1313
pre-defined test cases. Such a single test case might be "can host A reach neighbors X, Y, Z?" on many different devices.
1414
This is what nuts tries to achieve:
15-
Apply test cases based on your pre-defined network topology to your actual network and have the tests confirm the correct configuration.
15+
Apply test cases based on your pre-defined network topology to your actual network and have the tests confirm the correct state.
1616

1717
The project relies on the [pytest framework](https://docs.pytest.org/) to setup and execute the tests.
1818
Nuts itself is written as a custom pytest plugin. In the background, [nornir](https://nornir.readthedocs.io/)
1919
executes specific network tasks for the actual tests.
2020

21-
Additionally, nuts treats the test definition and the so-called test bundle as separate entities.
21+
Additionally, nuts treats the test definition and the so-called test bundle as separate entities. The *test definition* is modeled as a custom `pytest.Class`, and a predefined set of test definitions can be found in the module `base_tests`. New test definitions can be added easily by the user of the plugin.
2222

23-
The test definition is modelled as a custom `pytest.Class`, and a predefined set of test definitions can be found in the module `base_tests`. New test definitions can be added easily by the user of the plugin.
24-
25-
The test bundle is a file that is parsed by pytest. The file provides data on the actual network configuration and describes which test definitions should be collected and executed by pytest.
23+
The *test bundle* is a file that is parsed by pytest. The file provides data on the desired network state and describes which test definitions should be collected and executed by pytest.
2624
The structure of the test bundle should enable people without in-depth python knowledge to add new test bundles or update existing ones to reflect changes in the network.
2725

28-
While the readme here is only a short overview, find the [documentation of Nuts on readthedocs](https://nuts.readthedocs.io/en/latest/).
26+
While the readme here is only a short overview, find the [documentation of nuts on readthedocs](https://nuts.readthedocs.io/en/latest/).
2927

3028
## Test bundle structure
3129

3230
Currently only yaml files are supported as test bundles,
33-
but other sources such as other file formats or database entries can be considered in later versions.
31+
but other sources such as other file formats or database entries can be considered in later nuts versions.
3432

3533
Each test bundle contains the following structure:
3634
```yaml
@@ -52,11 +50,11 @@ Note that currently every test in this class will be executed.
5250
`label`: Additional identifier that can be used to distinguish between multiple occurrences of the same
5351
test class in a test bundle.
5452

55-
`test_execution`: Data that is exposed as part of the `nuts_parameters` property (explanation see below).
53+
`test_execution`: Data that is exposed as part of the `nuts_parameters` property.
5654
By convention this contains additional information that is passed directly to the nornir task in the background.
5755
Therefore the key-value pairs must be consistent with the key-value pairs of the specific nornir task.
5856
As an example, the test definition `napalm_ping.py` calls a nornir task to execute napalm's ping-command.
59-
This allows the additional `max_drop` parameter in `test execution`, since it is in turn pre-defined by napalm.
57+
This allows the additional `max_drop` parameter in `test_execution`, since it is in turn pre-defined by napalm.
6058

6159
`test_data`: Data that is used to parametrize the tests in the test class which have the `pytest.mark.nuts` annotation. It is additionally part of the `nuts_parameters` property.
6260

@@ -69,17 +67,17 @@ This example creates three different tests, one for each entry in the `test_data
6967
- test_module: nuts.base_tests.netmiko_cdp_neighbors
7068
test_class: TestNetmikoCdpNeighbors
7169
test_data:
72-
- source: R1
70+
- host: R1
7371
local_port: GigabitEthernet3
7472
destination_host: R2
7573
management_ip: 172.16.12.2
7674
remote_port: GigabitEthernet2
77-
- source: R1
75+
- host: R1
7876
local_port: GigabitEthernet4
7977
destination_host: R3
8078
management_ip: 172.16.13.3
8179
remote_port: GigabitEthernet2
82-
- source: R2
80+
- host: R2
8381
local_port: GigabitEthernet2
8482
destination_host: R1
8583
management_ip: 172.16.12.1
@@ -90,43 +88,36 @@ This example creates three different tests, one for each entry in the `test_data
9088
## Manual installation instructions
9189
Nuts uses [poetry](https://python-poetry.org/) as a dependency manager.
9290

93-
1. [Install poetry](https://python-poetry.org/docs/#installation)
91+
1. [Install poetry](https://python-poetry.org/docs/#installation).
9492
2. Clone this repository.
9593
3. Run `$ poetry install`
9694

9795
## Technical Overview
9896

99-
### Nuts custom marker
100-
101-
During test collection, the custom pytest marker "nuts" uses the data that has been defined in the test bundle mentioned above.
102-
This annotation is a wrapper around the `pytest.mark.parametrize` annotation and allows the plugin to use the data entries
103-
from the test bundle. For each entry in the `test_data` section of the test bundle, the custom marker generates a single test case.
104-
Each entry is a dictionary, but `pytest.mark.parametrize` expects a list of n-tuples as input.
105-
The plugin therefore transforms those dictionary entries from `test_data` into tuples.
106-
This transformation is currently fixed, but more flexibility is very likely to come at a later stage.
97+
When nuts is executed, pytest converts the test bundles (the yaml files) into tests. During test collection, the custom pytest marker `nuts` uses the data that has been defined in the test bundle mentioned above.
98+
This annotation is a wrapper around the `pytest.mark.parametrize` annotation and allows the plugin to use the data entries from the test bundle. For each entry in the `test_data` section of the test bundle, the custom marker generates a single test case. To achieve this, the plugin transforms the entries into n-tuples, since `pytest.mark.parametrize` expects a list of n-tuples as input.
10799

108100
The custom nuts marker takes two arguments: The first argument of the annotation determines the required fields.
109101
For each entry in `test_data` these fields are extracted and transformed to a tuple considering the correct order.
110102
If any of these fields are not present in an entry of `test_data`, the corresponding test case will be skipped.
111-
The second argument determines optional fields that can also be used in a test case as well - non-present values are passed into the function as `None`.
103+
A second argument determines optional fields that can also be used in a test case as well - non-present values are passed into the function as `None`.
112104

113105
#### Example of a test class with custom marker
114106

115107
```python
116108
@pytest.mark.usefixtures("check_nuts_result") # see below
117-
class TestNetmikoCdpNeighbors:
118-
@pytest.mark.nuts("host, remote_host, local_port, management_ip", "management_ip")
109+
class TestNetmikoCdpNeighbors:
110+
@pytest.mark.nuts("host,remote_host,local_port")
119111
def test_local_port(self, single_result, remote_host, local_port):
120-
assert single_result.result[remote_host]["local_port"] == local_port
112+
assert single_result.result[remote_host]["local_port"] == local_port
121113
```
122114

123-
This test run of CDP neighbors checks the local port.
115+
This test-run of CDP neighbors checks the local port.
124116

125117
Before each test evaluation, the fixture `@pytest.mark.usefixtures("check_nuts_result")` checks the result of the network information that has been gathered by nornir in the background: It asserts that that no exception was thrown while doing so.
126118

127119
The required fields are `host`, `remote_host` and `local_port` - they must be present in the custom marker,
128120
but also be provided as argument to the test method itself.
129-
`management_ip` is an optional field and not necessary for the test - an entry in `test_data` is not required to have it.
130121

131122
`single_result` uses the `host` field and provides the result that has been processed via the specific context of a test.
132123

@@ -137,15 +128,13 @@ Currently, the predefined test classes use [nornir](https://nornir.readthedocs.i
137128
with the network devices, therefore the test classes derive all from a more specific `NornirNutsContext`,
138129
which provides a nornir instance and nornir-specific helpers.
139130

140-
This context class is then integrated into pytest fixtures. It is passed on to another fixture called `sincle_result` that returns a per-host result of one test.
141-
142131
## Development
143132

144133
Nuts is essentially designed as a pytest-plugin and it is possible to add your own, self-written test classes.
145134
A dev documentation on how to write your own test classes is planned for the future.
146-
Until then, please read the regular [documentation of Nuts](https://nuts.readthedocs.io/en/latest/) on how to use it.
135+
Until then, please read the regular [documentation of nuts](https://nuts.readthedocs.io/en/latest/) on how to use it.
147136

148137
# Thanks
149138

150-
* [Matthias Gabriel](https://github.com/MatthiasGabriel), who laid the foundations of NUTS.
151-
* [Florian Bruhin](https://github.com/The-Compiler) for invaluable feedback.
139+
* [Matthias Gabriel](https://github.com/MatthiasGabriel), who laid the foundations of nuts.
140+
* [Florian Bruhin](https://github.com/The-Compiler) for invaluable feedback.

0 commit comments

Comments
 (0)