Skip to content

Commit 0a64bc6

Browse files
scetronscetron
committed
edit readme
1 parent 4db6805 commit 0a64bc6

File tree

1 file changed

+78
-57
lines changed

1 file changed

+78
-57
lines changed

‎README.md‎

Lines changed: 78 additions & 57 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,39 @@
11
# netcompare
22

3-
This library is meant to be a light-weight way to compare structured output from network devices `show` commands. `netcompare` is a python library targeted at intelligently deep diffing structured data objects of json type. In addition, `netcompare` can also provide some basic testing of key/values within a data structure.
3+
This library is meant to be a light-weight way to compare structured output. `netcompare` is a python library targeted at intelligently deep diffing structured data objects of json type.
4+
5+
`netcompare` also provides some basic testing of key/values within data structures.
46

57
The library heavily relies on [jmespath](https://jmespath.org/) for traversing the json object and finding the value(s) to be evaluated. More on that later.
68

9+
## Use Cases
10+
11+
`netcompare` is meant to be a building block used within testing frameworks. Originally devised specifically to test structured data returned from network device show commands, `netcompare` is designed to deeply and intelligently diff data structures while also providing tests for specific values in the data structure. `netcompare` evolved to be agnostic from the collection of that data and can compare and test and type of data structure.
12+
13+
With that said, it is perfectly suited to diff data gathered from network devices via show commands, Ansible playbooks, in Django based applications such as Nautobot or Netbox, and is targeted at being the 'plumbing' behind a full network automation validation solution.
14+
15+
`netcompare` was developed for checking the state of the network before and after a change; however, `netcompare` is also well suited for comparing intended state derived from a Source-of-Truth or other source of intended state.
16+
17+
The intended usage is to collect structured `show` command output before and after a change window. Prior to closing the change window, the results are compared to help determine if the change was successful as intended and if the network is in an acceptable state. The output can be stored with the change's documentation for easy reference and proof of completion.
18+
19+
With `netcompare` we've left the collection of the data up to you, to easily fit within your preferred method.
20+
21+
### Testing data structures
22+
23+
Briefly, these tests are provided to test the objects, to aide in determining the status of the data.
24+
25+
- `exact_match`: the keys and values much match between the two objects
26+
- `tolerance`: the keys must match and the values can differ according to the 'tolerance' value provided
27+
- `parameter_match`: provide a known good key/value 'parameter' and check for it's presence or absence in the provided objects
28+
- `regex`: provide a regex and check for the presence or absence of a match in the provided object
29+
- `operator`: evaluate or compare values with a number of different operators: 'in', 'bool', 'string', and numerical comparison with 'int' and 'float'
730

8-
## Use Case
931

10-
`netcompare` enables an easy and direct way to see the outcome of network configuration or operational status change. The intended usage is to collect structured `show` command output before and after a change window. Prior to closing the change window, the results are compared to help determine if the change was successful as intended and if the network is in an acceptable state. The output can be stored with the change's documentation for easy reference and proof of completion.
1132

1233
## Workflow
1334

1435
![netcompare workflow](./docs/images/workflow.png)
1536

16-
1737
## Library Architecture
1838

1939
![netcompare HLD](./docs/images/hld.png)
@@ -22,7 +42,7 @@ An instance of `CheckType` object must be created first before passing one of th
2242

2343
- `exact_match`
2444
- `tolerance`
25-
- `parameters`
45+
- `parameter_match`
2646
- `regex`
2747
- `operator`
2848

@@ -228,58 +248,6 @@ Targeting only the `interfaceStatus` key, we would need to define a reference ke
228248
The anchor logic for the reference key applies to all check-types available in `netcompare`
229249

230250

231-
### parameter_match
232-
233-
parameter_match provides a way to match keys and values in the output with known good values.
234-
235-
The test defines key/value pairs known to be the good value - type `dict()` - as well as a mode - `match`, `no-match` - to match or not against the parsed output. The test fails if any status has changed based on what is defined in pre/post. If there are new values not contained in the input/test value, that will not count as a failure.
236-
237-
238-
Examples:
239-
240-
```python
241-
>>> post_data = {
242-
... "jsonrpc": "2.0",
243-
... "id": "EapiExplorer-1",
244-
... "result": [
245-
... {
246-
... "interfaces": {
247-
... "Management1": {
248-
... "lastStatusChangeTimestamp": 1626247821.123456,
249-
... "lanes": 0,
250-
... "name": "Management1",
251-
... "interfaceStatus": "down",
252-
... "autoNegotiate": "success",
253-
... "burnedInAddress": "08:00:27:e6:b2:f8",
254-
... "loopbackMode": "loopbackNone",
255-
... "interfaceStatistics": {
256-
... "inBitsRate": 3403.4362520883615,
257-
... "inPktsRate": 3.7424095978179257,
258-
... "outBitsRate": 16249.69114419833,
259-
... "updateInterval": 300,
260-
... "outPktsRate": 2.1111866059750692
261-
... }
262-
... }
263-
... }
264-
... }
265-
... ]
266-
>>> my_check = CheckType.init(check_type="parameter_match")
267-
>>> my_jmspath = "result[*].interfaces.*.[$name$,interfaceStatus,autoNegotiate]"
268-
>>> post_value = my_check.get_value(output=post_data, path=my_jmspath)
269-
>>> # mode: match - Match in the ouptut what is defined under 'params'
270-
>>> my_parameter_match = {"mode": "match", "params": {"interfaceStatus": "connected", "autoNegotiate": "success"}}
271-
>>> actual_results = my_check.evaluate(post_value, **my_parameter_match)
272-
>>> actual_results
273-
({'Management1': {'interfaceStatus': 'down'}}, False)
274-
>>> # mode: no-match - Return what does nto match in the ouptut as defined under 'params'
275-
>>> my_parameter_match = {"mode": "no-match", "params": {"interfaceStatus": "connected", "autoNegotiate": "success"}}
276-
>>> actual_results = my_check.evaluate(post_value, **my_parameter_match)
277-
>>> actual_results
278-
({'Management1': {'autoNegotiate': 'success'}}, False
279-
```
280-
281-
In network data, this could be a state of bgp neighbors being Established or the connectedness of certain interfaces being up.
282-
283251
### Tolerance
284252

285253
The `tolerance` test defines a percentage of differing `float()` between pre and post checks numeric value. The threshold is defined as a percentage that can be different either from the value stated in pre and post fields.
@@ -364,6 +332,59 @@ Lets have a look to a couple of examples:
364332
This test can test the tolerance for changing quantities of certain things such as routes, or L2 or L3 neighbors. It could also test actual outputted values such as transmitted light levels for optics.
365333

366334

335+
### Parameter_match
336+
337+
parameter_match provides a way to match keys and values in the output with known good values.
338+
339+
The test defines key/value pairs known to be the good value - type `dict()` - as well as a mode - `match`, `no-match` - to match or not against the parsed output. The test fails if any status has changed based on what is defined in pre/post. If there are new values not contained in the input/test value, that will not count as a failure.
340+
341+
342+
Examples:
343+
344+
```python
345+
>>> post_data = {
346+
... "jsonrpc": "2.0",
347+
... "id": "EapiExplorer-1",
348+
... "result": [
349+
... {
350+
... "interfaces": {
351+
... "Management1": {
352+
... "lastStatusChangeTimestamp": 1626247821.123456,
353+
... "lanes": 0,
354+
... "name": "Management1",
355+
... "interfaceStatus": "down",
356+
... "autoNegotiate": "success",
357+
... "burnedInAddress": "08:00:27:e6:b2:f8",
358+
... "loopbackMode": "loopbackNone",
359+
... "interfaceStatistics": {
360+
... "inBitsRate": 3403.4362520883615,
361+
... "inPktsRate": 3.7424095978179257,
362+
... "outBitsRate": 16249.69114419833,
363+
... "updateInterval": 300,
364+
... "outPktsRate": 2.1111866059750692
365+
... }
366+
... }
367+
... }
368+
... }
369+
... ]
370+
>>> my_check = CheckType.init(check_type="parameter_match")
371+
>>> my_jmspath = "result[*].interfaces.*.[$name$,interfaceStatus,autoNegotiate]"
372+
>>> post_value = my_check.get_value(output=post_data, path=my_jmspath)
373+
>>> # mode: match - Match in the ouptut what is defined under 'params'
374+
>>> my_parameter_match = {"mode": "match", "params": {"interfaceStatus": "connected", "autoNegotiate": "success"}}
375+
>>> actual_results = my_check.evaluate(post_value, **my_parameter_match)
376+
>>> actual_results
377+
({'Management1': {'interfaceStatus': 'down'}}, False)
378+
>>> # mode: no-match - Return what does nto match in the ouptut as defined under 'params'
379+
>>> my_parameter_match = {"mode": "no-match", "params": {"interfaceStatus": "connected", "autoNegotiate": "success"}}
380+
>>> actual_results = my_check.evaluate(post_value, **my_parameter_match)
381+
>>> actual_results
382+
({'Management1': {'autoNegotiate': 'success'}}, False
383+
```
384+
385+
In network data, this could be a state of bgp neighbors being Established or the connectedness of certain interfaces being up.
386+
387+
367388
### Regex
368389

369390
The `regex` check type evaluates data against a python regular expression defined as check-type argument. As per `parameter_match` the option `match`, `no-match` is also supported.

0 commit comments

Comments
 (0)