RobotMBT v1.0 - Tretmans Release

Release 1.0, released in celebration of the final User Committee meeting of the Dutch national research project TiCToC. Named after project leader Jan Tretmans.

TiCToC, Testing in Times of Continuous Change, is one of many projects lead by Dr Tretmans on the topic of model-based testing. The project brings universities and industry together to deliver novel and practical results. Dr Tretmans has been a great inspiration and a driving force behind advances in model-based testing over the past decades.

In this release
Model-based testing integrated into Robot Framework with dynamic trace and test case generation. You use the popular Given-When-Then style to capture behaviour in scenarios. Then, using RobotMBT, you turn them into a model that produces test coverage that goes beyond the standard linear test suite flow.

How? By decoupling test suite design from the test run through:

• Head-to-tail sequencing
• Sequencing with loops
• Scenario variations from equivalence classes
• When-step action refinement

Supporting features:

• Set run targets on coverage, duration and/or number of scenarios
• Graphs for visualisation
• Rerunning a previously generated trace
• Support for action-driven keywords
• Embedded, fixed, and variable argument support
• Concise, yet elaborate modelling logging

Version 0.14 - Longer test runs

So far, test runs always ended when single coverage was achieved. I.e., when each scenario was hit at least once. This release introduces new options that allow you to continue generating traces after single coverage is achieved.

New run targets:

• Run duration
• Number of executed scenarios

Version 0.13 - on-the-fly suite generation

In preparation of more flexible trace generation, this release refactors how test cases are added to a test suite. Previously, the whole suite was generated at once and then started. Now, the model-based test suite is started with just a single test case and continues to add one test case at a time until the target coverage is reached.

Also in this release:

• Untouched test attributes are retained. E.g. tags and documentation.
• Fixed an issue where action refinement could be entered unintended.
• Fixed an issue where sometimes the wrong path was highlighted in the graph.

Version 0.12 - Handle larger models

This release helps in creating and running larger models. While converting real-life test suites in a commercial setting, it was found that models quickly became more challenging and RobotMBT could struggle to find valid traces within a reasonable time frame. Additionally, understanding these larger models from text logging alone brought their own challenges, while logging from the trace generation process got too large, making them difficult to navigate. To improve the overall experience this new version offers both technical and useability improvements.

New in this release:

• New algorithm for improved performance
• Graphs for model visualisation
• Logging optimisations

Also in this release:

• Fix for random seed generation
  When running multiple model-based suites in a single test run, after rerunning a trace using a specific seed, the next new seed to be generated was always the same.

Compatibility notice:
Due to the algorithm update, random seeds stored from an earlier version will not recreate the same trace after upgrading to this version.

Version 0.11 - Fix ignored timeouts

Fixed:
When running Treat this test suite model-based in a context where a Robot Framework timeout could be triggered, it could happen that the timeout would be ignored and trace generation continued regardless. Specifically, this would happen when the timeout occurred while executing the user's modelling code.

Also in this release:

• Direct options passed as argument to Treat this test suite model-based will be used during that call only. If settings are intended for multiple model runs, then the dedicated set or Update model-based options keywords should be used.
• Added Contributing guidelines, code improvements and cicd for improved collaboration with external parties.

Version 0.10 - Step modifiers for non-embedded arguments

In the previous release, support was added for action keywords and all their argument types. What was not supported yet, were step modifiers on arguments other than embedded. With this release that limitation is lifted. Now, all argument types can have modifiers.

New in this release:

• Step modifiers for non-embedded arguments
• Improved presentation of optional arguments
• Logging random seed when running without debug logging

Version 0.9 - Action keywords

Although RobotMBT is primarily designed for BDD style, we do occasionally see the need for using action keywords. Either during development and debugging, or to put a few actions in a teardown to get out of a dead-end.

For action keywords it is more common to have positional arguments, which was not previously supported. RobotMBT now also supports positional, named, variable and free-named arguments, as well as keywords that return a value. These non-embedded arguments are also available for given-when-then steps.

Limitations:

• Step argument modifiers for non-embedded arguments are planned for a later release.
• Only linear sequential scenarios are supported for modelling. Modelling will fail when other Robot constructs are used, like IF or FOR.

Version 0.8 - Robot Framework 7.3 compatibility

RobotMBT version 0.8 is a compatibility release. No new features are added. Due to a change in how Robot Framework parses embedded arguments, running older RobotMBT versions on Robot Framework 7.3 will yield a deprecation warning: 'EmbeddedArguments.match()' is deprecated since Robot Framework 7.3

When using an older version of Robot Framework, you will also need an older version of RobotMBT.

Version 0.7 - Rerun previous trace

Test trace generation is randomised by default. But what if you hit this unique case no one thought of before? Or that special combination that reveals a defect? Then you want to have the option to rerun that specific trace. Version 0.7 introduces a seed argument that lets you reproduce a previously generated trace.

Version 0.6 - Step argument modifiers

New features

• Step argument modifiers
• Scenario variables

Enhancements

• Python 3.13 support
• Various performance and useability improvements

Step argument modifiers are the main new feature for this release. They offer test run variation from an all new angle. Step argument modifiers let you define a set of valid options for each Robot argument in a step. At run time, a new concrete scenario example is generated, using the available options, each time the scenario is included in a test run.