|
1 | 1 | # EvoMaster GitHub Action |
2 | 2 |
|
3 | | -This is a GitHub Action for running EvoMaster as part of CI. |
| 3 | +This is a GitHub Action for running [EvoMaster](https://github.com/WebFuzzing/EvoMaster) as part of CI. |
4 | 4 |
|
5 | | -DOCUMENTATION UNDER CONSTRUCTION |
6 | 5 |
|
| 6 | +## Usage |
| 7 | + |
| 8 | +For now, it is recommended to use this action only for _black-box_ testing. |
| 9 | +Documentation and configuration settings for _white-box_ testing will be done in the future. |
| 10 | + |
| 11 | +Example of configuration: |
| 12 | + |
| 13 | +``` |
| 14 | +- name: Build the Application |
| 15 | + run: ... # this swill depend on your application |
| 16 | +
|
| 17 | +- name: Start Docker Compose |
| 18 | + run: docker compose up -d --build |
| 19 | +
|
| 20 | +- uses: webfuzzing/evomaster-action@v1 |
| 21 | + with: |
| 22 | + args: >- |
| 23 | + --writeWFCReport true |
| 24 | + --blackBox true |
| 25 | + --bbSwaggerUrl http://localhost:8080/v3/api-docs |
| 26 | + --maxTime 20s |
| 27 | + --showProgress false |
| 28 | + failOnErrors: "true" |
| 29 | +
|
| 30 | +- name: Upload Generated Files |
| 31 | + if: always() |
| 32 | + uses: actions/upload-artifact@v4 |
| 33 | + with: |
| 34 | + name: results |
| 35 | + path: ./generated_tests |
| 36 | +
|
| 37 | +- name: Stop Docker Compose |
| 38 | + if: always() |
| 39 | + run: docker compose down |
| 40 | +``` |
| 41 | + |
| 42 | + |
| 43 | +How you want to run _EvoMaster_ is CI is up to you, e.g., if on each commit, or only on specific branches, or only for PRs, etc. |
| 44 | +Then, there are five steps. |
| 45 | + |
| 46 | +1) Need to build your application. This will be programming language and framework dependent. |
| 47 | + |
| 48 | +2) The recommended way to start your application is with _Docker Compose_. This alo helps if you need to start databases (e.g., Postgres, MySQL and MongoDB). |
| 49 | + |
| 50 | +3) Use `webfuzzing/evomaster-action@v1` to run _EvoMaster_. |
| 51 | + |
| 52 | +4) Upload the generated tests created by _EvoMaster_. This will enable you to download them as a zip file. |
| 53 | + |
| 54 | +5) Shutdown the application using _Docker Compose_ |
| 55 | + |
| 56 | + |
| 57 | +At the moment, this action has 2 options: |
| 58 | + |
| 59 | +* __failOnErrors__: specify whether to fail the build if any fault was detected by _EvoMaster_. |
| 60 | + |
| 61 | +* __args__: command-line arguments given to _EvoMaster_. These are exactly the same you would use when running _EvoMaster_ on your local machine. |
| 62 | + |
| 63 | +In this example, we used the arguments: |
| 64 | + |
| 65 | +* _--writeWFCReport true_: this is essential to analyze the results of _EvoMaster_ (e.g., to fail the build if any fault is found). Future versions of _EvoMaster_ might have this option on by default, but still specifying it manually here does not harm |
| 66 | + |
| 67 | +* _--blackBox true_: specifying we are doing _black_box_ testing. for historical reasons, default testing mode in _EvoMaster_ is _white-box_. |
| 68 | + |
| 69 | +* _--bbSwaggerUrl http://localhost:8080/v3/api-docs_: when testing a REST API, need to tell where to find the schema. In this example, it is provided by the API itself, which is listening on port 8080. Note, however, that _Docker Compose_ needs to be configured to expose such port, so that _EvoMaster_ can communicate with the API running inside _Docker Compose_. |
| 70 | + |
| 71 | +* _--maxTime 20s_: specify for how long to run _EvoMaster_. The longer you run it the better results you can expect. Ideally, try to run for at least 10 minutes, but better if for 1 hour, or more. |
| 72 | + |
| 73 | +* _--showProgress false_: a progress bar is useful when running _EvoMaster_ on a local shell. But, on GitHub Actions, the line-overwrite feature does not work, and each update creates new lines. This can flood the logs of this action. If this is a problem, this option can be used to deactivate these logs. |
| 74 | + |
| 75 | + |
| 76 | +## New Features |
| 77 | + |
| 78 | +At the time of writing, this is a first version of this new GitHub Action. |
| 79 | +Several new features will be added in the next coming months. |
| 80 | +If there is any important feature you think it is missing, please let us know by creating a new post on the [Discussions page](https://github.com/WebFuzzing/evomaster-action/discussions). |
7 | 81 |
|
8 | 82 |
|
9 | 83 |
|
0 commit comments