[!] WARNING: Instrumentation output varies across runs.

[Noi98]

I have solved all WinAFL issues and I managed to get DynamoRIO working and everything is fine, even the harness. I re-checked everything, but I still get this error:

[!] WARNING: Instrumentation output varies across runs.

I am using afl-cmin.

I have 258 inputs, and when I run afl-cmin, sometimes I get 70 outputs, sometimes 71, and sometimes 69. So I’m not sure what is going on. When I run WinAFL, this warning appears for about 30 inputs.

I tried this solution:
#111 (comment)
and it reduced the occurrences to about 5, but I still don’t understand the root cause.

I also tried this:
#111 (comment)
but it didn’t make any difference.

So what information can I provide you so you can help me solve this issue?
@ifratric

[ifratric]

Hi, the "Instrumentation output varies across runs" is quite common in WinAFL and not necessarily a problem. It means that the program took different paths when running with the same input which can happen for a variety of reasons (e.g. windows events, different paths taken during memory allocation depending on memory usage etc.) This is further emphasized by the fact that WinAFL relies heavily on persistent mode fuzzing - meaning that multiple inputs get executed without restarting the target process. A program might take some different paths during the first run vs. subsequent runs due to e.g. some structures having to be initialized during the first run.

If you're interested in a solution, my other fuzzer, Jackalope (https://github.com/googleprojectzero/Jackalope), tries to solve this by running the same sample multiple times (only when new coverage is detected). It then splits the new coverage into two sets: blocks/edges that were present in each run (stable coverage, coverage that depends on the sample) and those that were not (unstable coverage that varies across runs). Only samples that trigger stable coverage are considered interested and get saved to corpus.

[Noi98]

So it’s not necessarily a problem, and I can ignore it and start the fuzzing session without worrying about the warning? Because I literally checked everything — the harness, the inputs — and my stability was 99.81% with an exec speed of 200, so I think it’s completely normal. Can I proceed with fuzzing while ignoring this warning?
@ifratric

[ifratric]

Yes.