Extracting mutation information from verbose log

[mohimoradi]

I'm attempting to extract mutation information from the verbose log. I ran the pit on the Saga Actuator project and saved the log file using the command below.

mvn clean test-compile org.pitest:pitest-maven:mutationCoverage -Dverbose=true -DtimeoutConstant=3000 -fn --log-file Pitlog.txt

By exploring the mutation coverage in the pit report for each sub-module and adding it all up. There were 468 mutations covered in total. However, when I searched the log file, I only found 354 mutations. I check by matching each line that contains "detected = SURVIVED by" or "detected = KILLED by," resulting in 291 killed and 63 survivors. So around 114 mutations are missing in the log file.
If I'm wrong in that assumption, what are the criteria for finding all of the covered mutations?

Thanks

[hcoles]

Pitest provides structured output (XML and CSV) which would be easier to work with than log files and might be worth considering instead.

There are more dection statuses than SURVIVED and KILLED, which probably explains your missing mutants. See

https://pitest.org/quickstart/basic_concepts/

[mohimoradi]

Thanks!

	<plugin>
	    <groupId>org.pitest</groupId>
	    <artifactId>pitest-maven</artifactId>
	    <version>LATEST</version>
		<configuration>
			<fullMutationMatrix>true</fullMutationMatrix>
			<outputFormats>
			 <param>XML</param>
			</outputFormats>
		</configuration>
	</plugin>

[mohimoradi]

If the status is TIMED OUT, pit will not report the failing tests, but those will be counted as killed mutations in the HTML report. How can I collect the tests that result in the TIMED OUT status?

<mutation detected='true' status='TIMED_OUT' numberOfTestsRun='0'><sourceFile>SagaExecutionQueryService.java</sourceFile><mutatedClass>org.apache.servicecomb.saga.spring.SagaExecutionQueryService</mutatedClass><mutatedMethod>lambda$querySagaExecutionDetail$2</mutatedMethod><methodDescription>(Lorg/apache/servicecomb/saga/spring/SagaEventEntity;)Z</methodDescription><lineNumber>133</lineNumber><mutator>org.pitest.mutationtest.engine.gregor.mutators.returns.BooleanFalseReturnValsMutator</mutator><indexes><index>8</index></indexes><blocks><block>3</block></blocks><killingTests></killingTests><succeedingTests></succeedingTests><description>replaced boolean return with false for org/apache/servicecomb/saga/spring/SagaExecutionQueryService::lambda$querySagaExecutionDetail$2</description></mutation>

[hcoles]

You would need to modify pitest so that the test that times out was captured. I believe this is less straightforward than it appears, but haven't looked at this area of the code for some years.

[mohimoradi]

Would you kindly provide me with some hints as to what part of the code could require editing? I'll try to change that.

[hcoles]

You would need to alter the Repoter interface and its default implementation so that test names were sent to the parent process before each test was run. Then, if the child process were to hang, the parent process would be able to infer which test caused that infinite loop.

https://github.com/hcoles/pitest/blob/master/pitest/src/main/java/org/pitest/mutationtest/execute/DefaultReporter.java

This would be a non trivial change to the code.