CI: `common` module: tests aren't verified to have run

[david-allison]

Tests for :common are now running, but we do not verify that these have run in CI

• build(ci): run tests on :common and :libanki #18805

Old description

Unit tests for the common module don't run in CI

---

if we have tests and they're just not running that's a "sad miss" because the tests are already there, should log the issue before merge

Originally posted by @mikehardy in #16617 (review)

[david-allison]

Script to check whether tests are running

 ./gradlew jacocoUnitTestReport --dry-run | grep ":common:testPlayDebugUnitTest" 
 ./gradlew testPlayDebugUnitTest --dry-run | grep ":common:testPlayDebugUnitTest"

I looked into this briefly, the following patch solved some, but not all issues

Subject: [PATCH] aa
---
Index: AnkiDroid/build.gradle
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/AnkiDroid/build.gradle b/AnkiDroid/build.gradle
--- a/AnkiDroid/build.gradle	
+++ b/AnkiDroid/build.gradle	
@@ -287,7 +287,6 @@
 }
 
 apply from: "./robolectricDownloader.gradle"
-apply from: "./jacoco.gradle"
 apply from: "../lint.gradle"
 
 dependencies {
@@ -423,3 +422,6 @@
     androidTestImplementation libs.kotlin.test.junit
     androidTestImplementation libs.androidx.fragment.testing
 }
+
+// dependencies adds additional tasks
+apply from: "./jacoco.gradle"
\ No newline at end of file
Index: common/build.gradle.kts
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/common/build.gradle.kts b/common/build.gradle.kts
--- a/common/build.gradle.kts	
+++ b/common/build.gradle.kts	
@@ -3,6 +3,7 @@
     alias(libs.plugins.kotlin.android)
 }
 
+
 android {
     // this cannot conflict with com.ichi2.anki
     // but we can define files in 'com.ichi2.anki' inside 'common'
@@ -26,6 +27,21 @@
             )
         }
     }
+
+    flavorDimensions += "appStore"
+    productFlavors {
+        create("play") {
+            dimension = "appStore"
+        }
+        create("amazon") {
+            dimension = "appStore"
+        }
+        // A 'full' build has no restrictions on storage/camera. Distributed on GitHub/F-Droid
+        create("full") {
+            dimension = "appStore"
+        }
+    }
+
     compileOptions {
         sourceCompatibility = JavaVersion.VERSION_1_8
         targetCompatibility = JavaVersion.VERSION_1_8

[Aditya13s]

Can you explain more about it and what need to be done to solve it.

[david-allison]

Tests aren't run for the common module in CI.

I don't know exactly what's required to fix it, but after it's resolved, tests for common will run in CI under Jacoco

[mikehardy]

note that it is possible to actually add a test (okay: technically build assertion that can fail the build) to make sure tests run!
That should be part of the solution

What do I mean? We had a similar problem with emulator testing where zero tests were run sometimes (there was a root cause, which we fixed, but the problem of "silent failure" / "false positive" where we thought tests were running but they weren't was also a problem)

We fixed it like so:

1- a gradle block that is able to determine by looking at test reports if a condition was met (in this case it is that connected tests ran, but in the solution here it could be that some well-known common test ran)
2- making the test task finalize by running that new gradle block

Anki-Android/AnkiDroid/build.gradle

Lines 267 to 287 in bd29627

	// Issue 11078 - some emulators run, but run zero tests, and still report success
	tasks.register('assertNonzeroAndroidTests') {
	doLast {
	// androidTest currently creates one .xml file per emulator with aggregate results in this dir
	File folder = file("./build/outputs/androidTest-results/connected/flavors/play")
	File[] listOfFiles = folder.listFiles({ d, f -> f ==~ /.*.xml/ } as FilenameFilter)
	for (File file : listOfFiles) {
	// The aggregate results file currently contains a line with this pattern holding test count
	String[] matches = file.readLines().findAll { it.contains('<testsuite') }
	if (matches.length != 1) {
	throw new GradleScriptException("Unable to determine count of tests executed for " + file.name + ". Regex pattern out of date?", null)
	}
	if (!(matches[0] ==~ /.* tests="\d+" .*/) || matches[0].contains('tests="0"')) {
	throw new GradleScriptException("androidTest executed 0 tests for " + file.name + " - Probably a bug with the emulator. Try another image.", null)
	}
	}
	}
	}
	afterEvaluate {
	tasks.named('connectedPlayDebugAndroidTest').configure { finalizedBy('assertNonzeroAndroidTests') }
	}

[github-actions]

Hello 👋, this issue has been opened for more than 3 months with no activity on it. If the issue is still here, please keep in mind that we need community support and help to fix it! Just comment something like still searching for solutions and if you found one, please open a pull request! You have 7 days until this gets closed automatically