Test fails with I20230615-1800

[iloveeclipse]

We have new reproducible test fails on all platform in JDT with I20230615-1800 build.

jdt.apt.tests

• https://download.eclipse.org/eclipse/downloads/drops4/I20230615-1800/testresults/html/org.eclipse.jdt.apt.tests_ep429I-unit-cen64-gtk3-java17_linux.gtk.x86_64_17.html
• AptReconcileTests | testGeneratedFile complains about workspace settings: Workspace options should be back to their default.

jdt.core.tests.model

• https://download.eclipse.org/eclipse/downloads/drops4/I20230615-1800/testresults/html/org.eclipse.jdt.core.tests.model_ep429I-unit-cen64-gtk3-java17_linux.gtk.x86_64_17.html
• JavaProjectTests.testAddExternalLibFolder6 complains about workspace settings: Workspace options should be back to their default.
• APIDocumentationTests.testJavaCoreAPI : some strange fail

One or many discrepancies, including: . ----------- Expected ------------ ------------ but was ------------ *.launch --------- Difference is ---------- expected:<[]> but was:<[*.launch]>

junit.framework.ComparisonFailure: One or many discrepancies, including: .
----------- Expected ------------

------------ but was ------------
*.launch
--------- Difference is ----------
expected:<[]> but was:<[*.launch]>
at org.eclipse.jdt.core.tests.junit.extension.TestCase.assertStringEquals(TestCase.java:266)
at org.eclipse.jdt.core.tests.junit.extension.TestCase.assertEquals(TestCase.java:242)
at org.eclipse.jdt.core.tests.dom.APIDocumentationTests.testJavaCoreAPI(APIDocumentationTests.java:302)

jdt.text.tests

• https://download.eclipse.org/eclipse/downloads/drops4/I20230615-1800/testresults/html/org.eclipse.jdt.text.tests_ep429I-unit-cen64-gtk3-java17_linux.gtk.x86_64_17.html
• PluginsNotLoadedTest | pluginsNotLoaded some bundle loading issue

Wrong bundles loaded: - org.eclipse.jdt.apt.core - org.eclipse.jdt.apt.pluggable.core expected:<0> but was:<64>

java.lang.AssertionError: Wrong bundles loaded:
- org.eclipse.jdt.apt.core
- org.eclipse.jdt.apt.pluggable.core
expected:<0> but was:<64>

There were not many changes in last build, most notable:

• (automatically adds all detected JVMs): Detect JVM installs at startup eclipse.jdt.debug#231
• (jakarta.inject.Inject is supported) Issue eclipse-jdt#1112 - jakarta.inject.Inject is now recognized #1121

I assume (but have not verified that yet) eclipse-jdt/eclipse.jdt.debug#231 could be the one who caused the fails above, since it may affect which JVM is selected a s default, or some platform change.

[mickaelistria]

I'll have a look and report possible remediations.

[mickaelistria]

JavaCore.getDefaultOptions() seems to return 1.4 compliance when called early, and 17 when called later. I believe the DetectJVM job is working in parallel of those tests, so the default options are changed in the meantime.
What I wonder is whether returning 1.4 compliance is a good thing in 1st place? What may cause that? Is this expected?

[jukzi]

during testAddExternalLibFolder6 the p.build(IncrementalProjectBuilder.FULL_BUILD, null); sets options to compliance

Thread [main] (Suspended (breakpoint at line 5277 in JavaModelManager))	
	JavaModelManager.setOptions(Hashtable<String,String>) line: 5277	
	JavaCore.setOptions(Hashtable<String,String>) line: 6260	
	JavaRuntime.updateCompliance(IVMInstall) line: 3390	
...
	JavaModelManager$CompilationParticipants.getCompilationParticipants(IJavaProject) line: 448	
...
	Project.internalBuild(IBuildConfiguration, int, String, Map<String,String>, IProgressMonitor) line: 609	
	Project.build(int, IProgressMonitor) line: 121	
	JavaProjectTests.testAddExternalLibFolder6() line: 240	

but later in org.eclipse.jdt.core.tests.model.AbstractJavaModelTests.tearDown() defaultOptions are assumed