Skip to content

Add a 1st migration guide#24

Merged
dweiss merged 2 commits into
randomizedtesting:mainfrom
dadoonet:doc-migration
Apr 22, 2026
Merged

Add a 1st migration guide#24
dweiss merged 2 commits into
randomizedtesting:mainfrom
dadoonet:doc-migration

Conversation

@dadoonet

Copy link
Copy Markdown
Contributor

Those are my first findings when I migrated my demo repo (for talks) to Junit6 / randomizedtesting-jupiter.

Those are my first findings when I migrated my [demo repo](https://github.com/dadoonet/randomizedtesting-demo/) (for talks) to Junit6 / randomizedtesting-jupiter.
Comment thread MIGRATION.md

@dweiss dweiss left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a good concise overview. There are *.md files next to tests that provide similar migration guidance but the top-level file is also fine, thank you.

Comment thread MIGRATION.md Outdated
Comment thread MIGRATION.md Outdated
@dweiss

dweiss commented Apr 21, 2026

Copy link
Copy Markdown
Contributor

@dadoonet

Copy link
Copy Markdown
Contributor Author

the md files are here - https://github.com/randomizedtesting/randomizedtesting-jupiter/tree/main/randomizedtesting-jupiter/src/test/java/com/carrotsearch/randomizedtesting/tests

Yeah. Saw it. It was super helpful BTW.
May be I should link to those pages from the migration guide.

BTW I saw that running from IntelliJ is triggering as well the ThreadLeak detector.

I added a class in my repo to avoid this but I'm wondering if we should add it or something similar as another option, may be not within the default system thread filter though.

@dweiss

dweiss commented Apr 21, 2026

Copy link
Copy Markdown
Contributor

Which thread is triggering thread leaks? I use intellij and I haven't seen this.

* Move Junit4 specific changes at the end
* Remove numbers in titles
* Add comments about repeated tests
* Add links to other docs
@dadoonet

Copy link
Copy Markdown
Contributor Author

I updated the doc. I'm still not super happy with the "Repeated tests"... Let see if I'm more inspired or feel free to suggest what you think would be more useful.

I'm going to share just after this the IntelliJ thing I saw.

@dadoonet

Copy link
Copy Markdown
Contributor Author

Which thread is triggering thread leaks? I use intellij and I haven't seen this.

When running for IntelliJ JUnit Runner:

/Users/david/.sdkman/candidates/java/21.0.2-open/bin/java -javaagent:/Users/david/Library/Caches/JetBrains/IntelliJIdea2025.3/captureAgent/debugger-agent.jar=file:///var/folders/zf/nn5j972n4rb4_x4kyb1ly7cr0000gn/T/capture2598953298372926234.props -ea -Didea.test.cyclic.buffer.size=1048576 -javaagent:/Applications/IntelliJ IDEA.app/Contents/lib/idea_rt.jar=65406 -Dkotlinx.coroutines.debug.enable.creation.stack.trace=false -Ddebugger.agent.enable.coroutines=true -Dkotlinx.coroutines.debug.enable.flows.stack.trace=true -Dkotlinx.coroutines.debug.enable.mutable.state.flows.stack.trace=true -Ddebugger.async.stack.trace.for.all.threads=true -Dfile.encoding=UTF-8 -Dsun.stdout.encoding=UTF-8 -Dsun.stderr.encoding=UTF-8 -classpath /Users/david/.m2/repository/org/junit/platform/junit-platform-launcher/6.0.3/junit-platform-launcher-6.0.3.jar:/Applications/IntelliJ IDEA.app/Contents/lib/idea_rt.jar:/Applications/IntelliJ IDEA.app/Contents/plugins/junit/lib/junit6-rt.jar:/Applications/IntelliJ IDEA.app/Contents/plugins/junit/lib/junit5-rt.jar:/Applications/IntelliJ IDEA.app/Contents/plugins/junit/lib/junit-rt.jar:/Users/david/IdeaProjects/talks/randomizedtesting-demo/target/test-classes:/Users/david/IdeaProjects/talks/randomizedtesting-demo/target/classes:/Users/david/.m2/repository/com/carrotsearch/randomizedtesting/randomizedtesting-jupiter/0.2.0/randomizedtesting-jupiter-0.2.0.jar:/Users/david/.m2/repository/org/junit/jupiter/junit-jupiter-engine/6.0.3/junit-jupiter-engine-6.0.3.jar:/Users/david/.m2/repository/org/junit/platform/junit-platform-engine/6.0.3/junit-platform-engine-6.0.3.jar:/Users/david/.m2/repository/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.jar:/Users/david/.m2/repository/org/jspecify/jspecify/1.0.0/jspecify-1.0.0.jar:/Users/david/.m2/repository/org/assertj/assertj-core/3.27.7/assertj-core-3.27.7.jar:/Users/david/.m2/repository/net/bytebuddy/byte-buddy/1.18.3/byte-buddy-1.18.3.jar:/Users/david/.m2/repository/org/junit/jupiter/junit-jupiter/6.0.3/junit-jupiter-6.0.3.jar:/Users/david/.m2/repository/org/junit/jupiter/junit-jupiter-api/6.0.3/junit-jupiter-api-6.0.3.jar:/Users/david/.m2/repository/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.jar:/Users/david/.m2/repository/org/junit/platform/junit-platform-commons/6.0.3/junit-platform-commons-6.0.3.jar:/Users/david/.m2/repository/org/junit/jupiter/junit-jupiter-params/6.0.3/junit-jupiter-params-6.0.3.jar:/Users/david/.m2/repository/org/apache/logging/log4j/log4j-core/2.25.4/log4j-core-2.25.4.jar:/Users/david/.m2/repository/org/apache/logging/log4j/log4j-api/2.25.4/log4j-api-2.25.4.jar com.intellij.rt.junit.JUnitStarter -ideVersion5 -junit6 fr.pilato.talk.randomizedtesting.RandomizedTest,stopOrIdentifyYourThreads30
(61 ) 🌍 Starting test suite with Locale [mg-Latn-MG]
(71 ) 🏁 Starting test [stopOrIdentifyYourThreads30()]
(130)  ➡️ starting a new Thread
avr. 22, 2026 12:27:25 PM com.sun.jmx.remote.internal.ServerCommunicatorAdmin$Timeout run
WARNING: Unexpected Exception: java.lang.InterruptedException

java.lang.AssertionError: 1 thread(s) leaked from suite [RandomizedTest]:
   1) Thread[id=36, name=JMX server connection timeout 36, state=TERMINATED, group={null group}]
        at java.base@21.0.2/java.lang.Object.wait0(Native Method)
        at java.base@21.0.2/java.lang.Object.wait(Object.java:366)
        at java.management@21.0.2/com.sun.jmx.remote.internal.ServerCommunicatorAdmin$Timeout.run(ServerCommunicatorAdmin.java:171)
        at java.base@21.0.2/java.lang.Thread.runWith(Thread.java:1596)
        at java.base@21.0.2/java.lang.Thread.run(Thread.java:1583)

	at com.carrotsearch.randomizedtesting.jupiter.internals.DetectThreadLeaksExtension.checkLeaks(DetectThreadLeaksExtension.java:281)
	at com.carrotsearch.randomizedtesting.jupiter.internals.DetectThreadLeaksExtension.afterAll(DetectThreadLeaksExtension.java:130)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)


Process finished with exit code 255

When running from the CLI:

mvn test -Dtest=RandomizedTest#stopOrIdentifyYourThreads30

It passes:

[INFO] --- surefire:3.5.5:test (default-test) @ randomizedtesting-demo ---
[INFO] Using auto detected provider org.apache.maven.surefire.junitplatform.JUnitPlatformProvider
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running fr.pilato.talk.randomizedtesting.RandomizedTest
(61 ) 🌍 Starting test suite with Locale [dsb-Latn-DE]
(71 ) 🏁 Starting test [stopOrIdentifyYourThreads30()]
(130)  ➡️ starting a new Thread
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.154 s -- in fr.pilato.talk.randomizedtesting.RandomizedTest
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO] 

The test is:

@DetectThreadLeaks
@DetectThreadLeaks.ExcludeThreads({RandomizedTest.FriendlyZombieFilter.class})
@Randomized
class RandomizedTest {

    @Test
    void stopOrIdentifyYourThreads30() {
        LOGGER.info(" ➡️ starting a new Thread");
        new Thread(() -> {
            while (true) {
                try {
                    Thread.sleep(1000L);
                } catch (InterruptedException ignored) {
                }
            }
        }, "friendly-zombie").start();
    }

    public static class FriendlyZombieFilter implements Predicate<Thread> {
        public boolean test(Thread t) {
            return "friendly-zombie".equals(t.getName());
        }
    }
}

If I'm adding this:

package fr.pilato.talk.randomizedtesting;

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;

import java.util.function.Predicate;

/**
 * This filter is only needed when running the tests from IntelliJ
 */
public class IntelliJThreadsFilter implements Predicate<Thread> {
    private static final Logger LOGGER = LogManager.getLogger();
    public boolean test(Thread t) {
        boolean intellijThreads = t.getName().startsWith("JMX server") || t.getName().startsWith("RMI TCP Connection");
        if (intellijThreads) {
            LOGGER.warn("Detected IntelliJ threads [{}], if you are running the tests from IntelliJ, " +
                            "you can ignore this warning or add [{}] to the thread leak filters",
                    t.getName(), IntelliJThreadsFilter.class.getSimpleName());
        }
        return intellijThreads;
    }
}

And:

@DetectThreadLeaks.ExcludeThreads({RandomizedTest.FriendlyZombieFilter.class, IntelliJThreadsFilter.class})

Then it passes (of course).

@dweiss

dweiss commented Apr 22, 2026

Copy link
Copy Markdown
Contributor

This passes for me without a glitch. How are you launching intellij? the jmx server is odd - I think that check thread leaks annotation is actually doing the right job! :)

@dweiss

dweiss commented Apr 22, 2026

Copy link
Copy Markdown
Contributor

If you can share a maven project zip with this, I can try to import it locally as a maven project. I've compared my intellij options but I don't see anything jmx related (I have a newer intellij but this shouldn't matter).

@dweiss

dweiss commented Apr 22, 2026

Copy link
Copy Markdown
Contributor

I see it at https://github.com/dadoonet/randomizedtesting-demo/, sorry. Can't reproduce though - it must be something with your intellij; crazy!
image

@dadoonet

Copy link
Copy Markdown
Contributor Author

I have this:

IntelliJ IDEA 2025.3.4
Build #IU-253.32098.37, built on March 17, 2026
Source revision: a1f2bc0e0d733
Licensed to File System Crawler for Elasticsearch / David Pilato
Subscription is active until October 6, 2026.
For non-commercial open source development only.
Runtime version: 21.0.10+7-b1163.110 aarch64 (JCEF 137.0.17)
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
Toolkit: sun.lwawt.macosx.LWCToolkit
macOS 26.4.1
GC: G1 Young Generation, G1 Concurrent GC, G1 Old Generation
Memory: 4096M
Cores: 12
Metal Rendering is ON
Registry:
  ide.experimental.ui=true
  maven.packagesearch.enabled=false
Non-Bundled Plugins:
  com.intellij.notebooks.core (253.32098.37)
  JavaScript (253.32098.37)
  com.intellij.java (253.32098.37)
  org.jetbrains.completion.full.line (253.32098.37)
  com.jetbrains.space (253.32098.37)
  com.github.xepozz.gitcodeowners (2026.1.0)
  Subversion (253.32098.60)
  PythonCore (253.32098.37)
  com.intellij.kubernetes (253.32098.40)
  com.intellij.jvm.dfa.analysis (253.30387.20)
  com.intellij.mermaid (0.0.27+IJ.253)
  palantir-java-format (2.89.0)
  org.jetbrains.plugins.spotbugs (1.2.8)
  com.intellij.spring.boot (253.32098.37)
  org.jetbrains.plugins.rest (253.28294.218)
  com.jetbrains.restClient (253.32098.37)
  Pythonid (253.32098.37)
  com.intellij.aqua (253.28294.251)
  com.intellij.javaee.jpa (253.32098.37)
  intellij.jupyter (253.32098.37)
  Docker (253.32098.40)
  tanvd.grazi (253.32098.37)
  com.intellij.settingsSync (253.32098.37)
  org.jetbrains.idea.maven (253.32098.37)
  PerforceDirectPlugin (253.32098.39)
  org.sonarlint.idea (12.0.1.84418)
  com.intellij.spring.mvc (253.32098.37)
  com.github.copilot (1.7.1-243)
Kotlin: 253.32098.37-IJ

It's not the community edition. Not sure about which one you are using.
Might be caused by a plugin?

@dweiss

dweiss commented Apr 22, 2026

Copy link
Copy Markdown
Contributor

There is a difference in how intellij launches subprocesses on MacOS and on Linux. There is a lot going on - the IDE is attaching to the jvm, this triggers jmx/ rmi. I'm not sure why this passes on Linux, still trying to figure this one out.

In any case, if you're using the ExcludeThreads annotation, you should also include the built-in filters. Look at the last line in migration notes here -

https://github.com/randomizedtesting/randomizedtesting-jupiter/blob/main/randomizedtesting-jupiter/src/test/java/com/carrotsearch/randomizedtesting/tests/F005_ThreadLeaks.md

The default value of @DetectThreadLeaks.ExcludeThreads points at SystemThreadFilter. 
There is no defaultFilters parameter; just use the SystemThreadFilter in any custom 
declarations filter declarations.

@dweiss

dweiss commented Apr 22, 2026

Copy link
Copy Markdown
Contributor

This looks very much like a race condition - if the initial snapshot of all live threads includes jmx/rmi, the test will pass. Otherwise it'll detect a leak and fail. This affects intellij because it does start rmi services... I'm not sure if it should be excluded or not (as part of the default filter). I'll think about it. Definitely something to consider though.

In the meantime, your demo should also include the built-in filters -

@DetectThreadLeaks.ExcludeThreads({RandomizedTest.FriendlyZombieFilter.class, IntelliJThreadsFilter.class, SystemThreadFilter.class})

@dweiss dweiss merged commit 4c94cf6 into randomizedtesting:main Apr 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants