Prevent temporary mat from being destroyed

[Rylern]

Running the following Instanseg script on an OME-TIFF image:

selectAnnotations()
qupath.ext.instanseg.core.InstanSeg.builder()
    .modelPath("C:/Users/leole/Documents/QuPath/QuPathUserFolder/instanseg/downloaded/fluorescence_nuclei_and_cells-0.1.1")
    .device("gpu0")
    .inputChannels([ColorTransforms.createChannelExtractor("DAPI"), ColorTransforms.createChannelExtractor("FITC"), ColorTransforms.createChannelExtractor("AF568"), ColorTransforms.createChannelExtractor("AF647")])
    .outputChannels()
    .tileDims(2048)
    .interTilePadding(64)
    .nThreads(4)
    .makeMeasurements(true)
    .randomColors(false)
    .outputType("Default")
    .build()
    .detectObjects()

is sometimes causing a QuPath crash with a log looking like this:

# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00000273a6363460, pid=2568, tid=15912
#
# JRE version: OpenJDK Runtime Environment Temurin-21.0.7+6 (21.0.7+6) (build 21.0.7+6-LTS)
# Java VM: OpenJDK 64-Bit Server VM Temurin-21.0.7+6 (21.0.7+6-LTS, mixed mode, tiered, compressed oops, compressed class ptrs, g1 gc, windows-amd64)
# Problematic frame:
# v  ~StubRoutines::jlong_disjoint_arraycopy 0x00000273a6363460

This happens on Windows 11, with both QuPath 0.6 and QuPath 0.7, and with the corresponding versions of Pytorch and CUDA detailed on https://qupath.readthedocs.io/en/latest/docs/deep/gpu.html.

I managed to get two logs describing the stack trace of the exception:

The last pc belongs to jlong_disjoint_arraycopy (printed below).
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
J 423  jdk.internal.misc.Unsafe.copyMemory0(Ljava/lang/Object;JLjava/lang/Object;JJ)V java.base@21.0.7 (0 bytes) @ 0x0000020b70c5202a [0x0000020b70c51fa0+0x000000000000008a]
J 2595 c1 jdk.internal.misc.Unsafe.copyMemory(Ljava/lang/Object;JLjava/lang/Object;JJ)V java.base@21.0.7 (33 bytes) @ 0x0000020b69677b6c [0x0000020b69677360+0x000000000000080c]
J 7822 c1 jdk.internal.misc.ScopedMemoryAccess.copyMemory(Ljdk/internal/foreign/MemorySessionImpl;Ljdk/internal/foreign/MemorySessionImpl;Ljava/lang/Object;JLjava/lang/Object;JJ)V java.base@21.0.7 (27 bytes) @ 0x0000020b69ff0c14 [0x0000020b69ff0920+0x00000000000002f4]
j  java.nio.FloatBuffer.putBuffer(ILjava/nio/FloatBuffer;II)V+155 java.base@21.0.7
j  java.nio.FloatBuffer.put(Ljava/nio/FloatBuffer;)Ljava/nio/FloatBuffer;+100 java.base@21.0.7
j  ai.djl.ndarray.BaseNDManager.copyBuffer(Ljava/nio/Buffer;Ljava/nio/ByteBuffer;)V+87
j  ai.djl.pytorch.engine.PtNDManager.create(Ljava/nio/Buffer;Lai/djl/ndarray/types/Shape;Lai/djl/ndarray/types/DataType;)Lai/djl/pytorch/engine/PtNDArray;+64
j  ai.djl.pytorch.engine.PtNDManager.create(Ljava/nio/Buffer;Lai/djl/ndarray/types/Shape;Lai/djl/ndarray/types/DataType;)Lai/djl/ndarray/NDArray;+4
j  qupath.ext.djl.DjlTools.matToNDArray(Lai/djl/ndarray/NDManager;Lorg/bytedeco/opencv/opencv_core/Mat;Ljava/lang/String;)Lai/djl/ndarray/NDArray;+175
j  qupath.ext.instanseg.core.MatTranslator.processInput(Lai/djl/translate/TranslatorContext;Lorg/bytedeco/opencv/opencv_core/Mat;)Lai/djl/ndarray/NDList;+13
j  qupath.ext.instanseg.core.MatTranslator.processInput(Lai/djl/translate/TranslatorContext;Ljava/lang/Object;)Lai/djl/ndarray/NDList;+6
j  ai.djl.translate.Translator.batchProcessInput(Lai/djl/translate/TranslatorContext;Ljava/util/List;)Lai/djl/ndarray/NDList;+50
j  ai.djl.inference.Predictor.batchPredict(Ljava/util/List;)Ljava/util/List;+210
j  ai.djl.inference.Predictor.predict(Ljava/lang/Object;)Ljava/lang/Object;+5
j  qupath.ext.instanseg.core.TilePredictionProcessor.process(Lqupath/lib/experimental/pixels/Parameters;)[Lorg/bytedeco/opencv/opencv_core/Mat;+240
j  qupath.ext.instanseg.core.TilePredictionProcessor.process(Lqupath/lib/experimental/pixels/Parameters;)Ljava/lang/Object;+2
j  qupath.lib.experimental.pixels.PixelProcessor$ProcessorTask.run()V+125
j  java.util.concurrent.Executors$RunnableAdapter.call()Ljava/lang/Object;+4 java.base@21.0.7
J 12443 c1 java.util.concurrent.FutureTask.run()V java.base@21.0.7 (123 bytes) @ 0x0000020b6a54f7dc [0x0000020b6a54f060+0x000000000000077c]
j  java.util.concurrent.Executors$RunnableAdapter.call()Ljava/lang/Object;+4 java.base@21.0.7
J 12443 c1 java.util.concurrent.FutureTask.run()V java.base@21.0.7 (123 bytes) @ 0x0000020b6a54f7dc [0x0000020b6a54f060+0x000000000000077c]
j  java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V+92 java.base@21.0.7
j  java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5 java.base@21.0.7
j  java.lang.Thread.runWith(Ljava/lang/Object;Ljava/lang/Runnable;)V+5 java.base@21.0.7
j  java.lang.Thread.run()V+19 java.base@21.0.7
v  ~StubRoutines::call_stub 0x0000020b7069100d

siginfo: EXCEPTION_ACCESS_VIOLATION (0xc0000005), reading address 0x0000020efb7b6080

In short, there is an invalid memory access during a copy from a source buffer to a target buffer. Since the target buffer is created and allocated just before the copy (and the crash may or may not occur with the same exact parameters), I don't think the issue comes from the target buffer, so it must come from the source buffer. The most likely scenario is that the memory of the source buffer is deallocated before the copy is made.

Looking at the stack trace, the source buffer is created on this line:

qupath-extension-djl/src/main/java/qupath/ext/djl/DjlTools.java

Line 407 in 95df7e0

array = manager.create(opencv_dnn.blobFromImage(mat).createBuffer(), shape, dataType);

This shows that the buffer is pointing to a memory owned by a temporary Mat (opencv_dnn.blobFromImage(mat)) created from an existing Mat mat. My first thought was that mat was deallocated or updated before the copy is made, but I didn't find anything to confirm that. Another idea was that the temporary opencv_dnn.blobFromImage(mat) was deallocated before the copy, so I made it a non-temporary object (see the changes of this PR). I am not convinced by this, but I never managed to get the crash again with this change, so it might be the solution.

[alanocallaghan]

Seems somewhat obscene that this would work, but I guess it makes sense that the GC might manage to grab the intermediate object. I'm happy to take this on faith and hope it resolved the issue; at worst it's a neutral change