Sluggish performance when panning - cache invalidation bilat -> finalscale

[kofa73]

Is there an existing issue for this?

• I checked and did not find my issue in the already reported ones

Describe the bug

While investigating #21426, I panned and zoomed a lot. Some of those ran quickly, others very slowly, with constant reprocessing of the pipe.
It turns out the differentiator is whether finalscale runs on the CPU (cache not used) vs GPU (cache used). (I'm still investigating why (on the same image, different runs) it used the CPU when the machine has 2 GPUs available; there were no OpenCL errors or anything similar logged).

Cache hits, usage, full pipe runs (check invalid and also Using -> weird, "Using 6784MB, limit=2007MB"):

$ grep -c 'cache HIT .*\[full HQ\]'  *_cache.txt
bad_cache.txt:0
good_cache.txt:60

$ grep 'cache HIT .*\[full HQ\]' good_cache.txt | grep -oE '\[full HQ\] +[a-z]+ +[0-9]+' | sort | uniq -c
     60 [full HQ]        bilat                  7300

$ grep 'cache report .*\[full HQ\]' bad_cache.txt  | tail -1
   137.1327 cache report                  [full HQ]                                     64 lines (important=8, used=53, invalid=17). Using 6784MB, limit=2007MB. Hits/run=0.00. Hits/test=0.000
$ grep 'cache report .*\[full HQ\]' good_cache.txt  | tail -1
    46.0890 cache report                  [full HQ]                                     64 lines (important=33, used=47, invalid=0). Using 2204MB, limit=2007MB. Hits/run=0.95. Hits/test=0.214

$ grep -cE 'process +(CPU|CL[0-9]) \[full HQ\] +rawprepare' *_cache.txt
bad_cache.txt:12
good_cache.txt:2

logs.zip

The root cause seems to be invalidation @jenshannoschwalm added in 9fbdc4b.
I understand that Hanno's fix is needed. If I remove the invalidation, and place local contrast after AgX, then disable local contrast, I see messages like:

   209.8499 cache HIT                     [preview]        agx                    6000  IOP_CS_LAB 2.268 1.000 1.217, hash=3ae1c00c9e020619
   209.8499 pipe data: from cache         [preview]        agx                    6000       (0/0)  2714x1799 sc=1.000; 

AgX's output should be RGB, but local contrast converted it to Lab (its input space). In this case, there is no visible corruption (the buffer is marked as Lab, and gets converted to RGB by finalscale without loss in quality), but we probably shouldn't rely on that?

Instead of a module overwriting a previous module's output, how about avoiding overwriting the cached data with the converted one, and repeating the conversion instead?

• conversions aren't very frequent
• they are relatively fast - probably better to pay a guaranteed small conversion cost than invalidating the pipeline; an example of on-CPU conversion of a 160 MPx image:

   298.2384 [dt_ioppr_transform_image_colorspace] IOP_CS_LAB-->IOP_CS_RGB took 0.154 secs (1.464 CPU) [channelmixerrgb]
   302.7310 transform colorspace      CPU [full HQ]        bilat                  7000       (0/0) 16296x10856 sc=1.000; IOP_CS_RGB -> IOP_CS_LAB `linear Rec2020 RGB'
   303.2527 [dt_ioppr_transform_image_colorspace] IOP_CS_RGB-->IOP_CS_LAB took 0.522 secs (5.975 CPU) [bilat]
   323.9192 transform colorspace      CPU [full HQ]        finalscale             8500       (0/0) 16296x10856 sc=1.000; IOP_CS_LAB -> IOP_CS_RGB `linear Rec2020 RGB'
   324.0910 [dt_ioppr_transform_image_colorspace] IOP_CS_LAB-->IOP_CS_RGB took 0.171 secs (1.420 CPU) [finalscale]

Steps to reproduce

See the image from #21426 -- use HQ in the darkroom and pan around. Sometimes, but not always (I don't know the reason) we use the CPU.

Expected behavior

No response

Logfile | Screenshot | Screencast

No response

Commit

No response

Where did you obtain darktable from?

darktable.org / GitHub release

darktable version

5.7.0+18~g673e76aa90

What OS are you using?

Linux

What is the version of your OS?

Ubuntu 25.10

Describe your system

No response

Are you using OpenCL GPU in darktable?

None

If yes, what is the GPU card and driver?

No response

Please provide additional context if applicable. You can attach files too, but might need to rename to .txt or .zip

No response

[jenshannoschwalm]

AgX's output should be RGB, but local contrast converted it to Lab (its input space). In this case, there is no visible corruption (the buffer is marked as Lab, and gets converted to RGB by finalscale without loss in quality), but we probably shouldn't rely on that?

No we must not rely on that, cachelines must be usable for any module following

Instead of a module overwriting a previous module's output, how about avoiding overwriting the cached data with the converted one, and repeating the conversion instead?

Yes, we could do that but that would likely require some more changes in pixelpipe code, blending cache ....

But i currently think it's not necessary. (I can't test for the next days as being at work for a few days and just my slow CPU notebook at hand). There has been #21251 about this (which basically reverts the commit iirc but is not fully correct).

You could also test yourself 198bebb of #21274 (i will also try to reproduce with what you tested here)

About finalscale running on CPU or not. I guess you mean something like this

    22.7400 distort raster mask           [full HQ]        lens                   1500       (1/0)  5194x3902 sc=1.000 -->       (0/0)  5198x3904 sc=1.000; 
    23.0035 got raster mask               [full HQ]        bilat                  7300  from module `denoiseprofile.1', distorted to 5198x3904
    23.0035 blend raster              CL0 [full HQ]        bilat                  7300       (0/0)  5198x3904 sc=1.000; IOP_CS_LAB inverted temp
    23.0234 [dt_opencl_write_host_to_image_raw] wrote image to device 'NVIDIA CUDA Quadro RTX 4000' id=0
    23.0237 [opencl copy_image] copied image on device 'NVIDIA CUDA Quadro RTX 4000' id=0
    23.0343 pipe cache get                [full HQ]        finalscale             8800  IOP_CS_RGB line  6(128) at 0x7581f5688040. hash=72115135e1f79879
    23.1947 [dt_opencl_read_host_from_image_raw] read image from device 'NVIDIA CUDA Quadro RTX 4000' id=0
    23.1952 transform colorspace      CPU [full HQ]        finalscale             8800       (0/0)  5198x3904 sc=1.000; IOP_CS_LAB -> IOP_CS_RGB `linear Rec2020 RGB'
    23.2152 process                   CPU [full HQ]        finalscale             8800       (0/0)  5198x3904 sc=1.000 --> (9390/4064)    722x468 sc=2.000; IOP_CS_RGB 492MB
    23.2152 resample_plain            CPU                                                    (0/0)  5198x3904 sc=1.000 --> (9390/4064)    722x468 sc=2.000; lanczos3
    23.2180 importance hints          CL0 [full HQ]        finalscale             8800       (0/0)  5198x3904 sc=1.000; 
    23.2180 pipe cache get                [full HQ]        colorout               8900  IOP_CS_RGB line  5(128) at 0x7581f5160040. hash=889dcbaa01eb31a3

[kofa73]

@jenshannoschwalm : I'll test your PR #21274, thanks! Something really weird was going on, with (same image, same stack) sometimes processing ran on the GPU, and there was no invalidation; exit to lighttable, return to darkroom, and suddenly it switched to the CPU, and all pan/zoom kept reprocessing -- see the very first grep above, showing 0 cache hits, and the cache expanding to 6 GB -- with a limit of 2.

[jenshannoschwalm]

very strange
Ad hoc ideas:

1. All cachelines used but no chance to cleanup? Never seen this but we take more host mem than "suggested" and cleanup before the next pipe run. What does
2. There are very few users with more than one cl device :-) And only one of the devs as far as i know! Absolutely blind guessing here but you don't have pinned tiling switched on by chance for one of your devices?

[jenshannoschwalm]

Also see comment in #20184

[kofa73]

The concrete flow:

• local contrast is enabled -> outputs Lab
  • there is no RGB module between it and finalscale
• we are at > 100% zoom (upscaling does not support OpenCL), or OpenCL is not available for some other reason
• finalscale
  • requires RGB
  • its CPU path converts Lab input to RGB in place, and invalidates the cache line because of the colour space change
  • it then tries to mark its input as important, but it's already been invalidated, so too late
• whenever the pipe reruns (zooming, panning), processing will take place; how much, depends on e.g. available memory (cache size)

[jenshannoschwalm]

1. Yes - opencl upscaling would be good. Maybe brute-force-upscale by 2 or 4 and back-downscale for simplicity
2. The cacheline invalidation will be gone pretty soon :-)