[4k 30fps] OOM after 15m while rendering hevc_nvenc profile

[Hideman85]

Dear all,

I'm posting this here cause I havent found a wiki to discuss about it.

I am running Ubuntu 24.04 (Noble), with kdenlive 23.08.5 for edinting and rendering in command line with melt 7.22.0. My project is 4k 30fps as most of my shoots follow this property.

I am facing huge memory issue (probably memory leaks), the first come with editing in kdenlive after 45min kdenlive have eaten the full of my 32G and have to restart it to work again with it.
Now for rendering, as kdenlive eat all my ram, I do close kdenlive and use rendering scrip with melt command line so melt is free to use the 32G of ram.
Now my project is about 40min and when I do render the project it crash at around 15min with OOM after eating my 32G of ram.
I have plenty of different clips, both videos and images and basic effects like fading, zoom and gaussian bluring. I have some issue to understand why it would lead to OOM since the clips can be unloaded after use.

I am using right now a custom profile

ab=700k acodec=flac channels=2 ar=44100 sample_fmt=s16 color_range=pc f=matroska hwaccel=vaapi hwaccel_output_format=vaapi real_time=-1 threads=0 vaapi_device=/dev/dri/renderD128 vb=70000k vcodec=hevc_nvenc

I am not sure if this is the cause of my OOM, as you can see I removed multi-threading so they do not grow exponentially on their own. My hardware is pretty recent and should follow for this project. (AMD Ryzen 9 7940HS + Nvidia RTX 2070 + 32G LPDDR5x 7400mhz)

I you have any advice to solve my problem or even making myself more efficient, I am welcome and would be very greatful.

[AlxHnr]

This doesn't seem to be limited to hevc_nvenc. With other codecs, even just lossless, I'm observing similar behavior trough KDEnlive. The memory usage of the melt process just keeps growing and growing linearly, until it gets OOM killed by the kernel. 4k@60hz project here, with memory usage surpassing 52 GB after ~35 minutes of rendering. Memory leakages seem to be a recurring theme when skimming trough previous GitHub issues.

[Hideman85]

The only solution I found from other topics to work with melt, is to temporarily create a swapfile of 64G, then it works and compile the whole project.

[AlxHnr]

Or export the video in smaller sections, plus a separate audio file. Then stitch it together with ffmpeg. Has the upside that a small project change won't need a complete rerender.

[Hideman85]

Yes that is one perspective they want to move to instead of parallel rendering, there is few topics about that already:
https://invent.kde.org/multimedia/kdenlive/-/issues/1868
https://invent.kde.org/multimedia/kdenlive/-/issues/439

[ddennedy]

There is not enough information here to reproduce this and be actionable. Something in your project triggers it. I understand that is not easy to figure out, but it is impossible for us to create every conceivable project and test it. Thus, this will be closed in the future if steps are provided that a developer can follow and reproduce.

[Hideman85]

Oh hello, I completely forgot I had open an issue on this. Actually it is a common problem, I remember that time I was trying to figure out something and there is plenty topics. If I remember it was with the use of melt, that in the end it has memory leaks with transitions and clips.
By that time the only working solution was "Yes it eats memory, you cant change it, either migrate to another tool or allocate swap", that what I did, 32G of swap was not enough, I ended up to create 120G of swap + my 32G of ram it did it in few hours.
Really the big issue wasnt speed at all, actually it was reasonable, but it was eating ram rapidly and hanging till OOM.
So I have no reproducible project, I guess use a few clips, use transition like fading/speed/blur/scaling and I guess you could get to it.
Also my project wasnt that big, the final video was 30min with maybe 20 clips, and 30 pictures. Maybe the harder part was the transitions with superposed clips & images.

Sorry to not be that useful but was literally a year ago.

[ddennedy]

"transitions and clips" is not it alone. I use Shotcut, which I also make and have not been affected. There might be some small memory leak there, but not enough to consume that much memory. Of course, if you make extremely complex projects at a high resolution, and use a lot of threads it does need significant memory. That is not necessarily a massive leak even if the memory appears to grow over time. Actually, it should grow and then level-off to a certain degree, but of course, it depends on many factors (the project could have some more complexity later in the project).

Another factor here is that in Shotcut, when making an export job, I add a property to all of the MLT <playlist> (i.e. timeline track as well as clip bin) elements: autoclose=1. It tells MLT to close each item when it's time has passed. It is only suitable for sequential processing such as in a render for export, not for interactive random access. I do not know if Kdenlive is doing that.