Question on ffmpeg hanging on vmaf score calculations

[e-dong]

I found your older reddit post, I wonder if I'm running into this issue?
https://reddit.com/r/cloudygamer/s/ZfCkwPwNM5

The hanging is very intermittent. If I try rerunning the permutor-cli tool a few times, I can get through the entire run. Will post logs later and try running the ffmpeg command locally.

Output of failed:

Reading option '-' ... matched as output url.           Finished splitting the commandline.                     Parsing a group of options: global .
Applying option y (overwrite output files) with argument 1.                                                     Applying option report (generate a report) with argument 1.                                                     Applying option filter_complex (create a complex filtergraph) with argument libvmaf='n_threads=12:n_subsample=5'.                                                       Successfully parsed a group of options.
Parsing a group of options: input url tcp://localhost:2000?listen.                                              Applying option r (set frame rate (Hz value, fraction or abbreviation)) with argument 120.                      Successfully parsed a group of options.                 Opening an input file: tcp://localhost:2000?listen.
[NULL @ 0x5599f933b980] Opening 'tcp://localhost:2000?listen' for reading                                       [tcp @ 0x5599f9301540] No default whitelist set         tcp://localhost:2000?listen: Immediate exit requested   Exiting normally, received signal 2.

The exit signal is due to me hitting Ctrl + c

Expected output:

...
[out#0/null @ 0x55d609d8d300] Terminating muxer thread  frame= 2383 fps= 62 q=-0.0 Lsize=N/A time=00:00:19.85 bitrate=N/A speed=0.52x                                   video:1117kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown               Input file #0 (tcp://localhost:2000?listen):
  Input stream #0:0 (video): 2383 packets read (100636779 bytes); 2383 frames decoded;                            Total: 2383 packets (100636779 bytes) demuxed         Input file #1 (1080-120.y4m):                             Input stream #1:0 (video): 2383 packets read (7412083200 bytes); 2383 frames decoded;                           Total: 2383 packets (7412083200 bytes) demuxed        Output file #0 (pipe:):                                   Output stream #0:0 (video): 2383 frames encoded; 2383 packets muxed (1143840 bytes);
  Total: 2383 packets (1143840 bytes) muxed
4766 frames successfully decoded, 0 decoding errors     [Parsed_libvmaf_0 @ 0x55d609ec5680] VMAF score: 95.399699
[AVIOContext @ 0x55d609d47b80] Statistics: 100636779 bytes read, 0 seeks                                        [AVIOContext @ 0x55d609eb96c0] Statistics: 7412097579 bytes read, 0 seeks
,

[e-dong]

Hmm maybe a timeout can be added and have the script do some sort of retry logic?
https://stackoverflow.com/questions/52021958/how-to-stop-ffmpeg-from-running-if-connection-to-input-stream-is-lost

[Proryanator]

I think that would be a good idea yeah, hanging forever is not ideal 🤔

[e-dong]

I added this to the listen tcp url:
tcp://localhost:2000?listen&listen_timeout=3000&timeout=1000000 and I was able to get the script to stop execution:
listen_timeout is in millis and timeout is in microsecs
ref: https://ffmpeg.org/ffmpeg-protocols.html#tcp

Calculating vmaf score; might take longer than original encode depending on your CPU...
V: Vmaf args calculating quality: -y -report -r 120 -i tcp://localhost:2000?listen&listen_timeout=3000&timeout=1000000 -r 120 -i 1080-120.y4m -filter_complex libvmaf='n_threads=8:n_subsample=5' -f null -
V: ffmpeg args: [-y -report -r 120 -i tcp://localhost:2000?listen&listen_timeout=3000&timeout=1000000 -r 120 -i 1080-120.y4m -filter_complex libvmaf='n_threads=8:n_subsample=5' -f null -]
V: ffmpeg args no network calls (copy this and run locally, minus the quotes): [-y -report -r 120 -i tcp://localhost:2000?listen&listen_timeout=3000&timeout=1000000 -r 120 -i 1080-120.y4m -filter_complex libvmaf='n_threads=8:n_subsample=5' -f null -]
V: Encoder fmmpeg args sending to vmaf: -progress tcp://localhost:1234 -stats_period 0.5 -y -i 1080-120.y4m -b:v 30M -c:v hevc_nvenc -preset p3 -tune hq -profile:v main -rc cbr -cbr true -gpu 0 -b_ref_mode 0 -f hevc tcp://localhost:2000
V: ffmpeg args: [-progress tcp://localhost:1234 -stats_period 0.5 -y -i 1080-120.y4m -b:v 30M -c:v hevc_nvenc -preset p3 -tune hq -profile:v main -rc cbr -cbr true -gpu 0 -b_ref_mode 0 -f hevc tcp://localhost:2000]
V: ffmpeg args no network calls (copy this and run locally, minus the quotes): [-y -i 1080-120.y4m -b:v 30M -c:v hevc_nvenc -preset p3 -tune hq -profile:v main -rc cbr -cbr true -gpu 0 -b_ref_mode 0 -f null -]
  [00:00:10] [-------------------------------------------------------------------------------------------------------------------------------------------------------] 0/2383 frames (00:00:00)
VMAF calculation finishing up...
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: ParseFloatError { kind: Empty }', engine/src/permutation_engine.rs:159:41
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

[e-dong]

I'm gonna try debugging and testing this.

So far changing the threads for calculating the vmaf score to 1 looks promising but not ideal.

libvmaf='n_threads=1:n_subsample=5'

[Proryanator]

I think this definitely needs some more investigation 🤔 I wonder what the underlying cause of it to hang is.

[e-dong]

I haven't looked too deeply into this, but I noticed all the timeouts happen when there is no encoding happening. It appears the encoding process died and the listener just remains open. That was the reason why it hanged without the flags I added.

We would need to look more into why the encoder process died (Maybe connection refused error when the previous thread to calculate fps didn't exit?) Perhaps we can make the localhost port to encode the score different from the one that tests fps to test this theory.