FFplay's video_thread deadlocks in frame_queue_peek_writable after queuing a few frames

[jakogut]

This seems to be an issue with Emscripten's pthread implementation which causes FFplay to deadlock while queuing frames. Switching to the generic SDL2 thread backend, which implements mutexes on top of semaphores, fixes the issue, as documented in PR #8429.

Running FFplay requires this patch to Emscripten's port of SDL2:
emscripten-ports/SDL2#77

To reproduce this problem, clone or download FFmpeg sources, and configure with:

mkdir prefix
emconfigure ./configure --prefix=prefix --disable-doc --disable-asm --disable-programs --enable-ffplay --enable-cross-compile --cc=emcc --disable-runtime-cpudetect --disable-fast-unaligned --disable-hwaccels --disable-network --disable-stripping

Patch ffplay to work around issue #7684

diff --git a/fftools/ffplay.c b/fftools/ffplay.c
index 8f050e16e6..b5e42cdbb5 100644
--- a/fftools/ffplay.c
+++ b/fftools/ffplay.c
@@ -53,6 +53,23 @@
 # include "libavfilter/buffersrc.h"
 #endif
 
+#ifdef __EMSCRIPTEN__
+#include <emscripten.h>
+#include <emscripten/threading.h>
+#include <emscripten/html5.h>
+
+static void create_webgl_context()
+{
+       EmscriptenWebGLContextAttributes attr;
+       emscripten_webgl_init_context_attributes(&attr);
+       attr.majorVersion = 2; attr.minorVersion = 0;
+       EMSCRIPTEN_WEBGL_CONTEXT_HANDLE ctx = emscripten_webgl_create_context(0, &attr);
+       emscripten_webgl_make_context_current(ctx);
+       printf("created webgl context\n");
+}
+
+#endif // __EMSCRIPTEN__
+
 #include <SDL.h>
 #include <SDL_thread.h>
 
@@ -3663,6 +3680,10 @@ void show_help_default(const char *opt, const char *arg)
 /* Called from the main */
 int main(int argc, char **argv)
 {
+#ifdef __EMSCRIPTEN__
+    create_webgl_context();
+#endif
+
     int flags;
     VideoState *is;
 

Make the install target:

make install

Symlink ffplay in the bin directory to ffplay.bc

ln -sf $(readlink -f prefix/bin/ffplay) prefix/bin/ffplay.bc

Download a video sample:

wget https://raw.githubusercontent.com/mediaelement/mediaelement-files/master/big_buck_bunny.webm

Build ffplay using Emscripten:

emcc prefix/bin/ffplay.bc -o ffplay.html -Lprefix/lib -lavfilter -lswscale -lpostproc -lavformat -lavcodec -lswresample -lavresample -lavutil -s TOTAL_MEMORY=$((512*1024*1024)) --emrun -s USE_PTHREADS=1 -s PROXY_TO_PTHREAD=1 -s USE_SDL=2 --embed-file big_buck_bunny.webm@/big_buck_bunny.webm

Run FFplay:

emrun ffplay.html -an big_buck_bunny.webm

The expected behavior is for the video to play to completion, but it stops after two or three frames. With the workaround from above, the video plays to completion. This may indicate a bug in Emscripten's implementation of the pthread API.

[jakogut]

I'm also working on turning this into a test.

[kripken]

Does this still happen with -s PROXY_TO_PTHREAD=1? That avoids deadlocks related to the main thread.

[jakogut]

@kripken Yes, it still happens with -s PROXY_TO_PTHREAD=1. I actually use that in the example provided above to allow the (mostly) unmodified ffplay app to run without returning control to the JavaScript event loop.

[kripken]

I see, thanks. Then I don't have a guess as to what might cause this. Perhaps @juj has an idea?

I wonder if we should spend time to improve tools in this area, like we have the AUTODEBUG and DETERMINISTIC options that make debugging many bugs really easy, but for multithreaded code those won't really help...

[juj]

Not exactly sure what would be going on here. Emscripten has a linker flag --threadprofiler that I originally created to help debug a related kind of thread issues, perhaps that could be used as a basis to debug what the threads are doing. Having a callstack from each of the deadlocked thread to where they lock up might be helpful.

[stale]

This issue has been automatically marked as stale because there has been no activity in the past year. It will be closed automatically if no further activity occurs in the next 7 days. Feel free to re-open at any time if this issue is still relevant.

[jakogut]

I can verify that this issue is still present, and that the original workaround is still relevant, which is a nice clue. Could we reopen this?

[stale]

This issue has been automatically marked as stale because there has been no activity in the past year. It will be closed automatically if no further activity occurs in the next 30 days. Feel free to re-open at any time if this issue is still relevant.