gstreamer 1.26 - fails to open camera

[ofTheo]

continuing the discussion in #6080

#6080 (comment)

[notice ] ofGstUtils: setPipelineWithSink(): gstreamer pipeline: v4l2src name=video_source device=/dev/video2 ! video/x-raw,format=DMA_DRM,width=640,height=360,framerate=30/1  ! videoscale method=2   ! appsink name=ofappsink enable-last-sample=0 caps="video/x-raw, format=RGB, width=640, height=480"
[ error ] ofGstUtils: setPipelineWithSink(): couldn't create pipeline: could not link video_source to videoscale0, video_source can't handle caps video/x-raw, format=(string)DMA_DRM, width=(int)640, height=(int)360, framerate=(fraction)30/1

[ofTheo]

@CarloCattano

Can you try setting a different format for the grabber.

	vidGrabber.setDeviceID(0);
	vidGrabber.setPixelFormat(OF_PIXELS_YUY2); <----------- try some of the different pixel formats
	vidGrabber.setDesiredFrameRate(30);
	vidGrabber.setup(camWidth, camHeight);

You can also list the available formats:

change /dev/video2 to the correct device

v4l2-ctl --device=/dev/video2 --list-formats-ext

[CarloCattano]

❯ v4l2-ctl --device=/dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
	Type: Video Capture

	[0]: 'MJPG' (Motion-JPEG, compressed)
		Size: Discrete 1920x1080
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 320x180
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 320x240
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 352x288
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 424x240
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 640x360
			Interval: Discrete 0.033s (30.000 fps)
                 etc etc
	[1]: 'YUYV' (YUYV 4:2:2)
		Size: Discrete 640x480
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 320x180
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 320x240
			Interval: Discrete 0.033s (30.000 fps)
                 etc etc 

So Im able to get the laptop infrared camera with OF_PIXELS_YUY2 for device id 1, as one would expect.

But if trying to get from device 0 with RGB ( which used to work )

[notice ] ofGstVideoGrabber: initGrabber(): selected device: Integrated Camera: Integrated C
[notice ] ofGstVideoGrabber: initGrabber(): selected format: 640x480 video/x-raw DMA_DRM framerate: 30/1
[notice ] ofGstUtils: setPipelineWithSink(): gstreamer pipeline: v4l2src name=video_source device=/dev/video0 ! video/x-raw,format=DMA_DRM,width=640,height=480,framerate=30/1  ! videoconvert   ! appsink name=ofappsink enable-last-sample=0 caps="video/x-raw, format=RGB, width=640, height=480"
[ error ] ofGstUtils: setPipelineWithSink(): couldn't create pipeline: could not link video_source to videoconvert0, video_source can't handle caps video/x-raw, format=(string)DMA_DRM, width=(int)640, height=(int)480, framerate=(fraction)30/1

Changes to videograbber example that trigger this message

    vidGrabber.setDeviceID(0);
    vidGrabber.setPixelFormat(OF_PIXELS_RGB);
    vidGrabber.setDesiredFrameRate(30);
    vidGrabber.setup(camWidth, camHeight);

[ofTheo]

@CarloCattano does OF_PIXELS_YUY2 work for device 0?

Can you try:
vidGrabber.setPixelFormat(OF_PIXELS_NATIVE);

for device 0 too?

[CarloCattano]

NATIVE pixels format segfaults for device 0

[notice ] 0: Integrated Camera: Integrated C
[notice ] 1: Integrated Camera: Integrated I
[notice ] ofGstUtils: setPipelineWithSink(): gstreamer pipeline: v4l2src name=video_source device=/dev/video0 ! video/x-raw,framerate=30/1  ! appsink name=ofappsink enable-last-sample=0 caps="video/x-raw,format={RGBA,BGRA,RGB,BGR,RGB16,GRAY8,YV12,I420,NV12,NV21,YUY2}, width=640, height=480"
[notice ] ofGstVideoUtils: allocating with 640x480 YUY2
[1]    3934 segmentation fault (core dumped)  env -u WAYLAND_DISPLAY bin/videoGrabberExample

@CarloCattano does OF_PIXELS_YUY2 work for device 0?
I get segfault too

[notice ] ofGstVideoGrabber: initGrabber(): selected device: Integrated Camera: Integrated C
[notice ] ofGstVideoGrabber: initGrabber(): selected format: 640x480 video/x-raw YUY2 framerate: 30/1
[notice ] ofGstUtils: setPipelineWithSink(): gstreamer pipeline: v4l2src name=video_source device=/dev/video0 ! video/x-raw,format=YUY2,width=640,height=480,framerate=30/1    ! appsink name=ofappsink enable-last-sample=0 caps="video/x-raw, format=YUY2, width=640, height=480"
[1]    5026 segmentation fault (core dumped)  env -u WAYLAND_DISPLAY bin/videoGrabberExample

[ofTheo]

Hmmm, that is interesting that it's not giving you an OF error just crashing.

One thing you can try is
gst-launch-1.0 v4l2src device=/dev/video0 ! autovideosink and see what formats it selects.

Also when running the video grabber example from the terminal you can set the GSTREAMER debug like:

export GST_DEBUG=3 
./videoGrabberExample

and it should show you more info about what is happening in gstreamer land

[ofTheo]

Some more info from ChatGPT:

**Yes — DMA_DRM is a relatively new video format in the GStreamer and Linux media ecosystem. It's tied to zero-copy buffer sharing via DMA-BUF, and DMA_DRM is essentially a caps format to indicate that buffers are in GPU-friendly memory — often for use with Wayland, KMS/DRM, or GPU-accelerated rendering.

✅ So, is DMA_DRM new?
Yes. Support for it has become more visible in recent GStreamer versions (like 1.24+ and especially 1.26) and in newer V4L2 drivers.
It often gets automatically chosen if your camera or system advertises support — even if it's partial or broken.**

🛑 Why is it problematic?
Many camera drivers do not fully support DMA_DRM, or only support it partially.
Some older cameras (especially UVC/USB) report support incorrectly.
It causes caps negotiation failures when other elements (like videoscale, appsink, etc.) don't accept or can't handle DMA_DRM.

That last line might be key as we use videoscale and appsink to grab pixels in OF

It doesn't look like there is a way to disable DMA_DRM suggestions I have seen involve specifying the format ( like YUV2 ).

I am guessing also this might be a Wayland issue - do you have a non Wayland OS you can try out easily?

cc @artificiel

[artificiel]

not running Wayland here, and a "quick try" turns out to be complicated (perhaps due to nvidia proprietary drivers). managed to get Wayland-gnome going, but OF seems unwilling to participate -- apps exit with ofAppGLFWWindow: X11: Platform not initialized errors even after clean rebuilds (incl. compiled of lib). I haven't read much on Wayland/OF (is it now officially supported?) maybe I'm missing some newbie thing but it's a rabbit hole I'm not running into now.

under Wayland, OBS does see the USB webcam and streams it in.

in case it helps:

[artificiel@assez-puissant videoGrabberExample]$ echo $XDG_SESSION_TYPE
wayland

[artificiel@assez-puissant videoGrabberExample]$ v4l2-ctl --device=/dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
	Type: Video Capture

	[0]: 'MJPG' (Motion-JPEG, compressed)
		Size: Discrete 1280x720
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 864x480
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.200s (5.000 fps)

[artificiel@assez-puissant videoGrabberExample]$ pacman -Qs Gstreamer
local/clutter-gst 3.0.27-5
    GStreamer bindings for clutter
local/gstreamer 1.26.0-3
    Multimedia graph framework - core
local/parole 4.18.2-1 (xfce4-goodies)
    Modern media player based on the GStreamer framework
local/qt6-multimedia-gstreamer 6.9.0-1
    Gstreamer backend for qt6-multimedia
local/totem 43.1+r37+gea6718427-1 (gnome)
    Movie player for the GNOME desktop based on GStreamer

[CarloCattano]

Running also on a KDE x11 session leads to same results, with the added gstreamer debug flag some extra info is presented

0:00:00.283859798  7118 0x5aa6483bad70 ERROR           GST_PIPELINE subprojects/gstreamer/gst/parse/grammar.y:1186:gst_parse_perform_link: could not link video_source to videoconvert0, video_source can't handle caps video/x-raw, format=(string)DMA_DRM, width=(int)640, height=(int)480, framerate=(fraction)30/1
[notice ] ofGstUtils: setPipelineWithSink(): gstreamer pipeline: v4l2src name=video_source device=/dev/video0 ! video/x-raw,format=DMA_DRM,width=640,height=480,framerate=30/1  ! videoconvert   ! appsink name=ofappsink enable-last-sample=0 caps="video/x-raw, format=RGB, width=640, height=480"
[ error ] ofGstUtils: setPipelineWithSink(): couldn't create pipeline: could not link video_source to videoconvert0, video_source can't handle caps video/x-raw, format=(string)DMA_DRM, width=(int)640, height=(int)480, framerate=(fraction)30/1

It seems to be a DMA_DRM problem indeed, and being something relatively new could explain the regression ( and the lack of information on the net about it )

 gst-launch-1.0 v4l2src device=/dev/video0 ! autovideosink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
0:00:00.131909609  7289 0x7fc8f0000b70 WARN                 basesrc gstbasesrc.c:3187:gst_base_src_loop:<v4l2src0> error: Internal data stream error.
0:00:00.131933946  7289 0x7fc8f0000b70 WARN                 basesrc gstbasesrc.c:3187:gst_base_src_loop:<v4l2src0> error: streaming stopped, reason not-negotiated (-4)
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
../gstreamer/subprojects/gstreamer/libs/gst/base/gstbasesrc.c(3187): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.000519532
Setting pipeline to NULL ...
Freeing pipeline ...

I confirm that in OBS it works, I get 2 devices one color camera stream and another with only infrared

[artificiel]

@CarloCattano at this point (since the problem manifests itself with a plain gst-launch command), perhaps solicit gstreamer for support? I guess your error above would be of interest to them.

if you can get gst-launch to work reliably, we can then figure out how to adapt ofGstVideoGrabber.cpp to negotiate a working pipeline. or if the conditional deductions are too complicated (or difficult to assert, not having a collection of hardware to test against), at least provide a settings flag like .disableDMA_DRM that would "blindly" insert something that prevents DMA_DRM (something that would be have been confirmed with gst-launch).

(ultimately OF should aim to support DMA_DRM but that will be a more complex "fix" than circumventing problematic pipelines to get your cam going)

[CarloCattano]

Yes indeed that last gst check suggested that, I did not encounter any other issue like this one before and was quite inclined to think it was a configuration error or gstreamer changes indeed.

Specially if no one else seems to have had this issue recently.

the hardware in my case its a ThinkPad P14s Gen 3 in case anyone lands on this closed issue with similar problems ( hope not ! )

thanks as usual for all the help

[kflak]

Possibly related: I could access my Thinkpad T480s' webcam just fine from my oF app, until I couldn't any more. If I run
gst-launch-1.0 -vvv v4l2src device=/dev/video0 ! autovideosink
everything works, but any other specification of the pipeline fails, including setting YUV2 as a format instead of DMA_DRM like this:

gst-launch-1.0 -vvv v4l2src name=video_source device=/dev/video0 ! video/x-raw,format=YUV2,width=640,height=480,framerate=30/1 ! videoconvert ! appsink name=ofappsink enable-last-sample=0 caps="video/x-raw, format=YUV2, width=640, height=480"

gives me the error:

WARNING: erroneous pipeline: could not link video_source to videoconvert0, neither element can handle caps video/x-raw, format=(string)YUV2, width=(int)640, height=(int)480, framerate=(fraction)30/1

Trying the "official" debugging techniques from gstreamer:

gst-launch-1.0 -vvv videotestsrc ! autovideosink
gst-launch-1.0 -vvv videotestsrc ! 'video/x-raw,width=1280,height=720,format=RGB,framerate=60/1' ! fpsdisplaysink
gst-launch-1.0 -vvv videotestsrc ! videoconvert ! fpsdisplaysink text-overlay=false

The first and last work, the second one doesn't.

Seeing as everything works fine if I run autovideosink, I was wondering if there is a way to have oF simply send out that command instead, as this does exactly what I need it to do (including the correct width, height, aspect ratio, frame rate, etc.)?

[artificiel]

@kflak if you look into ofGstVideoGrabber.cpp in the setup() you'll see how the GST pipeline is assembled. for a start you can hack in your working command to get things working in your specific conditions, and then perhaps figure out why it's is unable to figure "the right thing". that class is not getting any younger, and might need a good revisit in light of the recent GST evolution.

a way of passing a "raw" std::string gstPipeline would probably help edge case (at least explore variations of the pipeline without recompiling).