I am using a Raspberry Pi 4 with the Raspberry Pi NoIR Camera v2. The camera works perfectly with libcamera commands, and I can view the video stream without any issues. However, when trying to use the camera with OpenCV via GStreamer, I encounter problems. Specifically, the preview window opens, but it shows a black screen or fails to start the pipeline entirely.
For reference, I am running Raspberry Pi OS (Bookworm) with Python 3.11.2, OpenCV 4.6.0, and GStreamer 1.22.0. I have confirmed that OpenCV is built with GStreamer support. Additionally, I have verified that all relevant GStreamer plugins are installed, including gstreamer1.0-tools, gstreamer1.0-plugins-base, gstreamer1.0-plugins-good, gstreamer1.0-plugins-bad, and gstreamer1.0-plugins-ugly.
When using libcamera commands, the camera works without any issues. For example, I can preview the video feed using the command:
libcamera-hello
Additionally, I can capture videos using libcamera-vid with a command like:
libcamera-vid --inline --codec yuv420 --width 640 --height 480 --framerate 30 -o test.h264
These commands successfully display or save the video output.
I also tested the camera directly with GStreamer, and it works as expected. For instance, the following command opens a live video feed:
gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw, format=YUY2, width=640, height=480, framerate=30/1 ! videoconvert ! autovideosink
This shows that the camera is functional and GStreamer is configured correctly.
However, when integrating GStreamer with OpenCV, the camera feed does not work. For example, the following Python script opens a window but only displays a black screen:
import cv2
pipeline = "v4l2src device=/dev/video0 ! video/x-raw, format=YUY2, width=640, height=480, framerate=30/1 ! videoconvert ! appsink"
cap = cv2.VideoCapture(pipeline, cv2.CAP_GSTREAMER)
if not cap.isOpened():
print("Failed to open camera")
else:
ret, frame = cap.read()
if ret:
cv2.imshow("Camera", frame)
cv2.waitKey(0)
cap.release()
cv2.destroyAllWindows()
The script does not throw any errors but fails to display the camera feed. The debug output mentions:
OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
I also tried piping libcamera output to GStreamer using the following command:
libcamera-vid --inline --codec yuv420 --width 640 --height 480 --framerate 30 -o - | gst-launch-1.0 fdsrc ! videoconvert ! autovideosink
ERROR: pipeline doesn't want to preroll.
have verified the camera's supported formats using v4l2-ctl and confirmed that YUY2 and other common formats are supported. Memory usage has been checked with free -h, showing sufficient available memory (~2GB). I have also tried different resolutions, formats, and framerates without success.
My question is: why does the camera work seamlessly with libcamera and GStreamer individually, but not with OpenCV using GStreamer? Are there additional dependencies, configurations, or limitations related to the Raspberry Pi NoIR Camera v2 that I might be overlooking? I need to use it with OpenCV via GStreamer.
Any guidance or suggestions to resolve this issue would be greatly appreciated. Thank you in advance for your help!
For reference, I am running Raspberry Pi OS (Bookworm) with Python 3.11.2, OpenCV 4.6.0, and GStreamer 1.22.0. I have confirmed that OpenCV is built with GStreamer support. Additionally, I have verified that all relevant GStreamer plugins are installed, including gstreamer1.0-tools, gstreamer1.0-plugins-base, gstreamer1.0-plugins-good, gstreamer1.0-plugins-bad, and gstreamer1.0-plugins-ugly.
When using libcamera commands, the camera works without any issues. For example, I can preview the video feed using the command:
libcamera-hello
Additionally, I can capture videos using libcamera-vid with a command like:
libcamera-vid --inline --codec yuv420 --width 640 --height 480 --framerate 30 -o test.h264
These commands successfully display or save the video output.
I also tested the camera directly with GStreamer, and it works as expected. For instance, the following command opens a live video feed:
gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw, format=YUY2, width=640, height=480, framerate=30/1 ! videoconvert ! autovideosink
This shows that the camera is functional and GStreamer is configured correctly.
However, when integrating GStreamer with OpenCV, the camera feed does not work. For example, the following Python script opens a window but only displays a black screen:
import cv2
pipeline = "v4l2src device=/dev/video0 ! video/x-raw, format=YUY2, width=640, height=480, framerate=30/1 ! videoconvert ! appsink"
cap = cv2.VideoCapture(pipeline, cv2.CAP_GSTREAMER)
if not cap.isOpened():
print("Failed to open camera")
else:
ret, frame = cap.read()
if ret:
cv2.imshow("Camera", frame)
cv2.waitKey(0)
cap.release()
cv2.destroyAllWindows()
The script does not throw any errors but fails to display the camera feed. The debug output mentions:
OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
I also tried piping libcamera output to GStreamer using the following command:
libcamera-vid --inline --codec yuv420 --width 640 --height 480 --framerate 30 -o - | gst-launch-1.0 fdsrc ! videoconvert ! autovideosink
ERROR: pipeline doesn't want to preroll.
have verified the camera's supported formats using v4l2-ctl and confirmed that YUY2 and other common formats are supported. Memory usage has been checked with free -h, showing sufficient available memory (~2GB). I have also tried different resolutions, formats, and framerates without success.
My question is: why does the camera work seamlessly with libcamera and GStreamer individually, but not with OpenCV using GStreamer? Are there additional dependencies, configurations, or limitations related to the Raspberry Pi NoIR Camera v2 that I might be overlooking? I need to use it with OpenCV via GStreamer.
Any guidance or suggestions to resolve this issue would be greatly appreciated. Thank you in advance for your help!
Statistics: Posted by GentlemanBear — Mon Dec 09, 2024 4:16 pm