D455 RGB streaming
Hello, I want to stream the RGB camera of my D455 to a screen using MJPEG.
I have managed to do this previously with the D415, specifying yuv422 as the pixel format and 1280x720 as resolution. But while doing the same with the D455, I get the error the of "Deprecated pixel format" from ffmpeg. From the product specification both of these cameras stream RGB in YUV2 format, and should be able in 1280x720. Am I missing something? Do you have any advice in getting an RGB stream to work with the D455 and what could differ from D415?
Thank you in Advance!
-
Hi Liam Neric You are correct, both the D415 and D455 camera models use the YUY2 RGB color format, as shown in the table below.
However, YUY2 is the raw image format that the cameras generate RGB frames in. The final image that is rectified and output from the cameras as a stream is not in YUY2 format. It can be in a range of formats, the most commonly used of which are RGB8 and BGR8.
D415 and D455 have a mostly common shared list of supported RGB output formats. However, D415 supports YUYV as output whilst D455 does not.
-
It s possible for non-RealSense tools to capture raw data from the camera if the RealSense SDK is built in 'native' mode, as described at the link below.
https://github.com/IntelRealSense/librealsense/issues/6841#issuecomment-660859774
-
Thank you, this information was very useful.
I'm a little bit confused with the RGB/YUV2. If I run the realsense camera as a webcam, and request a videostream from the RGB camera at for example dev/video4, what is the returned pixel format then? Is it possible to configure this output format, and in that case how?
The link you sent was also very interesting. What is the advantage of capturing raw data, wouldn't the frames just be a bigger size and worse color quality according to the comment? And by building with this parameter, is any functionality in the pyrealsense SDK lost?
Thank you.
-
By default, the /video channels of a RealSense 400 Series camera are depth = /video0, infrared = /video1 and RGB = /video2. However, the /video number assigned to a device may change depending what other video devices are attached to the computer. It is possible to set fixed addresses for devices with symbolic links via udev device-handling rules.
https://github.com/IntelRealSense/librealsense/issues/11553#issuecomment-1463442865
The default output format of RGB on a RealSense camera should be RGB8. You can specify another format from the list of supported formats for a particular camera in a RealSense program script or the RealSense Viewer tool.
The main difference between raw frames and the final output frames is that the raw frames will be undistorted (the final frames have a Brown-Conrady distortion model applied to them) and the coloring may also be weird.
Here is a link to a discussion about accessing raw frames via the RealSense SDK with Python code by converting raw YUY2 to an SDK-usable format, where the color difference after conversion becomes apparent.
https://github.com/IntelRealSense/librealsense/issues/7275
-
Regarding pyrealsense2 functionality: no, there is not any functionality lost from it when building in native mode. It can just make installation of the SDK a little more complex, as you have to use a kernel version on your computer that is supported by the SDK and apply a patch script to the kernel.
Please sign in to leave a comment.
Comments
6 comments