stereo matching algorithm
Hello,
I'm interested in learning more about the feature matching technology algorithm used in the Intel RealSense D455 RGB-D camera. Specifically, I'd like to understand how the camera's stereo vision system byi itsdelf finds corresponding points between the left and right images.
If anyone has information or insights into the stereo matching algorithm employed by the RealSense D455 or can point me to relevant documentation, I would greatly appreciate it. I'm looking to explore its capabilities for a project I'm working on.
Thank you in advance for your assistance
-
Hi Dimouzoi There is limited information available about how RealSense cameras calculate a depth frame. The best available quote is on page 16 of the data sheet document for the RealSense 400 Series cameras.
"The stereo vision implementation consists of a left imager, right imager and an optional infrared projector. The infrared projector projects a non-visible static IR pattern to improve depth accuracy in scenes with low texture. The left and right imagers capture the scene and send imager data to the depth imaging (vision) processor, which calculates depth values for each pixel in the image by correlating points on the left image to the right image and via the shift between a point on the Left image and the Right image. The depth pixel values are processed to generate a depth frame. Subsequent depth frames create a depth video stream".
https://dev.intelrealsense.com/docs/intel-realsense-d400-series-product-family-datasheet
Intel's Beginner's Guide to Depth provides more general information about the principles of stereo depth.
-
The details of RealSense stereo algorithms are not open-source publicly available information, unfortunately.
Intel have another introductory depth guide that discusses stereo depth principles and SSD block-matching without talking specifically about the methods that RealSense cameras use.
-
MartyX Grover Is the Laser Power of the 400 Series in-built projector related to the Infrared? can we assume that this information is used to enhance the stereo matching algorithm?
-
Hi Javier The built-in projector is a separate component from the infrared sensors and projects a semi-random dot pattern onto the surfaces of objects in the real-world scene to aid the camera in performing depth analysis on those surfaces.
If the Laser Power value is increased (its default is '150' and its maximum is '360') then the amount of detail on the depth image may increase, whilst reducing the value below its default can reduce depth detail and cause holes and gaps in the depth image.
In regard to how Laser Power affects the infrared image, increasing Laser Power increases the brightness of the infrared dot pattern that is overlaid on the infrared image and makes the dot pattern more visible to the camera. Reducing the Laser Power reduces the visibility of the pattern, making it more difficult for the camera to analyze the dots and so also reducing the quality of the depth image.
-
Thanks for the clarification MartyX Grover
So if the infrared projector improves the accuracy of the stereo matching algorithm, can we add an external IR projector to improve depth capturing even more?
For example, could a standard IR Illuminator (e.g. a Long Range Infrared Flood Light for Security Camera) help?
-
Yes, you can add an external IR pattern projector that can enhance depth capture. Intel have a guide document at the link below for choosing and using an external projector for use with RealSense 400 Series cameras.
https://dev.intelrealsense.com/docs/projectors
You can also add a standard patternless IR illuminator lamp to a scene, as 400 Series cameras can make use of any visible or near-infrared light in the scene. This is especially beneficial in low-light or darkness settings More information about this can be found here:
https://github.com/IntelRealSense/librealsense/issues/2000
Please sign in to leave a comment.
Comments
8 comments