Get undistorted RGB images from realsense
Hi,
Is it possible to get undistorted RGB images directly from realsense? It seems like by default realsense RGB images are distorted.
-
Hi Saeid. You can access raw RGB frames in RAW16 format from the camera hardware by converting them with an OpenCV cvtColor instruction into another color format, though the colorization may be a little off compared to the original.
-
Hi MartyG,
Thank you for your reply. We have calibrated the RGB intrinsics using RAW images and uploaded it to the camera but when we read the calibration information using pyrealsense we get non-zero distortion only for the RGB sensor (for IRs the distortion reported by pyrealsense is zero which is correct) implying that the sensor is not undistorting the RGB image using the uploaded calibration. I was wondering if there is a way to have the sensor internally undistort the RGB images?
-
Are you using a RealSense D455, please? 400 Series cameras usually have all 5 coefficients deliberately set to zero for reasons described in the link below.
https://github.com/IntelRealSense/librealsense/issues/1430#issuecomment-375945916
However, the color coefficients are non-zero on the D455 model, so if you have that model then it would be expected behaviour.
https://github.com/IntelRealSense/realsense-ros/issues/1580#issuecomment-748038697
OpenCV has undistort capabilities if you wish to investigate that possibility, as the SDK does not have an undistortion feature.
https://docs.opencv.org/4.5.2/dc/dbb/tutorial_py_calibration.html
A RealSense user in the link below tried an OpenCV undistort of a RealSense image but found that it made little difference though.
-
The distortion model is applied to the captured frames in the hardware of the camera and then they are transmitted onward through USB to the computer / computing device.
Y16 format, unlike the other stream types, is non-rectified because it is used for camera calibration. You could therefore try selecting Y16 as the RGB format instead of using raw RAW16 frames, though it is a monochrome format.
-
Thanks MartyG , I am just capturing regular rgb8 format images but the reported distortion coefficients for this format are non-zero.
-
I performed a check of my D415's calibration details with rs-enumerate-devices -c and the color coefficients on it were all zero. It may therefore be worth performing a Dynamic Calibration on your D415, which carries out a robust calibration of both the depth and RGB sensors.
The software can be downloaded for Windows from the link below.
If you are using Linux, page 14 onwards of the tool's user guide has installation instructions.
Please sign in to leave a comment.
Comments
7 comments