Depth maps
Hello.
We are working with a D405 camera and having problems with depth measurement due to the poor surface texture.
Looking at the RealSense software and using Python with deprojection method and the instrinsic matrix given by the SDK, the depth measurement has a variation in the same point of up to 5mm at 225mm height, which is a high variation when performing a hand eye calibration. (See the following image, which is a heat map of the maximum variation in depth measurement between 20 images of the same point, for all points of an 8x7 chessboard) (Maximum variation in measurement is the difference between the maximum and minimum measurement for the same point, neither the camera nor the chessboard move during the process ) 
To try to solve the situation, we implemented the OpenCV PnP method, which has a high repeatability, and gives the same measurement with variations of less than 1mm for each point. The problem with this method is that gives a difference of about 6-7mm from the distance that gives RealSense software and Python.
The starting point in the camera's depth measurement according to the datasheet is at 3.7mm inside the front glass, so I can't make sense of the difference I'm getting.
Has anyone had this problem before?
Could I calibrate using the PnP method and then adjust the 6-7mm difference when sending the point to the robot?
Thanks
-
Hi F Ricardo Auto With RealSense cameras that have front glass, depth is measured from the glass and not at the lenses. You can subtract 3.7 mm from the depth value to find the ground truth depth value if you wish to but it is usually not necessary to do so.
The D405 camera model is already highly accurate, but if you wish to improve accuracy further then instead of using PNP, you could try setting the High Accuracy camera configuration preset using Python code at the link below.
https://github.com/IntelRealSense/librealsense/issues/2577#issuecomment-432137634
-
Hello MartyX Grover, thank you very much for the help...
About the accuracy of the measurements, the problem was with the intrinsic matrix, I ran the Dynamic Calibrator twice, once on the mobile App pattern and once on a printed one, and the result was the same, but different from the one obtained from the camera using "get_intrinsics()" function. Once the matrix was changed, the results improved!!!
The results were even better using the High Accuracy you suggested.
But, we are still having some error at points away from the center of the image, and we would like to know if you think that introducing distortion in the calculations can help to smooth this error... We have the distortion parameters from the calibration:
Thank you very much!!!
-
The RealSense depth stream already has a distortion model applied to it, so you should not need to add more distortion.
A RealSense user at the link below who was using a chessboard tried using OpenCV's cv::undistort function to remove the distortion that had been applied to the data.
https://github.com/IntelRealSense/librealsense/issues/3880
If you have error at coordinates away from the center of the image though then that sounds like what would happen if you were using depth-color alignment in combination with the GetDistance() command that retrieves the distance of a coordinate in meters. Is that what your project is doing, please? If it is then the error can be reduced either by using aligned intrinsics or by using the intrinsics of the stream that you are aligning to - for example, using color intrinsics if you are aligning depth to color.
-
Hello MartyG
We are not using alignment, after enable both streams, we take color frame and depth frame and does not apply alignment. Should we do it?? I read a previous post about this and understand that in D405 camera it is not neccesary, am I wrong??
I select the point coordinates (x,y) from the color frame, and do the deprojection with the value of the depth frame at that point.
-
The phenomenon of depth values increasing in error as the coordinates move away from the center should not occur if depth-color alignment is not being performed.
On the D405, depth and color do originate from the same sensor and so have the same field of view and should in theory be perfectly aligned to each other. However, the RGB image then undergoes further processing in an Image Signal Processor (ISP) chip in the camera hardware to produce the final RGB image.
Instead of using the RGB image, you could try using the RGB From Left Infrared Sensor image instead. This is retrieved by using the Infrared stream instead of the RGB stream, but setting its format to BGR8 in instead of infrared Y8. The left infrared RGB stream will not have undergone processing in the ISP chip.
Please sign in to leave a comment.
Comments
5 comments