D455 RBG and 3D - aglining points
I have an application where I find specific poits on 2D RGB image. I want to align the points to the 3D data. However, When I compare the RBG view and the Depth Image view, the points are moved and I cannot get appropriate results. Ho to algn the data properly?
-
Hi Patrycja Grabowska When you perform depth to color alignment, are you using the color intrinsics please? Using the depth intrinsics for alignment can result in inaccuracy, because after alignment the origin point has changed from the center-line of the left IR sensor of the camera to the center-line of the RGB sensor.
You can obtain perfect alignment between depth and color if you use the D455 camera model's alternative 'RGB color from left infrared sensor mode' described at the link below instead of the usual RGB stream.
https://dev.realsenseai.com/docs/tuning-depth-cameras-for-best-performance#use-the-left-color-camera
You can obtain the RGB from Left IR image by enabling the infrared stream instead of the RGB stream and setting its format to RGB8 or BGR8 instead of the default Y8 format of infrared.
-
Hello MartyX Grover! Thank you for your fast reply. Please be informed that thr application requires using the color camera to find the desired object and its points. The indpection cannot be done with the infrared stream. Could you help me with aligning the points?
-
Although the RGB from Left Infrared image is coming from the infrared sensor instead of the RGB sensor, its image format will still be RGB like the main RGB stream when the Infrared stream's format is set to RGB8 or BGR8.
The depth image comes from the left infrared sensor. So if the RGB stream is used, its viewpoint will be horizontally offset from the depth stream's viewpoint by a small amount because the RGB sensor is physically a small distance away from the left infrared sensor on the front of the camera.
Unless the RGB from Left Infrared image is used (where RGB is perfectly aligned with depth), if you want to align depth and color as points on a 3D image then it is necessary to use depth to color alignment to map the depth stream's points onto the color coordinates of the RGB stream to produce an aligned 'textured pointcloud'.
The RealSense SDK provides a couple of examples of program scripts for generating an aligned pointcloud where depth and RGB are mapped together,
C++
https://github.com/IntelRealSense/librealsense/tree/master/examples/pointcloud
Python
If you want to align depth and color put not produce a 3D pointcloud then there are example programs for that too.
C++
https://github.com/IntelRealSense/librealsense/tree/master/examples/align
Python
-
Hello MartyX Grover, thank you for your message and your support. Best regards
Please sign in to leave a comment.
Comments
4 comments