Is the infrared stream rectified or not?
Hi,
My question is regarding the infrared stream. Is it rectified or not using the following code:
cfg.enable_stream(RS2_STREAM_INFRARED, -1, 640, 480, RS2_FORMAT_Y8, 30);
rs2::pipeline_profile profile = m_pipeline.start(cfg);
rs2::frameset data = m_pipeline.wait_for_frames();
rs2::video_frame infrared = data.get_infrared_frame();
I found the rotation matrix from Infrared1 to Infrared 2 is an identity matrix as follows. Thus I guess the infrared stream should be the stream after rectification.
Extrinsic from "Infrared 1" To "Infrared 2" :
Rotation Matrix:
1 0 0
0 1 0
0 0 1
Translation Vector: -0.0500819087028503 0 0
Thanks.
Yixun
-
Hi Yixun Liu Most RealSense stream formats are rectified, including Y8 infrared. If you have a 400 Series camera model and a USB 3 connection then the infrared Y16 format is unrectified though, as it is used for camera calibration.
Y16 supports a more limited range of resolution and FPS combinations however. For example on a D455 camera model:
640x400, 15 or 25 FPS
1280x800, 15 or 25 FPS
-
By default the center-line of the left infrared sensor is the depth origin reference-point. If depth is aligned to color then the center-line of the color sensor becomes the origin.
The bottom of section 2-2 of Intel's Projection, Texture-Mapping and Occlusion white-paper document (linked to below) states: "On models with RGB sensor, for example, Intel RealSense D435 and D415 cameras, RGB do not have distortion parameters since the small distortion was determined to have insignificant performance impact and excluded in the calibration process. On models with larger RGB distortion, distortion parameters will be available".
On 400 Series camera models, the distortion coefficients are typically all set to zero. An exception to this is the color coefficients on the D455 model, which have non-zero coefficients.
-
I think I need to make my question clearer. In the following figure, I draw the coordinate spaces. For the left infrared camera, we have two coordinate spaces, one is the original space represented by the dashed line and the other is the rectified (or rotated) space represented by the solid line. It is the same for the right infrared camera. Let’s use the following naming:
Original left camera space: OriginalLeft
Rectified left camera space: RectifiedLeft
Original right camera space: OriginalRight
Rectified right camera space: RectifiedRight
My questions are:
1. Are the infrared streams (including both the left and right) reprojected onto the rectified image plane? Based on previous answer I think the answer should be Yes. I just want to get confirmation here
2. Is the depth stream defined in the RectifiedLeft space? Should be Yes I think.
3. For the color sensor (not depicted in the figure), what’s the extrinsic between the color and the depth sensor? From color sensor to the OriginalLeft or from color sensor to the rectifiedLeft?
Many thanks!
Yixun -
1. The section of Intel's introductory guide to stereo vision linked to below confirms that left and right original images are reprojected by the rectification process to a common virtual plane.
2. RealSense depth images are generated within the camera hardware from raw non-rectified left and right infrared frames (not the Infrared and Infrared 2 frames) at the point of capture at the camera lenses, and then the depth image has rectification applied to it and is transmitted along the cable to the computer. But since the left infrared stream and the depth stream are perfectly aligned to each other, it could be said that the depth image is equivalent to the 'RectifiedLeft' element on your diagram.
3. I would think that it would be RectifiedLeft but I'm not certain on that point.
Please sign in to leave a comment.
Comments
5 comments