View my account

some information needed about D455

Follow
Zahid Iqbal

Hi,

As a new person to the computer vision field, I recently started using the "intel realsense 455" camera. I can see, there appear to be 4 sensors on the camera. I understand, one of these is to capture the RGB frames, and one is to capture the depth data. What are the other two sensors for. Or there are two sensors to capture the RGB frames. Can please tell me the physical location and type for each of the sensors.

Another question I have is concerning calibration. I run the "realsense-viewer" on terminal, and tried to use the "On-Chip" calibration feature. It returns a value of -0.07 (Good). Another time, it returned -0.14. Does this mean, I do not need to employ different methods (software based) to calibrate the camera? Examples of these methods are the ones that used ChessBoard etc to perform camera calibration.

Using openCV and realsense sdk which functions are available to perform the camera pose-estimation.

I will post other questions later.

thanks,

 

0

Comments

5 comments

Date Votes
  • MartyG

    Looking at the D455 camera model from the front, the order from left to right is: Right Infrared sensor, RGB sensor, Infrared Projector and Left Infrared sensor. There is a diagram on page 144 of the current edition of the data sheet document for the 400 Series cameras.

    The Projector is not a sensor, so there are three sensors on the camera. The Projector, when enabled, casts a semi random invisible pattern of dots onto the surfaces of objects in the real-world scene to aid the camera's depth analysis.

    In regard to on-chip calibration, the lower the health score the better the calibration is, so -0.07 will be a better result than -0.14. The tool encourages users to perform repeated calibrations to obtain the best possible health check score and then save a particular calibration to the camera hardware once satisfied with the score achieved.

    In regard to pose estimation with RealSense, the link below discusses this subject.
    https://github.com/IntelRealSense/librealsense/issues/7560
    The official OpenCV documentation has a useful tutorial for pose estimation with a chessboard.
    https://docs.opencv.org/3.4/d7/d53/tutorial_py_pose.html

    https://dev.intelrealsense.com/docs/intel-realsense-d400-series-product-family-datasheet

    0
    Comment actions Permalink
  • Zahid Iqbal

    thank you @MartyG.

    I have some further questions that want to clarify.

    When we access the calibration table from realsense-viewer, there are different data corresponding to different resolutions. I understand, different resolutions belong to different image sizes, and consequently the RGB stream. 


    (1280,720), (1920,1080), (640,480), (848, 480), (640,360), (1280,720), (424, 240), (320,240), (480,270), (1280,800), (960,540), (640,400), (576,576), (720,720), (1152,1152)

    Within a program, I printed the intrinsic parameters of the RGB stream in the following manner:

    selection = pipe.start();
    auto color_stream = selection.get_stream(RS2_STREAM_COLOR).as<rs2::video_stream_profile>();
    auto resolution = std::make_pair(color_stream.width(), color_stream.height());

    auto i = color_stream.get_intrinsics();
    auto principal_point = std::make_pair(i.ppx, i.ppy);
    auto focal_length = std::make_pair(i.fx, i.fy);
    rs2_distortion model = i.model;
    //print the parameters
    cout << "Resolution: " << resolution.first << "," << resolution.second << endl;
    cout << "Principle point: " << principal_point.first << "," << principal_point.second << endl;
    cout << "Focal length: " << focal_length.first << "," << focal_length.second << endl;
    cout << "distortion model: " << model << endl;

     

    And, I receive the following output:

    Resolution: 1280,720
    Principle point: 652.595,360.33
    Focal length: 641.595,640.86
    distortion model: Inverse Brown Conrady

    Is the fault RGB stream always (1280,720) ? 

    When I run "realsense-viewer" on the terminal, the RGB camera by default is at (1280, 720). However, Focal length it states (659.186, 659.186) and the principal point is (646.592,353.885). Or does that correspond to a different sensor.

    Another useful command that was mentioned elsewhere is "rs-enumerate-devices -c".  This command prints intrinsic and extrinsic data from different streams. Here corresponding to different resolutions, there are intrinsic of "Color", "Depth", "Infrared", "Infrared-1", and "Infrared-2".  Any comments about that?

    In order for pose estimation programs,  the calibration data ( "calibration matrix, and distortion coefficients") that we use has to be the from an RGB stream.


    thanks,

    0
    Comment actions Permalink
  • MartyG

    As shown by rs-enumerate-devices -c, each resolution has different intrinsics.

     

    Unless you define your own stream configurations with cfg instructions, a program will use the default stream profiles of the camera.  On most of the RealSense 400 Series camera models including D455, this will be 848x480 depth and 1280x720 RGB.

     

    On the D455 camera model, depth and infrared will have all five distortion coefficients set to zero.  The D455's color though uses non-zero coefficients.

     

    0
    Comment actions Permalink
  • Zahid Iqbal

    thank you MartyG.

    So the values PPX, PPY, FX, Fy (Principal point, focal lengths) are specified in millimeter here? But, the distortion coefficients are already in meters.

    Just to confirm. 

     

    0
    Comment actions Permalink
  • MartyG

    The fx and fy fields describe the focal length of the image, as a multiple of pixel width and height.  The ppx and ppy fields describe the pixel coordinates of the principal point (center of projection)

    https://dev.intelrealsense.com/docs/projection-in-intel-realsense-sdk-20#intrinsic-camera-parameters

    0
    Comment actions Permalink

Please sign in to leave a comment.

Didn't find what you were looking for?

New post