View my account

RGB depth synchronization issue on D455

Follow
Qi Yong219

When I don't use auto exposure, the RGB and depth timestamps obtained by wait_for_frames have a large time deviation, and this deviation drifts over time. How can I get perfectly time-synchronized data in non-auto exposure mode?  rs2::syncer has been tried but has no effect》

when use auto exposure, time diff between rgb and depth is:


But when using autoexposure， time diff is:

0

Comments

5 comments

Date Votes
  • MartyG

    Hi Qi Yong219  On the D455 model you can get perfect sync between depth and RGB if you use the 'RGB From Left Infrared Sensor' RGB mode instead of streaming from the RGB sensor.  This is done by enabling the Infrared stream and setting its format to RGB8 or BGR8 instead of the infrared stream's default Y8 format.

     

    The Left IR camera has the benefit of always 1. Being pixel-perfect aligned, calibrated, and overlapped with the depth map, 2. Perfectly time-synchronized, 3. Requires no additional computational overhead to align color-to-depth, and 4. Gives no additional occlusion artifacts, because it is exactly co-aligned. 

    0
    Comment actions Permalink
  • Qi Yong219

    The specific model I used is D455F. When acquiring the depth and Left Infrared image simultaneously, there is projected speckle in the image. This is unacceptable for a SLAM system. So is there any way to sync true rgb and depth?

    0
    Comment actions Permalink
  • MartyG

    RGB From Left Infrared has speckle on its RGB image because it overlays the invisible infrared dot pattern of the camera's IR projector.  You can remove the pattern by disabling the IR Emitter, though doing so can have a detrimental effect on depth image quality and make the infrared image darker unless the camera is used in a well-lit environment.

    0
    Comment actions Permalink
  • Qi Yong219

    I know all this. I want to get good quality RGB and depth. Is it impossible to do hardware synchronization between RGB and depth?

    0
    Comment actions Permalink
  • MartyG
    • Edited

    When auto-exposure is enabled, if the FPS of the depth and RGB streams is the same then sync between depth and RGB automatically kicks in.  You may be losing the benefit of this sync function by using manual exposure.

     

    The only alternative to RGB from Left Infrared  may be to use genlock hardware sync and sync the RGB and Depth of the camera to an externally generated sync trigger pulse using Inter Cam Sync Mode number '3'.  That mode was designed for the D415 camera model though (which has the depth and RGB sensors mounted together on the same circuit board), so the feature may not work well with a D455 type camera.

    0
    Comment actions Permalink

Please sign in to leave a comment.

Didn't find what you were looking for?

New post