D455 RGB frames completely black when using hw sync
Hello,
We are using 2 d455 to get the depth images. We use hw sync, where one of the cameras is set to "inter_cam_sync_mode = 1" and the other to "inter_cam_sync_mode = 2". We need to also be able to get the color images from both cameras but we only get completely black frames (all pixel values are 0). Both cameras work correctly when used without hw sync (cable disconnected and inter_cam_sync_mode = 0).
We are aware that the color images cannot be synchronized using hw sync but we expected to still be able to get color frames. Any insight as to what could be causing the issue? How can we synchronize the depth frame and get the color frames as well? Is it possible to synchronize both using another synchronization method (genlock)?
relevant code:
int CameraRealsense::openPipeline(rs2::context context, rs2::device cameraDevice)
{
device=cameraDevice;
sensors=device.query_sensors();
setAdvanceConfiguration();
setCameraOption();
setAutoExposureROI();
serialNumber=device.get_info(RS2_CAMERA_INFO_SERIAL_NUMBER);
pipeline=rs2::pipeline(context);
rs2::configcfg;
cfg.enable_device(serialNumber);
cfg.enable_stream(RS2_STREAM_DEPTH, 848, 480, RS2_FORMAT_Z16, 5);
cfg.enable_stream(RS2_STREAM_COLOR, 640, 480, RS2_FORMAT_BGR8, 5);
pipeline.start(cfg);
cameraPipelineOpened = true;
return 0;
}
int CameraRealsense::readFrameset()
{
rs2::frameset newFrameSet = rs2::frameset();
if (!cameraPipelineOpened)
{
RCLCPP_WARN_THROTTLE(node->get_logger(), *node->get_clock(), 1000,
"%s: unable to read frameset: pipeline not opened", name.c_str());
return -1;
}
try
{
newFrameSet = pipeline.wait_for_frames(200);
}
catch (const std::exception& e)
{
RCLCPP_WARN(node->get_logger(), "%s: failed to retrieve the frameset (%s)", name.c_str(), e.what());
return -2;
}
frameSet = decimationFilter->process(newFrameSet);
return 0;
}
rs2::video_frame CameraRealsense::getBgrVideoFrame()
{
return frameSet.get_color_frame();
}
Thank you in advance for your assistance!
-
Hi Theo Engels How does the RGB behave if you change the slave camera from '2' (Slave) to '3' (Full Slave).
The Full Slave mode is part of the genlock system and was designed to sync both depth and RGB. Intel no longer provide support for genlock mode (as it was a proof-of-concept experiment) and modes 1 and 2 are now the only officially supported ones but the genlock modes are still accessible in the RealSense SDK.
-
Hi MartyG, thanks for your response.
Changing the slave camera to full slave mode correctly generates RGB images for the slave camera, but the issue persists for the master camera. I managed to generate rgb images on both cameras by increasing the fps of the RGB stream to 15 from 5, but it does not generate synchronized depth/RGB images (up to a 130ms difference between the timestamps of RGB and depth frames), even on the full slave camera.
Is there anything you could suggest to sync both the cameras and the streams together?
-
The Depth and RGB full slave mode was intended for the D415 camera model, where the depth and RGB sensors are mounted on the same circuit board. The depth-RGB sync did not work well in general though.
If depth and RGB FPS are the same then the camera should automatically attempt to sync the depth and RGB frames on the same camera. The best way to achieve this is to disable an RGB option called Auto-Exposure Priority so that the RealSense SDK tries to enforce a constant FPS for both streams instead of permitting FPS to vary. C++ code for disabling auto-exposure priority can be found at the link below.
-
Thanks for the lin MartyG
I disabled auto exposure priority and used 5 fps for both streams and still get fully black rgb frames.
sensors[RGB_SENSOR_IDX].set_option(RS2_OPTION_AUTO_EXPOSURE_PRIORITY, 0);
Please sign in to leave a comment.
Comments
11 comments