Best Approach for Synchronizing 4 Intel RealSense D455 Cameras on 4 Independent Computers
Hello,
We are currently using 4 Intel RealSense D455 cameras, each connected to a separate computer, to record high-resolution RGB-D data. While the official documentation primarily focuses on multi-camera synchronization on a single PC, it also acknowledges the severe frame drops that occur when recording at 30FPS with multiple cameras due to USB bandwidth limitations.
To overcome this issue, we opted for 4 independent computers, but we are now facing challenges in achieving precise multi-camera synchronization.
Current Approaches We Have Tried
1. Software Time Synchronization (PTPD / NTP)
- We attempted PTP (Precision Time Protocol) and NTP (Network Time Protocol) to synchronize the system clocks of the 4 computers.
- However, time drift and network latency introduce synchronization errors at the millisecond level, which is insufficient for frame-level alignment.
Questions
- Does Intel provide an official recommendation for synchronizing RealSense D455 cameras across multiple independent computers?
- Is it possible to achieve true frame-level synchronization using external trigger signals (e.g., hardware triggers)?
Any insights or suggestions from experienced developers or Intel’s team would be greatly appreciated!
Thanks in advance! 🚀
-
Hi Charles Wd It sounds as though you are referring to the guidance in the Multi-Camera Considerations section of the multiple camera white-paper guide document.
This guide was originally written in 2018 and the test results in the USB bandwidth charts, which have not been updated since then, will reflect the computer technology of the time. There have been significant advances in PC technology since then, especially with CPU processor chips.
However, the Bandwidth section of the guide states though that because there is an approximate 30% processing overhead, it is recommended not to exceed 3.2 gigabits per second (Gpbs) of total USB bandwidth out of the 5 Gpbs available in order to ensure robust streaming. When using high resolution depth and color, then, spreading the cameras across multiple individual computers would make sense.
Intel demonstrated this themselves in January 2018 with a 4-camera D435 'volumetric capture' setup at the Sundance film festival, where 4 cameras on 4 PCs were networked together and transferred their data to a fifth PC that performed final processing of the raw data from all of the cameras.
https://www.intelrealsense.com/intel-realsense-volumetric-capture/
In regard to your questions:
1. Aside from this Sundance article, the only example of official documentation I am aware of for using multiple cameras across more than one PC is a guide at the link below for doing so with ROS. This guide was written for the old ROS1 though and so is not directly usable for the modern ROS2, at least not without modifications to the procedure. The cameras in this guide also do not use hardware sync.
https://github.com/IntelRealSense/realsense-ros/wiki/showcase-of-using-3-cameras-in-2-machines
An unofficial reference to multiple camera networking on different computers is a project created and shared by the CONIX Research Center at Carnegie Mellon for networking together up to 20 RealSense 400 Series cameras, one on each computer, in order to stitch the data together into a single large pointcloud.
https://github.com/conix-center/pointcloud_stitching
2. A limitation of using external triggers with the officially supported form of the multiple-camera hardware sync system (Inter Cam Sync Mode 1 and 2) is that the frequency of the trigger pulse needs to be matched very precisely to the actual FPS of the cameras (such as 30.015 FPS), which is very difficult.
Intel later introduced a new experimental 'proof of concept' hardware sync system called Genlock that makes external trigger control much easier. As this was an experimental system, Intel no longer support Genlock mode, just the original mode 1 and 2. The Genlock modes (Inter Cam Sync Mode 4 and above) remain accessible in the RealSense SDK though.
The documentation for Genlock is no longer available online but you can download an archived PDF version of it in your browser here:
https://github.com/IntelRealSense/librealsense/files/11382074/External.Synchronization.pdf
Please sign in to leave a comment.
Comments
1 comment