About IMU frequency
Hello, my device is D435I. When I use examples/capture in the SDK, the following results appear:
Depth[0]: 18 [frames] ||
Color[3]: 18 [frames] ||
Accel[5]: 59 [frames] ||
Gyro[6]: 2125 [frames] ||
Depth[0]: 48 [frames] ||
Color[3]: 48 [frames] ||
Accel[5]: 123 [frames] ||
Gyro[6]: 4123 [frames] ||
The result shows that the frequency of the gyroscope has reached 2000HZ. I have a question: 200HZ or 400HZ shown in the data sheet does not conform to this procedure
-
Hi Jxpenggf There was a previous case in 2020 of the IMU providing a rate of 2000 hz.
https://github.com/IntelRealSense/librealsense/issues/6115
In that discussion, a RealSense team member confirmed that it was a glitch.
https://github.com/IntelRealSense/librealsense/issues/6115#issuecomment-603177990
https://github.com/IntelRealSense/librealsense/issues/6115#issuecomment-604359869
-
first:Thank you for your answer. When I set the gyroscope frequency of the IMU to 200 and the accelerometer frequency to 63. How to output ACC and GYRO at 200Hz at the same time in C++ without ROS. What are some examples for reference?
second:How to synchronize the RGB image with the depth image -
1. You can custom-configure gyro and accel IMU streams in C++ using code like the lines below:
cfg.enable_stream(RS2_STREAM_ACCEL, RS2_FORMAT_MOTION_XYZ32F, 63);
cfg.enable_stream(RS2_STREAM_GYRO, RS2_FORMAT_MOTION_XYZ32F, 200);An example of a full IMU-utilizing script in C++ language can be found in the link velow.
https://github.com/IntelRealSense/librealsense/issues/6737
I would personally recommend keeping to the standard frequency pairings though (200 / 400 Hz for gyro, 63 / 250 Hz for accelerometer).
2. Do you mean ensuring that depth and RGB have the same FPS speed, or aligning depth to color please?
-
On the first question, I think you seem to have misunderstood me. When I set the acceleration frequency to 250Hz and the gyroscope frequency to 400, I wanted the IMU to output acceleration and angular velocity at the same time every time, so that I could get 400 frames in 1s. I read that ROS SDK processes gyroscope data through linear interpolation, but I am not familiar with ROS, so I would like to ask if there is a C++ interface.
For the second problem, I found that the RGB image did not align with the depth image timestamp when I tested it -
1. IMU values can be printed out in C++ using a std::cout statement, like in the Intel-provided example script in the link below.
https://github.com/IntelRealSense/librealsense/blob/master/doc/d435i.md#api
Note that in the example, the std::cout instructions are preceded by two slashes - // - that indicate that these lines are 'commented out' so that the program ignores them when running the script. To activate the values printout, simply delete the // from the start of each of these two lines.
Also note that if you use the above script and want to customize the frequency of the streams then you should edit the cfg configuration lines to add the full configuration for gyro and accel. For example:
cfg.enable_stream(RS2_STREAM_GYRO, RS2_FORMAT_MOTION_XYZ32F, 63);
cfg.enable_stream(RS2_STREAM_ACCEL, RS2_FORMAT_MOTION_XYZ32F, 200);
2. The link below provides a list of information resources about RGB-depth sync.
https://github.com/IntelRealSense/librealsense/issues/8726#issuecomment-812556071
-
Thank you very much for your reply. But, to be honest, the above two questions seem to be a little bit unreasonable for you. Okay, let's not discuss those two issues.
I have found two problems using the following code:The first : It can be seen from the output that the depth/infra/icolor data is about 30ms later than the IMU data. Should the timestamps be consistent with my code?how to explain this?How to make them consistent.
the second:The color data and the depth/infra timestamp sometimes differ by about 8ms, sometimes the timestamp is similar, how to explain this?the code outputs:
id:Depth 1621658728663.1772461 accl 230 1621658728689.1469727 gyro 289 1621658728690.9760742
id:Infrared 1 1621658728663.1772461 accl 230 1621658728689.1469727 gyro 289 1621658728690.9760742
id:Infrared 2 1621658728663.1772461 accl 230 1621658728689.1469727 gyro 289 1621658728690.9760742
id:Color 1621658728656.6672363 accl 230 1621658728689.1469727 gyro 289 1621658728690.9760742
id:Depth 1621658728696.0915527 accl 238 1621658728720.2133789 gyro 301 1621658728720.6154785
d:Infrared 1 1621658728696.0915527 accl 238 1621658728720.2133789 gyro 301 1621658728720.6154785
id:Infrared 2 1621658728696.0915527 accl 238 1621658728720.2133789 gyro 301 1621658728720.6154785
id:Color 1621658728689.5803223 accl 238 1621658728720.2133789 gyro 301 1621658728720.6154785
int main() {
std::map<int, std::pair<int,double>> counters;
std::map<int, std::string> stream_names;
std::mutex mutex;
auto callback = [&](const rs2::frame &frame)
{
std::lock_guard<std::mutex> lock(mutex);
if (rs2::frameset fs = frame.as<rs2::frameset>())
{
// With callbacks, all synchronized stream will arrive in a single frameset
for (const rs2::frame &f : fs)
{
counters[f.get_profile().unique_id()].first++;
std::cout<<"id:"<<f.get_profile().stream_name()<<" "<<std::setprecision(20)<<f.get_timestamp()
<<" accl: "<<counters[5].first<<" "<<counters[5].second<<" "
<<" gyro: "<<counters[6].first<<" "<<counters[6].second<<"\n";;
}
} else
{
counters[frame.get_profile().unique_id()].first++;
counters[frame.get_profile().unique_id()].second=frame.get_timestamp();
}
};
// Declare RealSense pipeline, encapsulating the actual device and sensors.
const int width = 640;
const int height = 480;
int fps = 30;
rs2::pipeline pipe;
rs2::config cfg;
cfg.enable_stream(RS2_STREAM_COLOR, width, height, RS2_FORMAT_BGR8, fps);//向配置添加所需的流
cfg.enable_stream(RS2_STREAM_DEPTH, width, height, RS2_FORMAT_Z16,fps);
cfg.enable_stream(RS2_STREAM_INFRARED, 1, width, height, RS2_FORMAT_Y8, fps);
cfg.enable_stream(RS2_STREAM_INFRARED, 2, width, height, RS2_FORMAT_Y8, fps);
cfg.enable_stream(RS2_STREAM_ACCEL, RS2_FORMAT_MOTION_XYZ32F,250);
cfg.enable_stream(RS2_STREAM_GYRO, RS2_FORMAT_MOTION_XYZ32F,400);
rs2::pipeline_profile profiles = pipe.start(cfg,callback);/// 开启了线程入口
// Collect the enabled streams names
for (auto p : profiles.get_streams())
stream_names[p.unique_id()] = p.stream_name();
std::cout << "RealSense callback sample" << std::endl << std::endl;
while (true) {
std::this_thread::sleep_for(std::chrono::seconds(1));
std::lock_guard<std::mutex> lock(mutex);
for (auto p : counters) {
std::cout << stream_names[p.first] << "[" << p.first << "]: " << p.second.first << " [frames] || \n";
}
}
} -
RGB color and depth originate from separate sensors. Depth is generated from the left and right IR sensors (hence stereo depth), with the 0,0,0 origin of the camera's coordinate system being at the center of the left infrared sensor. RGB color though originates from the RGB sensor.
The left infrared stream is perfectly aligned with depth because infrared and depth originate from the same sensor, whilst RGB and depth are synced by a best match process because RGB does not originate from the same sensor.
https://github.com/IntelRealSense/librealsense/issues/1548#issuecomment-389070550
Page 17 of the current edition of the data sheet document for the 400 Series cameras provides an explanation for how depth is generated. I have provided an extract of this section below.
https://dev.intelrealsense.com/docs/intel-realsense-d400-series-product-family-datasheet
The discussion in the link below provides information regarding timing differences between the IMU and other stream types.
-
For other RealSense users reading this discussion in future, it is continued at the link below.
-
I am the one who posted the post. I'm disabling global time by setting RS2_OPTION_GLOBAL_TIME_ENABLED to '0'.But this depth data is still about 30ms earlier than the IMU time.This is really an uncomfortable question.How is this timestamp assigned?Is it when the device captures the image or when it is transferred to the PC via a USB driver?If it is the former, then it is possible that the difference between the IMU data and the depth data is 30ms. After all, the transmission takes time.
outputs:
name:Depth 4294228.1390000004321 acc 4294266.8679999997839 gyro: 4294268.9670000001788 Hardware Clock
-
I know you are the one who posted the GitHub message Jxpenggf - my message was aimed at other people, to let them know where to continue reading if this case here had no more comments.
Intel's guide for getting IMU data states: "The D435i depth camera generates and transmits the gyro and accelerometer samples independently, as the inertial sensors exhibit different FPS rates (200/400Hz for gyro, 63/250Hz for accelerometer). Each IMU data packet is timestamped using the depth sensor hardware clock to allow temporal synchronization between gyro, accel and depth frames".
https://www.intelrealsense.com/how-to-getting-imu-data-from-d435i-and-t265/
-
The link below provides further information about how IMU data is handled compared to how depth, RGB and infrared data is handled.
https://github.com/IntelRealSense/librealsense/issues/4803#issuecomment-528331115
Please sign in to leave a comment.
Comments
13 comments