C# Time out with D435i IMU data enabled
I have been working with D435 for a while. I recently ordered two D435i's. I am using the C# wrapper. My code continues to work fine with the D435i as long as I do not enable the IMU data - only the color and depth frames. If I start the pipeline with accel and gyro data enabled, WaitForFrames() never returns. It always times out. Please help.
cfg.EnableStream(Stream.Depth, 640, 480);
cfg.EnableStream(Stream.Color, Format.Rgb8);
_pp = _pipeline.Start(cfg); //If I Start this way, the below WaitForFrames works fine
//_pp = _pipeline.Start(); //If I Start this way, all four depth, color, accel, gyro are enabled and WaitForFrames times out.
using (var frames = _pipeline.WaitForFrames())
{
using (var depth = frames.DepthFrame)
{
InitializeText = "Initialized: The camera is pointing at an object " + depth.GetDistance(depth.Width / 2, depth.Height / 2) + " meters away";
}
}
-
If you define cfg statements and use pipeline.Start(cfg); then the cfg statement inside the brackets instructs the SDK to start the pipeline with the configuration that you defined in the cfg statements preceding the start instruction.
If the pipeline start statement has empty brackets - () - then the default configuration for that particular camera model is used when the pipeline is started.
Adapting the "Hello World" C# example program and defining the cfg setup based on scripts in the SDK's "C# Cookbook" sample scripts page, an alternative way to define the script may be like this:
var cfg = new Config();
cfg.EnableStream(Stream.Depth, 640, 480);
cfg.EnableStream(Stream.Color, Format.Rgb8);
var pp = new Pipeline();
pp.Start(cfg);
while (true)
{
using (var frames = pp.WaitForFrames())
using (var depth = frames.DepthFrame)
{InitializeText = "Initialized: The camera is pointing at an object " + depth.GetDistance(depth.Width / 2, depth.Height / 2) + " meters away";}
}
}
-
I obviously didn't explain my question very well. My point is that if I start the pipeline with either the gyro or accel streams enabled then the subsequent WaitForFrames always times out. If I use the config to only enable Color and/or Depth streams then everything works fine. The empty pipeline.Start() enables all four streams, of course, and this fails.
The example C++ projects work fine for me as does the RealSense viewer. My app is a WPF app in Win 10 using the C# wrapper and targeting .NET 4.7.2 and, as I'm trying to describe, the WaitForFrames always times out if either accel or gyro stream is enabled.
-
I understood the question. I wanted to ensure that the cfg instructions and the pipeline were set up correctly in C#, as it is the foundation for the functioning of all the code that comes after the pipe start.
I noted that the project did not seem to make any use of the gyro and accel streams after the pipeline started, as it was simply reading the distance. Are you planning to add functions to the script that use the accel and gyro later on, please? Or is the goal of using cfg just to prevent the accel and gyro streams from activating so that the wait_for_frames does not freeze?
-
The WaitForFrames times out. I have tried all of the combinations of streams. If I enable either the accel or gyro streams and start the pipe then the WaitForFrames times out. If I only enable Depth and/or Color streams then WaitForFrames is fine and everything works well. This is with two different cameras on two different computers. Excerpts from the exception are below. Thanks.
System.Exception
HResult=0x80131500
Message=Frame didn't arrived within 5000
Source=Intel.RealSenseInner Exception 1:
ExternalException: rs2_pipeline_wait_for_frames(pipe:00000198569DD160) -
Apologies for the delay in responding further, as I was carefully researching your problem.
I was reminded that I had a C# case a couple of months ago where the person was getting the 'frames didn't arrived within 5000' error. Like you, they were using a using block.
https://support.intelrealsense.com/hc/en-us/community/posts/360039270473-Start-D435i-with-C-
Dorodnic the RealSense SDK Manager recommended that dispose should be used on frames in C#. Otherwise, ..NET can become jammed up with frames, leading to the timeout.
https://github.com/IntelRealSense/librealsense/issues/2845#issuecomment-444448923
-
My research shows that others have encountered this error when enabling the IMU streams. The nature of developing for C# in RealSense is that it is difficult to advise on because there are far fewer information resources compared to C++ and Python.
Another strategy to try may be to use poll_for_frames to introduce some latency into the frame generating process.
-
I did find this link: https://github.com/IntelRealSense/librealsense/issues/2996
Where there is the comment: "The C# wrapper doesn't yet contain the high-level APIs to support IMU (i.e MotionFrame\Profile)..."
Is that pertinent to this case?
-
I think it's just not going to work in C#. I tried PollForFrames and it doesn't crash, of course, but the frameset is always null if I enable Accel or Gyro streams. If it's just Color and/or Depth than it works fine.
Thank you for the suggestions. I will move on without it for now. I hope that this functionality does appear in the C# wrapper sometime soon.
-
Following up with a bit more information. It seems like the problem is in creating the FrameSet in WaitForFrames(). If I start the pipeline with a callback function and handle the individual frames myself then it works. For example, this code works and I do receive Accel and Gyro frames as expected:
cfg.EnableStream(Stream.Gyro, Format.MotionXyz32f);
cfg.EnableStream(Stream.Accel, Format.MotionXyz32f);_pp = _pipeline.Start(cfg, f =>
{
var profile = f.Profile;
});
-
If you post your request at the RealSense GitHub website then I can mark it as both an official Feature Request and a Documentation request that can be tracked by the RealSense developers. You can do so by visiting the link below and clicking the 'New Issue' button. Thanks again for your effort!
Please sign in to leave a comment.
Comments
17 comments