View my account

D435 Frame Delay

Follow
Adrian

Hello,

I'm prototyping a multi-camera system using the D435 and seem to be having significant trouble with frame delays. This issue is intermittent and very tough to reproduce. What is happening is that when I first begin polling for frames (using the poll_for_frames sdk function), there is occasionally a delay in receiving frames from cameras that can last anywhere from 5-15 seconds. Usually, this doesn't happen and each camera receives its first frame in milliseconds. However, when it does happen, a subset (over half) of the cameras all experience delays.

 

Any suggestions are welcome since this has been a very difficult issue to root cause.

 

For context:

I'm using an 8 camera system

4 cameras are connected to a single hub and I'm using these hubs:
https://www.usbgear.com/CG-4PU3MGD.html?utm_source=googleshop

The cameras are using the usb c to standard cable that comes in the box.

These hubs are then connected to a machine running ubuntu 16.04, however the program I'm streaming the cameras with is running in a docker image based on ubuntu 18.04

 

Interestingly, when this occurs I've been able to correlate a higher count of uvc errors:

kernel: [395826.595131] uvcvideo: Failed to query (GET_CUR) UVC control 1 on unit 3: -32 (exp. 1024).

As well as this error:

kernel: [395802.675806] uvcvideo: Non-zero status (-71) in video completion handler

Any advice is helpful, I'm giving info on USB in particular since I suspect it could be the issue, but do not currently know how to confirm. Especially since I know I'm not using a particularly high amount of bandwidth. Only streaming depth at 6fps and 848x480 resolution.

 

Thank you!

0

Comments

5 comments

Date Votes
  • MartyX Grover
    • Edited

    Hi Adrian, may I first confirm whether your 8 cameras are attached to two industrial USB 3.2 hubs of the same model (the USB standard allows up to 5 hubs to be linked together).

    If 4 of the cameras are not attached to another hub of the same model then the two sets of 4 cameras would likely be using two different brands of USB controller, introducing potential unpredictability into the USB system in regard to how one set of 4 cameras will behave compared to the other set.

    0
    Comment actions Permalink
  • Adrian

    Hello Marty, I can confirm that all 8 cameras are attached to two identical industrial USB 3.2 hubs. 4 cameras per hub, and each hub is connected directly to the host computer. I've also used the same cables and power supplies for the hubs.

    0
    Comment actions Permalink
  • MartyX Grover

    Although it is recommended practice to use poll_for_frames() with applications that access multiple cameras, it also has the drawback that you need to control when the CPU is put to sleep and for how long, otherwise the CPU can max-out its utilization percentage at 100% of a single core, which could account for the delay effects that you are encountering.  This is described by a RealSense team member at the link below.

    https://github.com/IntelRealSense/librealsense/issues/2422#issuecomment-423254709

     

    Although poll_for_frames() is the recommended instruction for multicam applications, some RealSense users who have created multiple camera applications whose processing is not too demanding have opted to use wait_for_frames() instead and found that it did not have a negative effect on performance.  Control of CPU sleep periods is not required with wait_for_frames()

    0
    Comment actions Permalink
  • Adrian

    Hi Marty,

    Thank you for your prompt responses and suggestions. I tried a little experiment with wait_for_frames today and it did not seem to make a significant change. I also reviewed my use of poll_for_frames() and I have little concern about CPU usage.

    In my testing today I was able to reproduce my issue. It seems like poll_for_frames() is simply returning False for an extended period of time. The issue resolved after I disconnected and reconnected all of my cameras.

    This leads me to a couple of additional questions for you:

    Is there a way to query the state of the camera's FW? Perhaps some logs, or at lease some indication of whether the camera might be in some state where it is unable to provide frames?

    Do you think there could potentially be an issue with connecting 4 d435 cameras to a single hub? From what I've read in the whitepapers and given the resolution and frame rates I'm using, this didn't initially seem to be an issue. However, given the fact that I was able to resolve simply by reestablishing the USB connection would seem to indicate that the USB topology could be the source of the problem.

     

    Thanks

    0
    Comment actions Permalink
  • MartyG
    • Edited

    Using 4 cameras per hub at 6 FPS should not be exceeding the average 5 Mbps maximum bandwidth of a typical USB hub.

     

    There have been rare past cases where the stream has timed out or experienced other errors when 6 FPS is used because frames were not arriving quickly enough.  In such cases, increasing the FPS to 15 made the problem disappear.  

     

    If exception type errors are generated by the timeout then you could implement an error try-catch to initiate a pre-programmed action if an exception error occurs, such as triggering the hardware_reset() instruction to automatically reset the camera instead of having to unplug-replug it.

    https://dev.intelrealsense.com/docs/librealsense-error-handling-scheme

    0
    Comment actions Permalink

Please sign in to leave a comment.

Didn't find what you were looking for?

New post