Delay in change exposure/laser power
Hi,
Some context first: in my application, I need to switch between 2 settings (exposure & laser power). I set those with sensor.set_option(), then call pipeline.wait_for_frames() several times. I observed that the first frames I get after changing the settings are still with the old settings. In order to figure out which is the first realiable frame afther the change, I started tracking the frame number and time stamp. I was surprised to see that sometimes I get frames with the same number/timestamp, even though I am using wait_for_frames() and I set the frames queue size to 1. So now I have 2 problems:
1. Why do I get the same frame twice?
2. What is the most reliable way of deciding which is the first good frame after changing the settings, while minimizing the number of frames dropped. Do you know if those changes take a given amount of seconds or frames to be effective?
In case this is relevant, I am using fps=30 and am connected to 3 cameras at the same time, getting IR(RGB8) + depth from each of them. My application's framerate is lower than 30pfs, so there are some frames that get dropped and that's ok for me.
Thanks!
Raluca
-
Hi Ralucadianapetre Did you build the RealSense SDK (librealsense) from source code with the RSUSB backend please? If you did, the RSUSB backend can take longer to register an option change compared to the V4L backend (a kernel-patched build). This is discussed at the link below.
https://github.com/IntelRealSense/librealsense/issues/6952
It is possible to edit the SDK's types.h file and then build that customized SDK from source code so that an RSUSB build checks for events such as options changes more frequently than the default of every 5 seconds, as described here:
https://github.com/IntelRealSense/librealsense/issues/10188#issuecomment-1023153338
Even if you are not using an RSUSB build, changing an option can take at least a couple of frames to complete, due to the time taken for changes to propagate back to the user.
There is no need to set the frame queue size to 1, as it is already '1' by default. This queue size has an increased chance of frames being dropped, whilst setting the queue to '2' adds latency and therefore reduces the chance of frame drops. Some RealSense users have found that a queue size of '50' works well for them.
You may receive the same frame twice if there is a 'hiccup' during streaming that results in a bad frame, causing the SDK to automatically go back to the previous good frame and continue onwards from that point. The SDK should therefore decide what the first good frame after changing the settings is.
-
I used Intel.RealSense.SDK-WIN10-2.50.0.3785.exe to install the SDK. I don't know if it uses RSUSB or not, but the change occures in les than 0.5s / 15 frames. So I suppose it used the other backend (V4L).
Out of curiosity, does the queue size even matter when using wait_for_frames()?
-
The Windows version of the SDK uses Microsoft Media Foundation as its backend by default.
It can alternatively be built from source code with CMake instead of using the automated Intel.RealSense.SDK-WIN10 installer file, which allows the backend to be set to a backend based on a WinUSB rewrite of the UVC protocol by setting to true the CMake build instruction FORCE_RSUSB_BACKEND
https://github.com/IntelRealSense/librealsense/blob/master/doc/installation_windows.md
If you are using 3 cameras at the same time that are being controlled by the same program, it is recommended that you use poll_for_frames() instead of wait_for_frames(), as described at the link below.
https://github.com/IntelRealSense/librealsense/issues/2422#issuecomment-423254709
-
Queue size = 1 is the default 'performance' setting with minimum latency but a risk of frame drops. It is also advised to use a frame queue greater than '1' if you have enabled more than one stream type (i.e depth + color instead of just depth).
poll_for_frames returns frames instantly without blocking until a complete frame is available like wait_for_frames does.
A RealSense Windows user at the link below who was experiencing repeating frames stated that using FORCE_RSUSB_BACKEND provided the latest frames in their particular case.
https://github.com/IntelRealSense/librealsense/issues/9528#issuecomment-895550918
There is not information available regarding whether the Microsoft Media Foundation or RSUSB backend provides better response time for option changes in the Windows version of the SDK though.
Please sign in to leave a comment.
Comments
5 comments