Distance to Target on D415 (Python)
I have been trying to get the distance to target (currently a wall in front of me that covers the full frame of view of the camera) using D415 sensor. Using the Depth Quality Tool, I get the best result by setting the Resolution to 840x480, Frame Rate to 6 and Region of Interest upto 60%. The result is about 350 mm, which is what I get by measuring physically.
Now, when I try to replicate by using the Python code for depth alignment (https://github.com/IntelRealSense/librealsense/blob/master/wrappers/python/examples/align-depth2color.py):
- I am not aware how to set the Region of Interest Parameter.
- The value of the average depth image (even after multiplying it by depth scale) comes to only around 250 mm.
- The minimum and maximum values of the depth image are 0 and around 12 meters.
My questions are:
- How can I consider only a small, well defined region of interest when accessing depth image from python code?
- Am I missing anything when calculating the average distance to target?
- Since the sensor seems to be able to calculate the "Fill Rate" of the sensed area, is it possible to access just the valid result and eliminate the noise pixels in the output image being streamed?
Cheers!
-
If you are okay with recording the data into a 'bag' file first instead of doing the calculations on live-streams, Intel's Python 'Distance to Object' tutorial may meet your needs for calculating the distance to an object.
https://github.com/IntelRealSense/librealsense/blob/jupyter/notebooks/distance_to_object.ipynb
If you prefer to use live-streams, code for setting the auto-exposure ROI in Python can be found here:
https://github.com/IntelRealSense/librealsense/issues/2681#issuecomment-436391721
Another approach to a region of interest is to crop the X and Y of the image so that only data within a 'bounding box' is considered. This is more complicated to implement.
https://github.com/IntelRealSense/librealsense/issues/4838
You can enhance the image by applying post-processing filters to it. An Intel tutorial for doing this in Python is here:
https://github.com/IntelRealSense/librealsense/blob/jupyter/notebooks/depth_filters.ipynb
-
Thanks for your quick and extremely useful response. I shall look into these and try to implement the most suitable way to solve my problem.
If you don't mind, I would like to further pick your brain as to why should there be a need for a bounding box if my target covers the whole camera field of view. Why am I getting values like zero and extremely high values when the target is <1 m away. I am inclined to think that it is noise since the average of all pixels of the depth image yields a value that is somewhat near to my target distance. In which case, is there a way to eliminate the noise pixels in my depth image, by averaging multiple frames (for now my target and camera are static).
Thanks again!
-
With the 400 Series stereo cameras, the error factor (RMS error) increases over distance. It is very small at close range to an observed object but accumulates linearly as the object becomes further away. The error factor starts to become noticeable when further than 3 meters away.
For some people the error factor is problematic at > 1 depending on how precise they need their measurements to be (e.g even 4 mm error is unacceptable to their project).
RMS error can be reduced by around 30% by turning off the IR Emitter (the component that projects a semi-random pattern of dots onto the scene to enhance the readability of low-detail surfaces such as walls, doors and desks). In a well-lit scene, the camera can use ambient light to calculate depth instead of the dot projection.
If you feel you need a dot pattern projection, using an external LED projector instead of the camera's built-in laser based projector can reduce 'laser speckle' noise on the image.
If you get depth values of zero, this may represent detail in the scene that the camera is unable to read for some reason. This may include environmental factors such as reflections on an observed surface, or the surface being dark grey / black (this color absorbs light).
You may also experience greater noise at the left side of the image due to a phenomenon called 'invalid depth band'. This is described on pages 58-59 of the current edition of the 400 Series data sheet document.
-
Thanks for the insight! My target is going to be less than 2 meters away so the distance is definitely not an issue. I will try to experiment in bright sunlit area and get back in case of any further issues being faced. My whole problem seems to be boiling down to the difference between the way the Depth Quality Tool seem to be calculating the distance and the way it is being exposed through the Python wrapper. I need to dig deeper into it.
Cheers!
-
You are very welcome!
If you are in an office environment, fluorescent lights such as ceiling strip-lights may negatively impact the image due to gas inside the lights flickering at frequencies hard to see with the human eye. If you have such lights in the room, using a camera FPS close to the operating frequency of the lights (30 FPS for some lights, 60 FPS for others) may help.
Good luck!
Please sign in to leave a comment.
Comments
5 comments