Advice for tracking sports balls.
Hello,
I am successfully able to track many types of sports balls using a D455 camera and OpenCV together. I am able to get enough information of the balls in consecutive frames to calculate launch data such as speed, vertical and horizontal launch angles. Balls I can currently reliably track are anything about 6 inches in diameter and up. However I am having some trouble reliably tracking a ball the size of a baseball, especially when it is traveling at hitting speeds. Baseballs traveling at throwing speeds track reasonably well. The baseballs are foam and when hit with a foam bat the detection rate falls off, especially if the ball gets near the camera.
The camera is installed overhead, 9-10 feet. The user stands behind the player and hits or throws the baseballs at a projection screen. The room is on the darker side, like a theater before they switch off the lights when the movie starts.
I am looking for suggestions on how to improve the detection rate of fast, high angle baseballs traveling towards the screen. I did find that by putting the camera at more like 5 feet high and offset to the side the tracking improves. But that isn't the ideal mounting position for my application.
The two issues I see are the ball getting inside the minimum depth range and perhaps too little light.
My next experiment is to add an off the shelf 850nm IR flood light. Are there any other things I should consider?
As a side, there is another proprietary solution out there that uses a D435 successfully in detecting sports balls in the same type of environment and mounting configuration. So I know it can be done at least.
Any thoughts are appreciated!
-
Hi Wmaass88 When tracking fast motion of small objects, the higher the FPS speed used the better. Could you try using 90 FPS if you are not doing so already, please?
If your software application uses RealSense SDK code then you can reduce the minimum depth sensing distance of the camera using an option called Disparity Shift. Setting this to a value of between 100 and 200 would allow the ball to get closer to the camera, though also reduce the camera's maximum depth sensing range.
Using SDK code to increase the Laser Power option to its maximum value of '360' in order to maximize the camera's IR projector output should also help to make the projected IR more visible to the camera.
If your application only uses OpenCV code then a cruder method of reducing minimum distance is to reduce resolution to 640x480.
There is not anything special that you need to consider about using an IR illuminator lamp. RealSense 400 Series cameras can make use of any light source that it is in a frequency range that they are able to see, such as 850nm, in order to aid depth sensing.
-
MartyG,
Thanks for the suggestions, very helpful. I am currently capturing at 90fps with the D455 and also using maximum power on the laser.
I will try the Disparity Shift option. Is that the same as setting RS2_OPTION_MIN_DISTANCE?
EDIT: I see it is actually manipulating the Depth Table. -
RS2_OPTION_MIN_DISTANCE is different from Disparity Shift, as it sets the minimum depth distance in meters to be rendered on the depth image. The ball would still become undetected though if it moved below the minimum depth distance of the camera. This is why the camera's minimum depth distance needs to be reduced with Disparity Shift.
For example, if the D455's minimum depth distance is around 50 cm / 0.5 meters then even if you set RS2_OPTION_MIN_DISTANCE to 0.3 m, the ball would still likely disappear below 0.5 m unless Disparity Shift had been configured so that the ball could get closer to the camera.
-
Great, thank you for the clarification.
One more question if I may. As I mentioned I know of another solution that uses the D435 in the same use case as mine. However when I try the D435 my results are dramatically worse than the D455 for all types of balls regardless of size or speed. Do you have any insight as to why that may be? -
The key differences between D435 and D455 are that D455 (a) has better depth-to-color alignment, (b) the depth and color field of view sizes of D455 are approximately the same, whilst the color sensor has a smaller field of view than the depth sensor, and (c) D435 has a smaller minimum distance than D455 (around 0.1 meters) and so objects can get closer to the camera without having to use Disparity Shift to lower the minimum distance.
-
MartyG,
Thanks so much for the advice and explanations, I appreciate it.
One more question if I may. Currently I am using OpenCV to determine something of interest to look at and measure, in this case a ball moving thorugh the depth frame. I am basically using OpenCV to look for blobs that meet certian criteria, like area, circulatiry, etc. if something of interest is found I then get XYZ info using RealSense on the blob. My question is, is there a way to analyze the depth frame purely with the RealSense SDK without using OpenCV to determine what may be a ball?
Please sign in to leave a comment.
Comments
7 comments