Detect and track Ball with D435 in Unity
Hi,
I need to detect and Track a futball ball with the D435 camera.
I'm working on Unity, but i can change to other platform.
Thanks you.
-
If you want your application to follow only a ball and ignore everything else, object detection and coordinate reading using a Deep Neural Network (DNN) may work well for you. It can identify an object from an RGB color image and then calculate XYZ coordinates for the currently detected position of that identified object.
The RealSense SDK has an example DNN program that uses OpenCV and C++.
https://github.com/IntelRealSense/librealsense/tree/master/wrappers/opencv/dnn
You can find specific examples of using a DNN for ball detection by googling for 'dnn ball'. Here is an example project for ball recognition:
-
If your project budget allows it, you could add an Intel Neural Compute Stick 2 into a USB port alongside your RealSense camera for hardware acceleration of vision computing applications. The Stick is sold in the official RealSense store, in a bundle with a camera or on its own.
https://store.intelrealsense.com/buy-intel-neural-compute-stick-2.html
Applications created with the RealSense SDK can also have their speed improved by offloading work from the computer's CPU to its graphics GPU. This can be done by building the SDK with CUDA support if you have an Nvidia GPU, or with an SDK method called GLSL Processing Blocks if you have a non-Nvidia GPU.
https://github.com/IntelRealSense/librealsense/issues/4905#issuecomment-533854888
-
Thanks you MartyG,
I undertand that as OpenCV with dnn is made only for c++, python and also javascript. We can't use Unity to detect the ball with the realsense?
We need to do an application on c++ that detect the ball and pass the data to unity by messages?
I'm a little los with how to train my own coffemodel dnn to detect a ball. -
There is a Unity implementation of OpenCV in the Unity Asset Store, though it is not inexpensive and I could not make guarantees that an OpenCV DNN program would work in it.
https://assetstore.unity.com/packages/tools/integration/opencv-for-unity-21088
Another approach to tracking a specific object with a 400 Series camera is to attach image tags called Aruco markers to the surface of the object so that the camera can detect the tag and capture the object's "pose" (its position and rotation). It might not be appropriate for a ball used in real play though, as you would need a visible tag on every side of the ball.
In the early days of RealSense SDK 2.0, a developer created a RealSense Aruco tracker in Unity that used the OpenCV for Unity software mentioned above.
Please sign in to leave a comment.
Comments
5 comments