Beginner - need advise
Hello,
I and my colleagues are building a robot with a structure consistent with human anatomy.
https://www.facebook.com/automaton.group/
To ensure vision and begin to create basic AI we bought the D435i camera.
I need some advise how to start creating a project. My point is that we want to recognize objects, measure them, measure the distance to the object and on this basis robot should know how to grap this object.
Let's assume that we only know the basics of python and we want to learn how to create the aforementioned software based on the D435i camera.
Can you recommend any tutorials or guides? Where we should start?
Regards,
Michael
-
The first step that I would recommend is to install the RealSense SDK's pyrealsense2 Python compatibility 'wrapper' if you have not done so already.
https://github.com/IntelRealSense/librealsense/tree/master/wrappers/python
If you are able to use PyPi then the Python wrapper can be installed with this command:
pip install pyrealsense2
In regard to satisfying your project needs for object detection, distance reading and measuring ... there are "old" ways of doing such things with RealSense and Python. It may be better to start from the beginning with learning a modern method that you can actually develop into a production-level project though. For that reason, a good fit for your project may be to use your RealSense camera with ROS and the OpenVINO Toolkit vision computing software.
I suggest ROS because with the RealSense ROS wrapper, you can write ROS 'node' programs in the Python language to add to the project's capabilities.
The link below has an example tutorial using RealSense, ROS and the OpenVINO Toolkit. It suggests using a hardware acceleration device such as Intel Neural Compute Stick (NCS) for boosting the performance of vision computing applications. Having such a device is not listed in the prerequisites for the project though.
https://github.com/gbr1/ros_openvino
The official RealSense Store offers the current NCS model, Intel Neural Compute Stick 2, as a standalone purchase or in a bundle with a RealSense camera if you are interested in pursuing hardware acceleration for your project.
-
Intel have a RealSense Python tutorial for using data in a pre-recorded 'bag' file to align depth and color data and then identify the object and measure the distance to the object.
https://github.com/IntelRealSense/librealsense/blob/jupyter/notebooks/distance_to_object.ipynb
The pre-recorded nature of the data may not make it suitable for your robot application though, which made me hesitant to recommend it. There are other similar Python-based examples that can do object and person detection in this way with live data though.
https://github.com/IntelRealSense/librealsense/issues/3086#issuecomment-455181310
In regard to measuring, the robot nature of the project means that it may not be appropriate to recommend examples that require the detected object to be placed against a target image such as a chessboard / checkerboard (unless the object will always be roughly in the same place, such as looking at a tub containing objects). I will direct you to a calibration target-based measuring system in the SDK for Python in case you wish to investigate it. It is designed for multiple cameras but should be able to work with one.
Please sign in to leave a comment.
Comments
4 comments