Camera to CPU Requirement
I was worry is how good is the SDK as i want to detect certain difficult posture like kickbutt, high knee, burpee and etc.
Besides, I wish to use horizontal view of the camera as my vertical view due to the angle of view of horizontal is wider than vertical view and without rotate the physical camera.
Lastly, what is the operation system requirement and hardware requirement to use the D435 camera that i need to concern. this is because i plan to buy a mini pc box to work together with realsense camera and the skeletal detection will be show on 50 inch LED vertical screen. does it matter to graphic card / processor / RAM or etc?
-
The 400 Series cameras are highly flexible in the hardware they will work with. They can be used with any Intel or ARM processor. There have also been a couple of documented successful cases with AMD Threadripper processors, but Intel and ARM are the ones that can most be relied upon to work.
The capability of the computing hardware will matter if you are going to be running a high-demand application. The camera contains a 'Vision Processor D4' hardware board though that does processing on captured data, reducing the need for a powerful CPU and GPU in the computing hardware that the camera is attached to.
if you are going to be doing skeletal tracking, the best available software to use is a commercial package called Nuitrack SDK. It costs $39.99 a year and also has a free trial version. Nuitrack SDK may have difficulty recognizing complex poses such as sitting and laying. Nuitrack is working on a new version called Nuitrack AI though that is due in the second half of 2019 and claims to be able to cope with such poses.
In regard to the computing hardware, the new Raspberry Pi 4 Model B that has just been released may be an attractive option. It claims performance comparable to entry-level desktop PCs, has USB 3.0 ports and offers support for 4K displays.
-
thanks marty being responsive so high power mini pc is not necessary is it since D400 Series camera contain 'Vision Processor D4' which reduce the need for a powerful CPD and GPU?
How about the Realsense SDK? As i know Realsense did provide SDK with need of support OpenCV to perform skeletal tracking as well. Do Nuitrack AI will have free trial version in future also?
Since Intel can are most be relied upon D400 series is it I get and Intel mini PC of quad core will be will be more stable?
-
Whether you need a high-powered PC will depend on your application really. For example, if you were attaching several cameras to the same PC then a PC with an Intel Core i7 processor or better would be recommended. If you are just running a single camera, maybe with a color and depth stream at the same time, then the Vision Processor D4 hardware can likely handle the majority of the graphics processing load.
Another factor in how well the camera performs on PC hardware is the storage, if you are going to be recording the data to a file. You may experience slowdown when recording unless you use a speedy solid state drive (SSD), as slower storage may struggle to keep up with the recording otherwise and create a 'bottleneck' in processing.
If you expect to be putting a heavy processing load on the hardware, then certainly get the best mini-PC that your budget allows. Intel offer a range of NUC mini-PCs of various specifications to meet every budget, from low-spec $300 machines to super-powerful $1000+ kits such as the NUC 8 VR model from 2018.
Nuitrack is made by a different company from Intel, so I do not have any knowledge about their release plans. It is certainly possible to use OpenCV for skeletal tracking instead, though it will take more technical knowledge to set up than Nuitrack would. There are guides available on YouTube.
Body tracking support is not built into the RealSense SDK. However, Intel have also done some work on body tracking with the RealSense 400 Series cameras and produced a video presentation on the subject that uses machine learning. Again, this is not an instant 'usable out of the box' solution, as you would have to create a set of 'training data' with multiple cameras before you could use the tracking with a single camera.
https://www.youtube.com/watch?v=VSHDyUXSNqY
-
Hi Marty,
Thanks for your patience upon me. I really appreciate the help, time, assistance & support you’ve given me. Provided example of several camera and single camera, factor of choose PC hardware, concern of experience slow down, guidance of Nuitrack and information in your comment are noted.
-
Hi MartyG, I am thinking of creating a setup with my raspi 4 paired with 6 realsense d435. I am only using it for recording of RGB videos at 960p and 24fps. I have 6 of these cameras leftover which is why I have decided to use them for this project and I hope to complete this project without purchasing more cameras.
There are three issues I am aware of:
1) Power
- To handle this issue, I am plugging all these cameras to a powered usb hub and this hub is connected to the rpi
2) Bandwidth
- I checked online and it seems doable using USB 3.0 and only transmitting said video spec. this is my reference Multiple_Camera_WhitePaper_rev11.pdf (mouser.com)
3) Computing capability
- This is what I am unsure of. It seems entirely logical for me that this would work but I would need to disable certain things so that the rpi cpu only has to do this one task of reading and writing the videos into my ssd.
I would appreciate if you could validate my ideas, correct and advise me. Thanks!
-
Hi Bluekksailor Your project sounds technically achievable if you are only using a single RGB stream from each camera. In 2015 Intel demonstrated a drone with six R200 cameras from the original generation of RealSense cameras that were attached to a single computing board with an Intel Core i7 processor.
If you wanted to run the six cameras off a single hub then you may have enough bandwidth with only one RGB stream per camera.
In regard to computing capability, an Nvidia Jetson board has been more commonly used for attaching multiple cameras to the same board. And the approach that Intel takes in its open-source multiple camera RealSense ethernet networking paper is to assign a Pi 4 to each individual camera.
Having said that, you are certainly free to try six cameras on a single board first to see whether it is achievable.
-
Hi @MartyG
I was trying to use Skeleton Tracking SDK by Cubemos with JETSON NANO B01 microcomputer and Intel Realsense D435 camera module. It seems like this SDK is not supportive to ARM processor. I also tried to use NUITRACK SDK but not compatible with this microcomputer( processor: ARM 64 bit processor. NUITRACK SDK works with ARM 32 bit/AMD 64 processor). I think Nuitrack SDK is working with raspberryPi2 V1.1. or lover version
Do you have any other suggestions to use Skeleton tracking SDK by cubemos?
Thank you
-
Hi Sameerapiphotonics The manual for the Cubemos Skeleton Tracking SDK states that the processor requirements are "6th to 10th generation Intel Core and Xeon Processors". I have not heard of any workaround method for using it with Arm.
The most recent news about Arm device support for Nuitrack SDK that I am aware of is that support for Raspberry Pi 3 and Tinker Board was added in 2020.
https://community.nuitrack.com/t/license-tool-compatability-test-not-working/2149/3
Intel published a seminar on YouTube for using RealSense for training your own dataset for body joint recognition and applications such as avatar control. That approach would likely not be affected by having an Arm processor, since Jetson boards and 400 Series cameras can be used together.
-
MartyG I see...Thank you very much for your kind support.
Please sign in to leave a comment.
Comments
10 comments