Skeletal tracking behind desk or with partial viewport
Hi All,
As part of newb bootstrapping, I am wondering how much recognition capability to expect when the person is sitting behind a desk or otherwise when some of the body is occluded from the viewport ― either because the waist down is hidden by the desk, or just because the RealSense camera is located such that it focuses only on the upper body part e.g. due to limited room space.
This is as opposed to most demo videos for this class of devices, which will typically show whole body tracking in largish rooms/spaces where the camera can be easily placed far away to capture the entire body, not to mention without any occlusion of most of the body.
I'm extra happy to hear of community experience with this situation, from circumstantial experience or similar scenarios alike!
At worst, a-priori I would assume that a more raw level of the RealSense API can still be used with high precisino by training my own models ― happy to have your comments to that as well!
Many thanks!
Matan
-
Hi Matan Traditionally, performing skeletal detection on people in unusual poses such as laying down, or when "off screen" from the camera's view, has always been difficult. Microsoft's Kinect 1 dealt with the problem of non-visibility of joints with algorithms that predicted where body joints should currently be if not visible. Developers also had to create custom algorithms to handle poses that the SDK software did not support by default (such as "if the user is detected to be in Pose A + Pose B, they must be in Pose C").
Modern skeletal SDKs such as Cubemos Skeleton Tracking SDK are better these days though at automatically working out where joints are if the body is occluded (for example, by walking behind another person). If you play the Cubemos demo video and go through it carefully by pausing / unpausing with the spacebar, you can see glimpses of how Cubemos handles occlusion situations. For example, on the horse riders, Cubemos predicts the joints of both legs and displays them even though one leg is hidden from the camera angle's view on the other side of the horse.
https://www.youtube.com/watch?v=PENY6Q24c6A
-
Thanks for the broad perspective reply. Happy to hear more from community members. In particular in my case there is no need to guess where the feet are ... they are just not interesting for my scenario and I wonder how much does the recognition of the upper body, mostly arms and hands, head, is adversely affected when the lower body is occluded!
This might really help prior to my own experiments and choice of RealSense hardware model!
-
Cubemos seems to treat joints individually. Parts that cannot be calculated disappear whilst the rest of the joints remain, and invisible joints instantly reappear as soon as they can be calculated again. In the image below where a person walks in front of another person, their lower body temporarily disappears but their upper joints remain.
Please sign in to leave a comment.
Comments
3 comments