View my account

6 DoF 3d Object Tracking for Android (Oculus Quest 2)

Follow
Mcasaleg

Hello,

we are working on a Mixed Reality experience with Oculus Quest 2 and want to achieve precise 3d Object Tracking in conjunction with the inside-out oculus hand tracking. A possible scenario can be ad example, picking up a physical object (1x1x1 inches) tracked by its VR representation in real time.

Anyone has had any luck achieving this? Does the D400 series work with Android in Unity , and how shall we go for 3d object tracking? 

Thank you in advance,

Matt

0

Comments

3 comments

Date Votes
  • MartyG

    Hi Mcasaleg  There was a past project for the original Oculus Quest to use a RealSense D435i camera with the headset, its Android-based OS and Unity.  The RealSense community member who developed it fully documented the project at the link below, so it will hopefully be a useful starting point for your project on Oculus Quest 2.

    https://github.com/GeorgeAdamon/quest-realsense 

    The development process that led to the above guide is documented here:

    https://github.com/IntelRealSense/librealsense/issues/4155 

    You may also find useful an article on mixed-reality VR on the official RealSense blog.

    https://www.intelrealsense.com/co-presence-in-vr-wake-project/ 

     

    0
    Comment actions Permalink
  • Mcasaleg

     

    Thank you MartyG for the links, I was aware of the github project and will look into it further. I'm most interested in the 3d tracking aspect and was wondering whether anyone has experience with accurate 3d object tracking ( 6 degrees of freedom ) with Android and Unity. I just found out this one: https://visionlib.com/ but not sure it works with Realsense. 

    Does the Realsense SDK come with any 3d object tracking solution?

    Thank you all in advance,

    Matt

    0
    Comment actions Permalink
  • MartyG
    • Edited

    Intel did some past work in this area in 2016-17.  They prototyped a "Merged Reality" headset with RealSense stereo depth camera technology called Project Alloy.  The product was not released though.  Project Alloy could convert real world objects into virtual ones in the headset view and enable the headset wearer to interact with those virtual objects.  It could also mix real-world elements such as the wearer's real hand and nearby people into the virtual environment..

    https://youtube.com/watch?v=Be-MJxeCpvU 

    The SDK does not have a built-in tracking solution for the 400 Series cameras.  The information in the link below may be useful though.

    https://support.intelrealsense.com/hc/en-us/community/posts/360051876853-Using-D435-for-tracking-3D-color-objects 

     

    0
    Comment actions Permalink

Please sign in to leave a comment.

Didn't find what you were looking for?

New post