Accessing Intel RealSense Camera in Segway Loomo

• MartyG

  • July 26, 2019 09:43
  • Edited

  According to a 2018 article that Intel did on Loomo, it can support a SLAM library.

  https://www.intelrealsense.com/loomo/ 

  The list of features that it is quoting makes it seem as though they are using a form of the RealSense SDK For Linux, an SDK that was created by Intel specially for the ZR300.  The link below from February 2019 indicates that some people have managed to get Loomo to do SLAM.

  https://twitter.com/braincor/status/1017834048050384896 

  The Loomo developer docs say that if SLAM can't be used, odometry can be used for localization instead:  Loomo's documentation on Odometry starts about halfway down the page in the link below.   Do a page find for 'odo' to find it quickly.

  https://developer.segwayrobotics.com/developer/documents/segway-robots-sdk.html 

  Further down that page, under the heading 'Visual Localization System', the docs describe how using Visual Localization on Loomo is more accurate than Odometry for navigation, as it uses the camera's depth images.  They provide a sample program for Visual Localization.

  https://github.com/SegwayRoboticsSamples/VLSSample 

  0

  Comment actions Permalink
• Silvio Vasiljevic

  • July 26, 2019 11:03

  Thanks for the swift reply!

  I'm aware that the ZR300 camera has support for the SLAM library. The problem is that the Loomo SDK still has not implemented this feature and that it is on their roadmap without an ETA. That is the reason why we are searching for a possibility to access the camera directly (without the Loomo SDK, but maybe an Intel SDK?) to implement the support by ourselves. According to the Loomo documentation we need to request an APK from them to even get access to the fish-eye camera. We requested this but did not receive an answer yet.

  Visual Localization and Odometry is something that we already use, but it does not take the circumstances of the robots surroundings into account which is why we need SLAM.

  0

  Comment actions Permalink
• MartyG

  • July 26, 2019 11:38
  • Edited

  The modern RealSense SDK 2.0 supports Android but unfortunately does not support the ZR300.  Since Loomo can make use of ROS (and the RealSense SDK fully supports it too), one option might be to mount a modern RealSense 400 Series camera to the Loomo and connect it up to the robot's motion motors via ROS.  There would have to be a USB port on the Loomo robot's Android hardware to attach the camera to though.

  You can build an Android app around the RealSense SDK, though that would likely mean that you would have two separate apps for the Loomo, the official Loomo one and another that manages the navigation functions of the robot.  If these SLAM functions are largely automated once setup is complete then this might not be too much trouble though.

  https://github.com/IntelRealSense/librealsense/blob/master/doc/android.md 

  https://github.com/IntelRealSense/librealsense/tree/master/wrappers/android 

  I do not have knowledge about using the RealSense Android SDK with ROS though, so you may be able to get some useful guidance on that by posting to the RealSense ROS GitHub site.

  https://github.com/IntelRealSense/realsense-ros/issues 

  0

  Comment actions Permalink
• MartyG

  • July 26, 2019 12:09
  • Edited

  There is an alternative route to investigate that comes to mind.  In 2016, Intel prototyped an Android phone with a ZR300 in it called the RealSense Smartphone Developer Kit.  It used 'Google Project Tango' augmented reality (AR) software, and Tango could do SLAM.

  Unfortunately the Smartphone Developer Kit was cancelled before it was released, and Project Tango was later also cancelled by Google.  Google's successor to Tango, called ARCore, can also do SLAM though, apparently.

  https://community.arm.com/developer/tools-software/graphics/b/blog/posts/indoor-real-time-navigation-with-slam-on-your-mobile 

  So, that offers a couple of possible paths for connecting to Loomo's ZR300 directly:

  (a) Find an old download of Google Project Tango on the net and see if it can talk to Loomo's ZR300 like Tango could to the Smartphone Developer Kit's ZR300.  Or 

  (b) Use the current Google ARCore software and see if that can make use of Loomo's ZR300.

  0

  Comment actions Permalink

Please sign in to leave a comment.