Open source software for realtime 3D scanning with the D435i and T265
Hi,
I am looking at getting the RealSense D435i and T265 to use them in tandem and produce printable 3D models. I know that DotProduct3D and a couple of other companies have the software, but these are expensive paid solutions. I am doing this for a non-commercial hobby and just want to ensure that I can utilise the hardware without ongoing fees for software (or an additional and substantial upfront cost).
Has anyone used or is aware of an open-source option that will support both cameras together? This is not the first place I have come, but it has been difficult to find.
Thanks
-
Intel has published a guide to using a T265 and depth camera together, which includes a section on the benefits of this pairing for scanning and 3D reconstruction.
https://www.intelrealsense.com/depth-and-tracking-combined-get-started/
A cost-free approach would be to take a point cloud scan with the two devices and save it as a point cloud data file.
https://github.com/IntelRealSense/librealsense/tree/master/examples/tracking-and-depth
That file can then be converted to a solid mesh model using free / open source software such as MeshLab or Pyntcloud.
https://github.com/daavoo/pyntcloud
If you would prefer a commercial quality pre built solution that is free, you could try a D435i on its own with LIPScan 3D.
https://www.microsoft.com/en-us/p/lipscan/9nc85cq8b1p9
https://youtube.com/watch?v=N14Pi6z-MkE
For your particular application actually, the ideal choice of depth camera will likely be the $149.99 D415, which is a good option either alone or paired with T265 for 3D reconstruction purposes. This is because the D415 excels at 3D scanning small objects and has around 2.5x the image quality of the D435 models.
A D435i would be an appropriate choice if you had been using it with the T265 on a fast moving mobile robot due to the D435's faster shutter. But for model scanning, D415 should give you better results and is also compatible with the LIPScan 3D free software..
-
Hi Marty,
Thank you very much for your response. I looked into LIPScan and it looks like a great resource (that I will get) but I think using them both would be beneficial - I had read about the applications together and will print the 3D mount if I get them.
This idea of getting a point cloud image could be a great way. I had a quick read and it appears that point cloud can piece many images together and use tracking data for additional accuracy, so I could handle the extra step to convert the images. What program would you suggest for the point cloud capture?
Alex
-
You are very welcome. You may find additional useful RealSense mount blueprints at the Thingiverse website.
https://www.thingiverse.com/search?q=realsense&dwh=95e23ffeb1607d
RealSense does not currently support iOS (it does support Android), though the RealSense SDK Manager suggests a workaround for iPad Pro.
https://github.com/IntelRealSense/librealsense/issues/5552
The tracking and depth sample program references a link to a calibration routine to align the point cloud with D4xx and T265 on the same mount. The link was broken but I tracked down the correct one.
Assuming that a calibration routine is needed to incorporate the T265 pose data into the point cloud, using the sample program with the provided mount blueprint and saving the resultant point cloud from it by adding some point cloud export to file code may be the way to go.
If you do not mind programming, a member of the RealSense team also wrote a program to do the device calibration via OpenCV and the Python language. Though that is for people who want to calculate their own transformation matrix instead of using the calibration supplied in the link above.
Please sign in to leave a comment.
Comments
4 comments