using d435 and t265 on mobile robot 1.6 meters high
I have robot almost like turtlebot3 but with mast. Whole construction is 1.6 meters high.
I want to build map. To do this I intend to use d435+t265 and a lot of ultrasonic sensors around the robot. Now, I am testing only d435+t265 and software for mapping algorithm.
what can you suggest about what is the best solution about locating these cameras? For example, in the middle of the robot oriented straight, or somethere in upper part oriented towards the floor.
Also, I use occupancy-mapping branch from realsense-ros as mapping node, here is the link:
https://github.com/IntelRealSense/realsense-ros/tree/occupancy-mapping
There are parameters for mapping algorithm which is not clear for me. In occupancy.launch file:
Here, I draw this picture about how i understand it:
is this correct?
The problem is that I tried to do as it drawn on this picture and it didnt worked, there is no obstacles on the map. I did some experiments with changing this numbers and it worked almost well, but I need clear accurate definition of this numbers and how it applies in my case with cameras angled downwards.
Thank you in advance!
-
Your robot concept is similar to Tally, a retail inventory robot using open-source tech that combines lidar with RealSense. So it is a good reference source for working out your own design.
https://www.youtube.com/watch?v=k2lBP2dP0is
I would think that if you are aiming to build up a 3D map in real-time by having the robot patrol then having the D435 oriented straight-ahead will be fine. Even if the field of view cannot see the floor at close range, it should be able to see the floor as it enters the range of the camera's field of view. If you have the robot keep patrolling then eventually it should be able to build a map that has sufficient detail, because it can keep going over the same area and recording detail that it might have missed during previous patrols.
The ideal situation may be to have two D435 cameras stacked in a vertical 'upper and lower' orientation on the robot if possible. That way, you will be able to capture the full detail of the location faster and should also have higher-quality data (because where the two fields of view overlap, there will be redundancy in the data due to both cameras covering the same area). I think you could certainly do it with just a single D435 though.
Once you have recorded a map and are satisfied with the amount of detail in it, you can turn off the real-time recording mode and have the robot follow the fixed map but still be able to respond to obstacles in real-time.
In regard to advice about occupancy mapping, the best place to ask about that will be on the RealSense ROS GitHub forum.
-
Thanks, I think its a good idea, using two cameras oriented straight-ahead. I think it is the best solution for this kind of robot.
I have got idea about cameras oriented towards floor from Tory robot:
https://www.youtube.com/watch?v=LGTkL_gacvU
It is also tall robot. What do you think why they decided to use such design?
Looks like Tally uses one camera straight-ahead, or they use two cameras oriented straight-ahead? Where are cameras located?
-
Tally has a lidar camera for navigation and a RealSense camera - I believe it was a ZR300 model originally (the stereo-camera predecessor of the D435), I do not know if they have upgraded it to a 400 Series since then. I believe the cameras are located at mid-height, just below the display.
The current Tally model apparently has over a dozen on-board cameras, most of them located on the side edge of the robot.
Details of Tally's cameras can be found here:
https://www.simberobotics.com/platform/capture/
Another ZR300-equipped robot is Segway's Loomo, though its height is shorter than Tally.
https://youtu.be/K0WEynLsMI0?t=3960
My best guess about the height of the Tory robot is that the upper part is its RFID antenna.
-
So they have only one depth camera located at mid-height, oriented straight-ahead. Thats good to know. Thank you.
About side cameras, are they also using either for mapping or navigation? I guess they are only for scanning inventory.
About details of Tally's cameras, there is nothing about Tally's cameras in your link. But its actually not important i think, just some depth camera. I am interested more about position of camera.
About Tory, I meant, why they decided to put camera oriented towards floor?
-
The angled edges of Tory's casing around the camera make it hard to see the camera component's orientation but it does seem to be tilted slightly downward diagonally. This would make sense for scanning the bottom level of shelves.
Given that Tally is facing forwards whilst travelling down the aisles and past the shelves, it does seem likely that the side cameras are doing the stock scanning whilst the forward cameras handle functions such as obstacle avoidance (not running into customers).
-
During follow-up research, I found a tall robot called Digit that uses 1 lidar camera and 4 RealSense cameras that "provide complete ground coverage from approx +2 meters to -1 meter ... there are 2 RealSense cameras on the front, 1 between the legs, and 1 on the rear".
https://twitter.com/agilityrobotics/status/1100477143241216000?lang=en
-
It is hard to tell for sure what part of Tory is doing what without technical details available about the camera placement. Being able to see the floor is certainly useful though. As mentioned about, the Digit robot can see to -1 m height (so it can handle descent down stairs), and that is likely provided by the camera between its legs rather than the front cameras.
Wikipedia has an article on the basics of occupancy mapping:
https://en.wikipedia.org/wiki/Occupancy_grid_mapping
I am not familiar enough with occupancy mapping on RealSense to comment on its algorithm. The RealSense ROS GitHub is the best place to ask about it.
Please sign in to leave a comment.
Comments
8 comments