[D455]Stairs detection and measurement

• MartyG

  • July 15, 2022 07:40
  • Edited

  Hello,

  A robot product at the link below that is equipped with 4 RealSense cameras implemented handling of stairs by placing a camera between the legs of the robot.

  https://twitter.com/agilityrobotics/status/1100477143241216000?lang=en

   

  A simple approach to measuring stair height may be to measure the distance at the center of the camera's field of view and subtract the known distance between the camera lenses and the base of the equipment that the camera is mounted to.  The remainder of the detected distance should be the distance to the top of the step that the camera is currently positioned above.  This approach will only calculate one step at a time though.

   

  For calculating multiple stair steps simultaneously, a deep learning based solution may be required.  Such a tool can look at an image and automatically pick out and analyze all the occurrences of a particular element of the scene that it has been trained to recognize.

   

  Some researchers have used point cloud data from a tool called PointNet for stair detection.  

  http://stanford.edu/~rqi/pointnet/

  https://github.com/charlesq34/pointnet

   

  An example of a stair detection project that used Python and PointNet is here:

  https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9782312

  0

  Comment actions Permalink
• 623123112

  • August 05, 2022 08:41

  Hi MartyG,

  Thank you for your reply.

  Now, I want to use point cloud of each side of the stair to measeure the size of the stair by calculating the length of each direction, e.g. maxY-minY. Is there any solution to detect and divide each side of the stair? I have tried a project with region growing but it cannot detect in real time (normally 30 fps but reduced to <10 fps when stairs are detected). Can you give me some advice?

  Thanks.

  0

  Comment actions Permalink
• MartyG

  • August 05, 2022 14:03

  Would measuring between two points instead of between the camera and a point meet your requirements?  The link below has an example of such a script for Python.

  https://github.com/soarwing52/RealsensePython/blob/master/separate%20functions/measure_new.py

   

  This script was originally a conversion of the RealSense SDK's rs-measure C++ measuring example.

  https://github.com/IntelRealSense/librealsense/tree/master/examples/measure

   

  The Python measuring script was later used as part of a larger project to measure between the two far sides of a road, similar to the opposite ends of a stair step.

  https://github.com/soarwing52/RealsensePython#in-arcgis

  0

  Comment actions Permalink
• 623123112

  • September 22, 2022 03:31

  Hi MartyG,

  Thanks to your help, I have achieved the detection and measurement of stairs.

  Now I want to know if I can use imu data to transform the coordinates of the point cloud.

  Just like the image below, when the camera is not horizontal, I want to change the coordinates of the point cloud from xyz to x'y'z ( from a camera based xy coordinates to a ground based x'y' coordinates).

  Thanks.

   

  

  0

  Comment actions Permalink
• MartyG

  • September 22, 2022 05:47

  You could perhaps monitor the IMU's roll and pitch angles (tilt left-right and up-down respectively) and if the angles are a certain amount away from the straight-ahead and flat position, adjust the coordinates.  A Python discussion at the link below looks at how yaw and pitch values might be retrieved.

  https://github.com/IntelRealSense/librealsense/issues/4391

   

  In regard to transforming between the camera's 0,0 xy and a custom 0,0 xy position, a RealSense user at the link below did so by doing a subtraction sum of the camera xy - their offset position's xy.

  https://github.com/IntelRealSense/librealsense/issues/9749

  0

  Comment actions Permalink

Please sign in to leave a comment.