INTEL D455 MATLAB DEPTH IMAGE
Hello, I am doing my dissertation and need help.
I have an intel d455. I know how to take depth images in MATLAB but they come out with values from 0 to 255. Using this image, I wanted MATLAB to find for each column of the image the value of the closest pixel and then for each column to limit the depth between that depth value to the next 20mm.
So in all columns I would get only the depth of the closest point until 20mm later and the rest behind would disappear from the image.
-
Hi Up202109482 The mathematics is a little complicated. RealSense 400 Series depth cameras such as D455 represent their depth as 16-bit uint16_t 'pixel depth' values which have a minimum-maximum range of 0 to 65535. The real-world distance in meters is found by multiplying this depth pixel value by the depth unit scale of the camera, which for D455 is 0.001. For example, 6500 depth pixel value x 0.001 depth unit scale = 6.5 m real-world distance.
Every depth pixel of uint16_t is two bytes. The 0-65535 depth pixel values of uint16_t can be converted to OpenCV formats such as CV_16UC1 which have a range of 0-255. So it is not wrong that you are only getting values up to 255.
-
I considered your question carefully but could not identify a means to implement that mechanism in the MATLAB wrapper, unfortunately.
A less precise way of excluding background detail after a certain distance may be to set a paramer called Disparity Shift in a json camera configuration file and then load that json file into the MATLAB wrapper to apply the distance rendering limitation. An example of a MATLAB json loading script is **advanced_mode_example.m** at the link below.
https://github.com/IntelRealSense/librealsense/blob/master/wrappers/matlab/advanced_mode_example.m
In regard to creating a json file, an easy way to do this is to open the RealSense Viewer tool and click on the json export option near the top of the Viewer's options side-panel.
After the json file has been saved, open it with a text editor program such as Windows notepad or wordpad and look for a parameter called aux-param-disparityshift. It should be near the top of the list of parameters.
The default value of aux-param-disparityshift is '0'. Increasing the value of the Disparity Shift option reduces the maximum depth of the camera, causing depth values that are further away not to be rendered on the image. A good test value to change '0' to in the json file would be '100', and then increase it in increments of 50 until you are satisfied with the amount of background depth that has been removed.
-
Thanks for the answer. Yes, it could work. However I dont want to limit the depth as equal in all the image. I want to know the minimun depth corresponding to each column of the image. Then limit depth like [min Depth Value of the column, min Depth Value of the column + 10mm] in all the columns.
Real Application: I have a little communication tower made of wood. And I take one depth picture. However, it captures all the tower and background, but I only want the first surface of the tower, so the others behind and background are deleted. That's the real application. -
The link below provides advice about accessing the distance values of depth coordinates in the MATLAB wrapper instead of just rendering the colorized depth image like depth_example.m does.
https://github.com/IntelRealSense/librealsense/issues/9923#issuecomment-961845156
(x,y) represents horizontal width X and vertical height Y, so X will be columns and Y will be rows.
To find the nearest depth value of a column, you would therefore likely need to know all the row values for a particular X column value instead of looking at one XY coordinate at a time or all of them.
I do not have an example of how such a mechanism could be implemented with code in the MATLAB wrapper though, unfortunately.
-
Thank you so much, I already have the dis_matrix which helps a lot and I compared the values with Intel RealSense Viewer and for the same coordinate XY they match.
My main difficult was to get here. Now, I think I can evolve pretty quick. If I get a good result, I post it here.
One more time, thank you for answering me all the times.
Kind regards -
Hi Marty, when I take a depth picture there are severall black pixels. This is a big problem, since Matlab and Intel RealSense assume these black pixels are in a depth of 0.00m.
My Professor told me to take severall depth pictures in order to smooth the image and try to delete these black pixels. Do you know how can I do this?
Another question, it is possible to do 3D reconstruction in a certain depth from the scene that the camera captures in Matlab? -
It sounds as though you are being advised to implement a post-processing Temporal Filter in your project that uses historical data from several frames to handle missing or invalid pixel data. The Temporal Filter has the parameters 'Filter Smooth Alpha' and 'Filter Smooth Delta'.
https://dev.intelrealsense.com/docs/post-processing-filters#temporal-filter
The default value of Filter Smooth Alpha is '0.4'. Changing it to a lower value such as '0.1' can stabilize rapid fluctuations in depth values and result in less holes.
You can test the Temporal Filter in the RealSense Viewer by expanding open the Stereo Module > Post-Processing Filters section of the options side-panel, expanding open the 'Temporal Filter' option and editing the value of the Filter Smooth Alpha sub-option to change it to '0.1'. You can do this with the slider or by clicking on the pencil icon beside the option and manually typing the new value in with the keyboard.
An alternative approach to using the Temporal Filter would be to use the Hole-Filling post-processing filter to fill the holes in. You can also test this in the Viewer.
The Threshold Filter post-processing filter enables a minimum and maximum depth to be defined. Depth values outside of the defined min-max range are excluded from the depth image.
-
The MATLAB wrapper's scripting language is almost 1:1 with C++ and so looking at C++ references for a particular function can be useful, though there are of course differences when writing scripting for the MATLAB wrapper and there are not pre-existing MATLAB post-processing scripts available, unfortunately. It should be possible to adapt C++ code to access the filters from a MATLAB wrapper script though.
https://github.com/IntelRealSense/librealsense/issues/3311#issuecomment-466408511
The C++ post-processing example that the above link refers to is at the link below, and the RealSense user in that discussion was able to succeed in implementing filters in the MATLAB wrapper based upon this example.
-
Hi MartyG, hope everything is fine.
Meanwhile, I could extract depth correspondent to all the pixels of the image as shown in the next example:https://github.com/IntelRealSense/librealsense/issues/9923#issuecomment-961845156
After that, I go to every pixel of the "dis_matrix" and ignores all the values bellow 0.4, because I think that is the minimum distance that the D455 camera can measure. All values bellow that value are "errors". So I place them by NaN, so in the next steps they are ignored. This, because when capturing the depth image there are severall black pixels that correspond to a depth value of 0.
[rows, cols, color] = size(DepthImage);for i = 1:rowsfor j = 1:colsif sumDisMatrix(i,j) < 0.4sumDisMatrix(i,j) = NaN;endendendAfter ignoring these pixels, I go to every column and detect the minimum depth value. I do it in every column because in my specific case, the object is placed in a position that in every column as aprox. the same depth but along the same line depth change.
for j = 1:colsmin_col(j) = min(sumDisMatrix(:,j));endAfter this, I give one depth_tolerance to make sure that for the same column, all the pixels from the object are obtained.depth_tolerance = 0.02;for i = 1:rowsfor j = 1:colsif (sumDisMatrix(i,j) > min_col(j) && sumDisMatrix(i,j) < min_col(j) + depth_tolerance)img(i,j, color) = DepthImage(i,j, color);endendendimshow(img);
Please sign in to leave a comment.
Comments
14 comments