Realsense_ROS Point Clouds Callback Error
I have encountered error messages as below when I am running two d455 on my machine (realsense_ros 2.42.0):
No stream match for point cloud chosen texture process - Color
An error has occurred during frame callback: Error occurred during execution of the processing block
It was fine until I set the frame rate of both cameras to 30. I am using NUCi7 with 32G RAM. It shouldn't reach the limit of performance from what I can see. I would like to know how to fix this error.
-
Hi Victor Which method are you using to launch your two cameras in the RealSense ROS wrapper, please. Are you launching each camera in a separate ROS terminal with rs_camera.launch or both with a single rs_multiple_devices.launch instruction?
https://github.com/IntelRealSense/realsense-ros#work-with-multiple-cameras
When performing depth to color alignment, the RealSense SDK's align processing block will automatically adjust for differences in resolution and any differences in field of view size of the depth and RGB sensors. A resolution difference should not affect point cloud quality.
-
I'm using rs_camera.launch to launch two cameras under two terminals.
The error message is linked to frame drops. I tried to turn off the infrared stream transmission of two cameras and it seems to improve a little.
When I updated to 2.50.0, this error message is gone. I have looked into the release note and found there is a fix to enable patch Linux kernel. Does that suggest if I install from source code it will also patch the kernel? What does it mean if I have new error messages of "Pixel format xxxx likely requires patch for code RW16" ?
-
I would like to ask another question since I don't want to pepper the community with my thread of questions :)
I would like to crop the point clouds. But it is not a bounding box to highlight the object of interest. The target is about 5m away from the camera. I want to remove the edge of point clouds and only keep the middle FOV because of the ghost noise. I know there is clip distance applied for the camera range. Is there anything similar to it for camera (point cloud) FOV?
-
Yes, disabling infra2 is a valid way to reduce bandwidth usage in the ROS wrapper if you do not need the right-hand infrared stream. Doronhi the RealSense ROS wrapper developer has said about doing so: "It will have no effect on the depth quality. It only disables the infra2 images' transmission via the USB port. They are still taken and the depth is calculated using them inside the device".
The reference in the 2.50.0 release notes [Linux] Enable patches for kernel 5.11 refers to adding support for use of 5.11.x versions of Linux kernels with librealsense and does not mean that kernel patching is automatically performed when building from source code.
If librealsense is built from source code then you usually need to apply a kernel patch to enable support for hardware metadata and to avoid the pixel format .... likely requires patch for fourcc code RW16! errors that you mentioned, which patching resolves.
You do not need to patch though if you build from source code using the RSUSB backend build method, which is not dependent on Linux versions or kernel versions. Patches and metadata support are also included without the need for patching when installing librealsense from DKMS packages instead of building from source code.
Builds on the librealsense SDK on Linux are configured to use the V4L2 backend by default. The link below has information about V4L2 pixel formats.
https://www.kernel.org/doc/html/v4.14/media/uapi/v4l/pixfmt-intro.html
It is possible to crop x and y on a 3D pointcloud using bounding boxes, but it is not simple and there are few RealSense related references for doing so. The best of the available references that I know of is in the link below.
-
I found the other post that talks about Realsense backend: https://github.com/IntelRealSense/librealsense/issues/7904
I understand what their stack looks like in a general sense. But I am hesitant between two backend options. Could you explain what difference they have? I'm looking for answers in performance when running two cameras with 1280*720 at 30 fps on NUCi7
-
The link below discusses the advantages and disadvantages of using an RSUSB installation (also known as libuvc backend) versus a patch-based installation. Please scroll down through the comment to the section headed What are the advantages and disadvantages of using libuvc vs patched kernel modules
https://github.com/IntelRealSense/librealsense/issues/5212#issuecomment-552184604
It is worth adding that at a later date after the posting of that information in 2019, RealSense SDK version 2.35.2 in 2020 made improvements to multiple camera handling under RSUSB.
https://github.com/IntelRealSense/librealsense/pull/6467
Further work on RSUSB was done in the recent SDK 2.50.0.
-
I rebuilt from the source code 2.50.0 using -DFORCE_RSUSB_BACKEND=true
I can verify that RSUSB is enabled by checking viewer -> port. It gives me 2-1.1-9 which is of libusb format. Though I try to run one camera, I can see periodical warnings say that control transfer return error: resource temporarily unavailable, number 11. I use 848x480 at 30Hz for both color and depth stream and the warnings message pops every 5 seconds
Do you have any idea about that?
-
The control_transfer returned warnings in the log can be safely ignored if they are only appearing at the start of launch or appearing about once a minute instead of generating continuously. If the control_transfer returned warnings are generated continuously though then this may indicate a serious communication problem with the device.
The topics may still be able to be correctly published in the RViz interface even if these messages appear. If the data is publishing correctly and smoothly in RViz despite the warnings then the warnings can again be ignored.
Could you test please whether adding initial_reset:=true to your roslaunchh instruction to reset the camera at launch makes a positive difference, please? For example:
roslaunch realsense2_camera rs_camera.launch initial_reset:=true
-
The hardware reset does help somehow.
Now I have been testing the use cases of streaming four d455 with point clouds on my i7 NUC machine. I launched four cameras individually in different terminals by using rs_camera.launch. What I have found is that the fourth camera received the error of failing to allocate memory, number 12. No point cloud message was sent from the fourth camera. The average rate of point cloud topics from camera1 to camera4 respectively are: 29Hz, 30Hz, 29Hz and no message. The camera1 also has control_transfer returned warnings at an occurrence rate of 3-5s while the camera2 and the camer3 are fine.
I want to have four cameras to run with point cloud filters on one machine stably (15-30 Hz). Could you provide some opinion about how to optimize camera performance without failure or errors?
-
Whilst an i7 processor would normally be sufficient for 4 cameras active simultaneously, generating a pointcloud from each camera may be creating an additional processing burden.
If you are not able to reduce the resolution, you could try using a decimation filter to reduce the depth image's complexity. For example:
roslaunch realsense2_camera rs_camera.launch filters:=pointcloud,decimation
https://github.com/IntelRealSense/realsense-ros/issues/1924#issuecomment-858369624
Please sign in to leave a comment.
Comments
11 comments