two D435i set 1920x1080 can't start pipeline at the same time
I am testing this using two D435i cameras with the image size set to 1920x1080. While I can start the pipeline on the first camera, calling `pipeline.start(config)` on the second camera causes it to freeze, and the process does not complete. I have tested this on a Raspberry Pi 4 (2GB), Raspberry Pi 5 (8GB), Jetson AGX, and a PC (Ubuntu 24.04), and the same issue occurs on all systems. Below is sample code that reproduces this issue. You will need to connect two D435(i) cameras to the PC. If you set `image_size = [1280, 720]` on line 15, two images (abc0.jpg and abc1.jpg) are captured from the two cameras. However, if you set it to `[1920, 1080]`, the program freezes on line 25 (`profile = pipeline[-1].start(config)`).
Thank you
import pyrealsense2 as rs
import numpy as np
import cv2
pipeline = []
context = rs.context()
for idev in context.query_devices() :
serial_number = idev.get_info(rs.camera_info.serial_number)
pipeline.append( rs.pipeline() )
config = rs.config()
config.enable_device(serial_number)
image_size = [1280, 720]
# image_size = [1920, 1080]
fps = 6
config.enable_stream(rs.stream.color, image_size[0], image_size[1], rs.format.rgb8, fps)
if image_size[0] == 1920 :
config.enable_stream(rs.stream.depth, 1280, 720, rs.format.z16, fps)
else :
config.enable_stream(rs.stream.depth, image_size[0], image_size[1], rs.format.z16, fps)
try :
print("start pipeline")
profile = pipeline[-1].start(config)
except RuntimeError:
pipeline = None
print("Error initializing Realsense failed in setup_camera !")
raise
print("pipeline ")
device = profile.get_device()
color_sensor = device.first_color_sensor()
color_sensor.set_option(rs.option.enable_auto_exposure, True)
for idx in [0, 1] :
frames = pipeline[idx].wait_for_frames()
color_frame = frames.get_color_frame()
depth_frame = frames.get_depth_frame()
color_image = np.asanyarray(color_frame.get_data())
depth_frame_data = depth_frame.get_data()
depth_image = np.asanyarray(depth_frame_data)
cv2.imwrite("abc{:d}.jpg".format(idx), color_image)
-
Hi Taku Osada Thanks very much for your code, which is very useful for diagnosis to see your method. I am unable to test RealSense users' scripts on my own computer though.
Please test the pyrealsense2 image file script at the link below (which saves image files when the 's' key is pressed), editing it to change the color resolution to 1920x1080 and changing the serial numbers of the two cameras to the serials of yours.
https://github.com/realsenseai/librealsense/issues/1735#issuecomment-390840337
This script defines a separate pipeline and config for the two cameras. If it works with 1920x1080 then it could indicate that there is an issue in your script's two-camera pipeline start method.
-
Hello MartyX Grover
Thank you for the advice. I copied the script from #1735 and changed the two serial numbers and the color stream size to 1920x1080.
When I run the script, it freezes at the same line as in my script (pipeline_2.start(config_2)). This means that my error is reproduced even in the script from #1735.
I also tried adding a self-powered USB hub to supply power to the two RealSense cameras, but the result was the same.Since the script freezes just before the line `pipeline.wait_for_frames()`, I haven’t tried the second suggestion yet.
-
This may not make a difference to obtaining 1920x1080 resolution if 1280x720 resolution is working correctly in your script, but please next try identifying the RGB sensor by an index number of '1' in your color_sensor definition instead of looking for the 1st color sensor.
color_sensor = profile.get_device().query_sensors()[1]
Also, please try running your script with only one camera attached at a time with the resolution set to 1920x1080. If it works with one camera but not the other, this could suggest that one of the cameras has a problem with accessing 1920x1080 resolution.
-
Hello MartyX Grover
I did two experiments.
test 1) only one camera set to 1920x1080 is connected to PC
I connected the each camera set to color1920x1080 and depth1280x720 on PC. Both camera worked fine.
test 2) changing the resolution
I tested #1735 script in various resolution. result is below
1st camera | 2nd camera | result
case | color | depth | color | depth |
---------------------------------------------------------------
A1 | 1920x1080 | 1280x720 | 1280x720 | 1280x720 | bad
A2 | 1920x1080 | 848x480 | 1280x720 | 848x480 | good
B1 | 1920x1080 | 1280x720 | 640x480 | 1280x720 | bad
B2 | 1920x1080 | 1280x720 | 640x360 | 1280x720 | good"good" means both camera capture image
"bad" means the script freeze in the line 'pipeline_2.start(config_2)'
In case A, color camera resolutions are fixed, and depth camera resolutions are changed.
In case B, color camera resolution of the 2nd device are changed, and depth camera resolutions are fixed.
In both cases, swapping 1st and 2nd device setting yields the same result.
I guess that there is a limitation of total amount of data in a USB system or librealsense2 ?
-
The USB system can typically transfer a maximum data 'bandwidth' of 5 Gigabits per second (Gbps). According to a multiple camera testing table at the link below, two cameras with depth and RGB at 1280x720 will use around 1769 Mbps / 1.77 Gbps. It is conceivable that you could be hitting the 5 Gbps limit when using 1920x1080 resolution for the RGB color with two cameras.
https://dev.realsenseai.com/docs/multiple-depth-cameras-configuration#2-multi-camera-considerations
A way to test this would be to have one of the cameras attached to a built-in USB port on the computer and one on a USB hub (as the hub will have its own 5 Gbps bandwidth allowance).
There are also USB hubs available with a 10 Gbps bandwidth allowance.
-
I got a StarTech PEXUSB3S44V which is 4 port PCI Express USB3.0 card with 4 dedicated 5Gbps channels. When I can simultaneously display two D435i RGB(1920x1080) and depth(1280x720) images with 30fps by Realsense viewer. However, My code in my first post still freezes on line 25 (`profile = pipeline[-1].start(config)`).
-
I’ve finally managed to capture RGB (1920x1080) and depth (1280x720) images from two D435i cameras using my code. The issue was caused by an outdated version of the pyrealsense2 library (ver. 2.50.0). After installing the latest library (ver. 2.57.7) using the command “pip install pyrealsense2,” I was able to capture 1920x1080 images from both cameras connected to the same USB 3.0 controller. This means that the data volume in this case falls within the bandwidth limits of the USB 3.0 controller. Thank you for all the advice. I also apologize for my simple configuration error.
Please sign in to leave a comment.
Comments
11 comments