ZED Multicamera Startup Errors

[privvyledge]

We successfully connected 4 ZED 2i cameras to an Nvidia Jetson AGX Orin using a PCIe expansion card and tested opening them using the ROS2 Humble ZED wrapper. However, when starting all cameras, the following error always pops up and persists for about 15 seconds but then the cameras are able to start.

[zed_wrapper-2]  FileLocker - Thread 281473114798144 - #32 File lock timeout. Is the process that keeps the file frozen?
[zed_wrapper-4] [WARN 1714924697.351228095] [lidar.right.zed_right.zed_node]: Error opening camera: CAMERA NOT DETECTED (startCamera() at /kart_ws/src/zed-ros2-wrapper/zed_components/src/zed_camera/src/zed_camera_component.cpp:3574)

The cameras successfully start after displaying the errors above but when closing and relaunching, the drivers don’t start again.

We are using ZED SDK 4.0.8 with the associated ROS2 Humble Wrapper Release version.

[Myzhar]

Hi @privvyledge

@privvyledge

using a PCIe expansion card

What is the card that you are using?

@privvyledge

The cameras successfully start after displaying the errors above but when closing and relaunching, the drivers don’t start again.

I recommend calling reboot for all the cameras before restarting the node.

I suspect you have a USB3 conflict that causes the controller to disable one of more cameras.

@privvyledge

We are using ZED SDK 4.0.8 with the associated ROS2 Humble Wrapper Release version.

I recommend installing the latest ZED SDK v4.1 and updating the ROS 2 Wrapper to the latest version of the master branch of the repository.

How did you install ROS 2 Humble? Are you using Docker or did you compile it from the source?

[privvyledge]

@Myzhar

What is the card that you are using?

The StarTech 4-Port USB PCIe Card - 10Gbps USB 3.2 Gen 2 Type-A PCI Express Expansion Card with 2 Controllers found here: link. From the spec sheet, it should have 2 USB controllers with a total bandwidth of 20Gbps. Although, I only connected 2 Zed 2i’s to it while I connected the other 2 to a USB 3.1 hub connected to the Jetson (I’m guessing I should just connect the other 2 directly?).

@Myzhar

I recommend calling reboot for all the cameras before restarting the node.

Is there a ROS2 service to accomplish that? Or could I add a respawn or TimerAction/delay to accomplish this? Do you guys have any examples of doing this via ROS2, or do I need to do it in a Python node or something?

@Myzhar

I suspect you have a USB3 conflict that causes the controller to disable one of more cameras.

I see. This might be related to connecting 2 of the cameras to the USB hub. Even though they work once, I think I’ll connect them all to the PCIe expansion card.

@Myzhar

I recommend installing the latest ZED SDK v4.1 and updating the ROS 2 Wrapper to the latest version of the master branch of the repository.

I will do that. I tried previously, but it failed to compile because the ROS2 driver didn’t support 4.1, but now I see it’s been updated, and I’ll try again with SDK 4.1.1 since that version supports GNSS fusion as well.

@Myzhar

How did you install ROS 2 Humble? Are you using Docker or did you compile it from the source?

Yes, I am using Docker. I used a ROS humble Ubuntu base image and manually installed all the required Jetpack/CUDA/TensorRT components. I also have versions where ROS was compiled from the source and your docker images, but I haven’t tested those with the multi-camera setup. However, I don’t think the docker image used is a factor.

[Myzhar]

@privvyledge

The StarTech 4-Port USB PCIe Card - 10Gbps USB 3.2 Gen 2 Type-A PCI Express Expansion Card with 2 Controllers found here: link. From the spec sheet, it should have 2 USB controllers with a total bandwidth of 20Gbps. Although, I only connected 2 Zed 2i’s to it while I connected the other 2 to a USB 3.1 hub connected to the Jetson (I’m guessing I should just connect the other 2 directly?).

Please do not use the hub. I recommend connecting instead only one ZED 2i to the Jetson and 3 to the PCIe card.

@privvyledge

Is there a ROS2 service to accomplish that? Or could I add a respawn or TimerAction/delay to accomplish this? Do you guys have any examples of doing this via ROS2, or do I need to do it in a Python node or something?

A Python script is the better option. You could create a Python node that advertises a service to do that. Please note that you do not need to call the open function to call the reboot function for a specific camera.

@privvyledge

I see. This might be related to connecting 2 of the cameras to the USB hub. Even though they work once, I think I’ll connect them all to the PCIe expansion card.

@privvyledge

I will do that. I tried previously, but it failed to compile because the ROS2 driver didn’t support 4.1, but now I see it’s been updated, and I’ll try again with SDK 4.1.1 since that version supports GNSS fusion as well.

We will improve the GNSS support with v4.1.1 the next week.

@privvyledge

However, I don’t think the docker image used is a factor.

I agree

[privvyledge]

@Myzhar

Please do not use the hub. I recommend connecting instead only one ZED 2i to the Jetson and 3 to the PCIe card.

I set it up this way but the error File/thread lock issue still occurs causing a startup delay and now one of the cameras does not start. For the camera not starting it shows the following error: Error opening camera: CAMERA NOT DETECTED.