Visual SLAM with multiple cameras facing in opposite directions

Software
1

Hi everyone,

I want to integrate up to 4 ZED2i cameras that are rigidly mounted to a robot, all facing in different directions into the SDK’s provided visual SLAM (or the spatial mapping) via the Fusion API on a Nvidia Thor AGX Dev Kit.

I’ve been reading both the Fusion API and positional tracking documentations, but I could not find entirely clear information on if this is possible for the use case of mobile VSLAM, as the examples were mostly focussing on body tracking applications.

It would be great to know, if this could be done as it would be of great help in my upcoming research project.

Thanks already in advance for the information and best regards

Noel

2

Hi @noelblunder
Welcome to the Stereolabs community.

The main problem here is that the Nvidia Thor AGX Dev Kit provides a single USB 3 channel, so it cannot support more than 2x USB3 ZED 2i cameras.
Furthermore, the devkit does not provide a PCIe expansion port to connect a PCIe USB3 adapter to add additional channels.

To use 4 stereo cameras with Thor, you should consider using 4x ZED X GMSL2 cameras with a GMSL2-enabled carrier board from one of our partners, like ConnectTech and Advantech.

3

Hi @Myzhar,

Thanks for your swift response!

So, do I understand correctly, that both the 2x USB-A and the 2x USB-C connections run through the same channel and therefore the overall transmittable bandwidth is the limiting factor?

Regarding the GMSL2 carrier board, that was initially my ideal solution and I already contacted ConnectTech, but the response was that the GMSL2 carrier board they sell is only compatible with their proprietary edge devices and cannot be used with the Thor T5000 Dev Kit. For Advantech, the products in their store look to only serve a similar purpose.

Do you have experience with a specific carrier board of one the two manufacturers that you know works with the Thor Dev Kit to successfully connect the ZED X cameras? That would honestly be my ideal solution.

Thanks already and best regards,

Noel

4

Yes, this is correct.

Yes, this is correct. There are no existing solutions for the development kit. It has no expansion ports.

5

Okay, so to summarize what I’ve learned:

  • For the Jetson AGX Thor Development Kit, there is no known possibility to connect and fuse more than 2 ZED2i USB cameras.
  • For the Thor Dev Kit, there is also no way of integrating the GMSL2 ZED X cameras, as neither the Orin compatible boards can be connected to Thor devices, nor the listed partners are providing boards that are compatible with the Thor Dev Kit.

One last question before resolving this thread:

  • Considering only 2 ZED2i cameras, is there a way to combine them via the Fusion API or the new Sensors API from the 5.1 SDK Version to use as a single input for the visual SLAM from the positional tracking examples to widen the covered FOV?
6

This is a feature that will be released soon. Currently, the Fusion API only uses the positional tracking information from the main camera of the multi-camera system. Fusion is only available with GNSS information.