Correct way to integrate ZED2i in the urdf and setup ZED odom and map frames + GNSS fusion - ROS2

Software
1

Hello,

I’m having issues figuring out the correct setup for my multiple ZED2i cameras in an autonomous mobile robot, that also mounts 4 lidars.
I’d like to understand what’s the standard for it, as the documentation doesn’t cover the full scenario i have.

URDF:
I have two ZED2i cameras, one mounted at the front and the other at the back of the vehicle. The robot’s base_link serves as the reference frame.

Expected Configuration:

  • The front camera is supposed to publish /odom and /map frames, with /odom resulting from the fusion of VIO, IMU, and GNSS RTK.
  • The rear camera doesn’t participate in pose tracking.
  • The robot’s frame tree should follow the REP105 convention, with the structure (utm) -> map -> odom -> base_link -> … -> zed_front_camera_link.

I’ve tested several setups and am currently able to successfully fuse the GNSS RTK data and integrate the cameras with the desired functionalities.

However, I’m encountering issues when I try to properly reference the /odom and /map frames to the base_link, as they should be used for algorithms from slam_toolbox and nav2.
These frames are still attached to the zed_front_camera_link. Even after hardcoding the mBaseFrameId in zed_camera_components_main.cpp to base_link, I can only reference /odom (and the pose vector) to it, while the map remains fixed to the front camera position. Changing the initial_base_pose parameter doesn’t seem to resolve the issue for me.

I can explain further in subsequent replies and share tf2_trees and images, but for now, I just wanted to understand how you envision integrating the camera on a complex robot with pose tracking and GNSS fusion capabilities.

Thank you

2

Hi @foiegreis
Welcome to the StereoLabs community.

This section of the ROS 2 documentation

and this tutorial on the Nav2 website
https://docs.nav2.org/tutorials/docs/integrating_vio.html
will help you solve your doubts.

3

Hi Myzhar,

thank you for your reply. Of course I’ve checked the documentation before writing here in the forum.
The documentation doesn’t cover my full problem. As an example, the zed integration post supposes that with ZED pos_fusion (use_zed_localization) the reference frame of the robot would become zed_camera_frame.
Having a tree with the zed camera as reference is something i would not consider for my application.

Can we dive into the setup in deep?

Thank you

4

In this case, you must disable publish_tf for all the cameras and provide the correct TF chain as described here.

You can use the disable_tf parameter of the multi-camera launch file.

5

Hi @myzhar,

thank you for your reply, but my problem is quite more complex.
Of course I’ve tried everything and i know profoundly about REP105, which is basically the reason why I’ve started this thread.
The Front zed camera in my setup is the only provider of the odom and map frames, and odom data get fused with GNSS rtk. If i don’t publish the tfs with publisgh_tf, I will need static transforms to do that to me, which i would avoid. The odom frame in the zed camera is directly connected to zed_camera_link, which is not the REP105 standard nor the setup in my urdf.

I’m just pointing here that there must be a way you guys from Stereolabs imagined the cameras in a complex setup. And i may be missing something simple maybe, even if I’ve been working on these issues for months. I was able to perform autonomous navigation on a real big amr by tweaking the source code (which shouldn’t be necessary!) to refer the odom frame to base_link. But every time I add a functionality (Gnss fusion, this time), everything gets messed up again.

Note:
As I’ve mentioned above, in your guide about ZED integration, the only example with use_zed_localization builds a tree with zed_camera_link before base_link, contradicting REP105.

I would really appreciate 1to1 support/video call if possible in any way.
Thank you
Greta

6

No, if you use “static transforms” nothing will work.
You need instead a localization node.

This is not a problem as long as you provide a static TF from the base_link to the zed_camera_link in the URDF of your robot.
A while ago, the ZED ROS2 Wrapper was publishing odometry in the base_link frame, but this was causing a lot of integration issues.

This is incorrect. The REP105 indicates that a TF chain of type map → odom → robot_frame is required. If the robot frame is called base_link or zed_camera_link, it remains unchanged; it must serve as the reference frame for robot localization. When adding a GNSS sensor, you only need to provide the static transform from the GNSS antenna to the zed_camera_link, just as you would for a frame named base_link.