Fusing ZED odom and ZED IMU with GPS for robot localization


I’m trying to fuse the sensors using the robot localization package from ROS to produce a filtered odometry and filtered GPS. I would like to clarify a few things as I’m not getting the expected result I need. Here are the sensors I’m planning to fuse with:

  1. Wheel encoder odom: frame ID odom and publishing at 100 Hz
  2. IMU in ENU convention: frame ID imu_link and publishing at 50Hz
  3. Zed2i odom
  4. Zed2i imu data

Firstly, how can I make sure that all the sensors I’m fusing are aligned? These sensors all have different initial values and publishing rate. Do I need to calibrate them? For examples my IMU heading and the heading from zed node are different. Other than that, I also realised the TF axis for ZED is different from the default convention used by robot localization, as ZED z-axis is forward-backward while robot localization’s z-axis is up-down. How do I make sure that they are all aligned? Is it by simply publishing a static TF between the frames?

Secondly, I’m also hoping to make use of the pose published by ZED ROS wrapper as well as a GPS sensor. From the github issue ([Question] Fusing ZED2 odometry and IMU with wheel odometry · Issue #603 · stereolabs/zed-ros-wrapper · GitHub), I noticed that it is necessary to have 2 stages of pose estimation. So does it mean that I can only use the pose for the second stage along with the odometry/gps coming from my navsat transform node?

Sorry for asking vague questions and it is more related to the robot localization package from ROS. Any help or guidance in regard of this is greatly appreciated.

Thank you.

Hi @joewong00

You must calibrate your system and understand how axis are aligned.

This is wrong. The ZED ROS Wrapper follows the specifications of the REP 103 and REP 105 ROS documents, so all the data is provided in the ROS coordinate system: X FORWARD, Y LEFT, Z UP.

Yes, URDF and eventually static TF publishing are used to provide correct axis alignment if sensors and not aligned while mounting them on the robot.

It’s up to you to decide how to fuse all your localization information, according on what type of information you have and how they are precise.

1 Like


Thanks for the reply. So if I want my ZED imu data to be aligned with my other IMU data such that they will report the same yaw angle, I need to calibrate the magnetometer in the ZED camera right? I saw the tutorial on using ZED sensor viewer to perform the calibration by rotating the camera in space while the camera is mounted on the robot. However, I’m using ZED camera on a heavy ground robot and I can only rotate the robot (in yaw axis) but not the other axis. Is it sufficient? Otherwise how do you suggest I perform this calibration?

Thank you.

Unfortunately, that’s not enough for a 3D calibration. You can search instead 2D calibration tools and use them with ZED RAW magnetometer data.

I see. May I know where the calibration file is stored? I would like to calibrate only one camera and use the same calibration file for the other cameras.
Thank you.

That’s not possible. The magnetometer calibration must be performed in place.
Each sensor has its own calibration that depends on many factors.
Furthermore, the calibration is stored internally on the camera.

@joewong00 FYI the ZED ROS2 Wrapper provides the support for the GeoTracking module of the ZED SDK v4 that allows you to directly fuse GNSS information with the ZED Visual Inertial Odometry:
GitHub: GitHub - stereolabs/zed-ros2-wrapper: ROS 2 wrapper for the ZED SDK
DOC: Adding Geo Tracking in ROS 2 | Stereolabs

Alright thanks. May I know what are some tool that allows me to 2D calibrate the magnetometer?

I’m sorry, we never tested them, you should search on the internet