Synchronization between IMU and images using Ros1

I am using ZED2 for vidual inertial odometery, I collected the data over ROS1. Is the IMU data in synch with the images to be able to use tightly coupled approaches. Is there a way to run the camera at such hardware level synchronization?

1 Like

Hi @kha83263
the default configuration of the ZED ROS Wrapper node sets the IMU in free-run mode.
You can enable image-IMU sync be setting the parameter sensors/sensors_timestamp_sync to true

This is the link to the parameter for ZED2 (zed2.yaml), you can find it also in zed2i.yaml and zedm.yaml:

Thanks a lot that is helpful. I am not sure this question makes sense but do you know if this synchronizes the sensors on a hardware level, or software? Please correct me if my question does not make sense.

1 Like

Hi @Myzhar I have one question here…

How are images and IMU measuremetns synchronized in ZED 2/ZED 2i?

The following doubt comes out because I am wondering if the ZED2 camera would work for outdoor SLAM.

I was reading the document: https://www.stereolabs.com/docs/sensors/time-synchronization/#getting-time-synced-sensors-data

And as far I understand a frame will be paired with only one IMU measurement using any of the TIME_REFERENCE flags (TIME_REFERENCE::IMAGE and TIME_REFERENCE::CURRENT).

Then I read this issue related to ZED mini: https://github.com/stereolabs/zed-ros-wrapper/issues/243. It points out that a bunch of measurements are going to be retrieved along a frame (and not just one as is explained in the official document). Moreover, all the IMU measurements will have the frame timestamp.

I need to have IMU and camera synchronized but with their correct timestamps (without override IMU timestamp with Frame timestamp). Is this possible with the ZED 2i?

Thanks in advance!

Hi @taihu
that’s an old issue and it have been fixed a long time ago.
When you activate IMU/frame synchronization the sensors topics will be published only when the relative frame is published.

1 Like

Thanks @Myzhar for your answer. Now, I have a few more doubts:

What happens when the synchronization between both sensors is disabled?
As far as I understand the IMU and image timestamps will be set to the host clock? Are these timestamps reliable? what i mean: is there any drift, offset or lag between the IMU and the camera timestamps?

For example, would the ZED 2 work with a tightly coupled visual-inertial odometry (such as ORB-SLAM3 or VINS-Fusion)?

If you disable the synchronization the IMU data will be published at a higher frequency than the image/depth data, but the timestamps will continue to be reliable.
It is improbable that you will find frames and IMU data with the exact same timestamp, in that case, the maximum association error is 1/400 sec, that it the IMU data period.

1 Like

Thanks Myzhar for your answer again. From what I understand the ZED does not provide information about its internal clock, since the timestamp comes from the host, right? This means it could happen that IMU and camera measurements do not have the exact time of retrieval since they depend on when the OS is able to process them (they are buffering by the OS before the timestamps are written down, and the OS processing time depends on how busy it is). Such delays, are not necessary negligible for some applications as SLAM. Is this correct or am i missing anything here?

What we expect to find is that the IMU data is being published at his own rate (for example 100hz) and with his hardware-obtained timestamp, while the images are at a much lower rate (e.g. 10hz) also with his own hardware-obtained timestamp. And there is a hardware mechanism that triggers both sensors at the same time when rates align. For example, there would be 99 IMU readings with his own hardware-timestamp and one imu+image reading with the same hardware-timestamp.

Getting software-overrided imu timestamps matched with the previous frame timestamp is not enough for us, neither using ros::time::now() as transmission/buffers/queues times messes with “real/hardware” timestamps.

Hi @taihu

You are not far from what internally happens.
Look at here: zed-open-capture/sensorcapture_def.hpp at master · stereolabs/zed-open-capture · GitHub
That variable is internally set by the MCU that acquires the IMU data to detect what it the nearest frame to the last received sensors values (IMU data are published @ 400 Hz)

I cannot explain you in details how this mechanism works because it’s closed information, but I want to assure you that the timestamps that you get from video frames and sensors data are precisely synchronized.

I suggest you look at the ZED Open Capture source code to understand how you can achieve the same precision.

This basically means that we get an IMU publishing rate equal to the frame rate of the camera. If I need to have the full 400hz of IMU data and yet have them in sync is it possible? I tried using the rgbd synch nodelet, however if you set apporx_sync to false, it fails. How good is approximate sync?