Simulated ZED X IMU data missing from SVO2 recordings in Isaac Sim

Software
1

I am working on a visual odometry (VO) and visual-inertial odometry (VIO) evaluation project using a simulated ZED X camera in NVIDIA Isaac Sim, integrated with the ZED ROS 2 Wrapper.

My goal is to record one SVO2 dataset containing both stereo images and simulated IMU measurements, and then replay the same dataset twice:

  • VO: pos_tracking_mode: GEN_1, imu_fusion: false
  • VIO: pos_tracking_mode: GEN_1, imu_fusion: true

I would like to compare both estimated trajectories against Isaac Sim ground truth under identical sensor inputs.

Environment:

  • Camera: Simulated ZED X
  • ZED SDK: 5.5.0
  • ROS 2: Jazzy
  • Simulator: NVIDIA Isaac Sim
  • Integration: ZED Isaac Sim extension + ZED ROS 2 Wrapper
  • SVO format: SVO2
  • Camera resolution: 960 × 600
  • Camera frame rate: 30 FPS

What works:

I can successfully run positional tracking in Isaac Sim with:

pos_tracking:
    pos_tracking_enabled: true
    pos_tracking_mode: 'GEN_1'
    imu_fusion: true

sensors:
    sensors_image_sync: false
    sensors_pub_rate: 100.0
    publish_imu: true

The ZED Wrapper publishes both odometry and IMU measurements.

For example, a ROS 2 bag recorded during one simulation run contains:

Duration: 6.83 seconds

/clock                    206 messages
/zed/ground_truth/pose    204 messages
/zed/zed_node/odom        183 messages
/zed/zed_node/imu/data    145 messages

Therefore, simulated IMU measurements are available through the ROS 2 interface, and live positional tracking works with IMU fusion enabled using GEN_1.

The problem:

When I record an SVO2 file from the simulated camera using the ZED ROS 2 Wrapper’s /zed/zed_node/start_svo_rec service, the resulting file contains stereo frames but no IMU measurements.

Running:

ZED_SVO_Editor -inf recordings/T_GEN1_IMU_test.svo2

produces:

[ZED][WARNING] [Init]
No inertial data was found in this SVO.
Certain modules may operate with reduced accuracy as a result.

SVO Infos: SVO v 2
Image Size: [960 x 600]
Framerate: 30
Number of Frames: 207
Camera model: ZED X
ZED SDK version: 5.5.0

I also tested recording directly through the ROS 2 service, without using my Python recording script:

ros2 service call /zed/zed_node/start_svo_rec \
  zed_msgs/srv/StartSvoRec \
  "{svo_filename: '/path/to/test.svo2', compression_mode: 5}"

The resulting SVO2 file again contained stereo images but no inertial data.

During SVO playback with IMU fusion enabled, the SDK reports that no IMU data is available in the source.

My questions:

  1. Does the ZED Isaac Sim integration currently support recording simulated IMU measurements into SVO2 files?
  2. If supported, is there any additional SDK configuration, Isaac Sim extension setting, or recording option required to include IMU data?
  3. Why can the ZED ROS 2 Wrapper publish simulated IMU measurements during live operation, while the same measurements are missing from the SVO2 recording?
  4. Is this a known limitation or issue with the simulated camera stream or ZED SDK 5.5.0?
  5. Is there a recommended workflow for recording a simulated stereo + IMU dataset once and then evaluating the native ZED SDK VO and VIO algorithms on that exact same dataset?

I have read the Stereolabs documentation explaining that SVO2 files can store IMU measurements and that positional tracking can be enabled during playback even when it was disabled during recording.

I would like to know whether this functionality is also supported when the source is a simulated ZED X camera in Isaac Sim rather than a physical ZED camera.

Thank you very much for your help!

2

Hi @mani,
Welcome to the StereoLabs community and thank you for the detailed report.

Yes. IMU data received from the Isaac Sim stream is expected to be stored in SVO2 recordings of that stream; the ZED Isaac Sim extension v5.3.1 release notes explicitly compare the IMU orientation returned live and from SVOs recorded from the stream.

No additional setting is required. On the Isaac Sim side, the camera IMU must provide readings (Rigid Body Enabled ticked and Imu_Sensor enabled), and this seems to be already the case since the wrapper publishes IMU data.

This is not a known limitation, so we must isolate where the inertial data is lost. Can you please share:

  • the Isaac Sim and ZED Isaac Sim extension versions
  • the ZED ROS 2 Wrapper version or commit
  • a short SVO2 file showing the problem

It would also help to know whether an SVO2 recorded directly with the ZED SDK from the same stream (InitParameters.set_from_stream plus enable_recording) contains inertial data; this tells us whether the issue is in the wrapper or upstream.

I also recommend upgrading to the latest ZED SDK v5.5.1 and to the extension v5.3.1. The latter fixes GEN_1 positional tracking with IMU fusion inverting the camera pitch, both in streams and in SVOs recorded from them, which would directly affect your VIO evaluation.

Your workflow is correct: record a single SVO2, then replay it twice with the ZED ROS 2 Wrapper using the svo_path launch argument, changing only pos_tracking.imu_fusion. Keep svo.svo_realtime: false (default) so that every frame is processed in both runs and the two trajectories are comparable.