Hi, we are trying to use the object tracking feature to localize objects in frame of a ZEDm camera. It works flawlessly for a while (haven’t timed it, but seems like about 30-45s), but at some point all the objects start drifting on the z axis ONLY. Interestingly, it doesn’t look like expected IMU drift, where the drift is highly biased towards a certain direction & increases over time. It’s oddly oscillatory, with the tracked objects oscillating up and down on the z axis. The camera remains able to track their x and y positions while this error is happening as well.
The fact that this happens repeatably after some arbitrary amount of time suggests that it’s some kind of accumulated error, but camera positional tracking (VIO) is off and I am viewing the targets in the camera’s own coordinate frame (left lens frame I believe). If it helps clarify my situation, I am using the ROS2 wrapper to pull and view data, but the actual calculations are done in the ZED SDK which I can’t really look through.
Hi @jonathan,
Welcome to the Stereolabs forums! 
If you are using the enable_tracking parameter in the object_detection parameters (here), you are using the camera’s VIO as it is activated in the background. This is required to differentiate between the camera’s and the objects’ movements.
If you only wish to have the objects detected and not tracked in time, I can recommend switching the od_tracking to false.
Another recommendation I can make is if your camera is static, you can set the set_as_static parameter in the pos_tracking parameters to true, this will help the object detection module to better track objects.
1 Like
We need the object to be tracked in space (x y z) but only with respect to the camera (NOT the global reference frame). For reference, the stereo camera is mounted on a drone, and thus absolute localization can be done knowing the drone’s position via RTK corrected GPS or any separate self localization method.
Would the best solution then be to pretend the camera is fixed and change the set_as_static parameter to true? I assume this parameter should negate whatever is causing the tracked objects to drift.
Thanks for the advice & quick response!
The objects in the ROS2 wrapper are indeed retrieved in the CAMERA reference frame (cf this line.
I would recommend trying the following parameters, could you confirm that you have the same behavior with these settings?:
pos_tracking:
pos_tracking_enabled: true # True to enable positional tracking from start
pos_tracking_mode: "GEN_2" # GEN 2 is our latest VIO algorithm
imu_fusion: true # enable IMU fusion
publish_tf: false # Disables odom -> base_link TF transformation
publish_map_tf: false # Disables map -> odom TF transformation as you are not using VSLAM for global localization
area_memory: false # Disables loop closure from SLAM but keeps VIO
I don’t believe using set_as_static will actually help, as a static object will be given a velocity different from 0, as the drone will be considered static.