I see several other threads on this, all with latest posts in the 2021-2023 range, so I’m hoping a new thread is a lesser sin than a deep necrobump. In brief, independent of a different question already posted of how to decouple position tracking from e.g. body/blob tracking, I’m noticing that ZED 2 position tracking itself seems to be yielding fairly egregious errors under Unity, Win10, CUDA 12.1, SDK 4.1.0. We have a single stationary camera sitting on a windowsill pointed at an open roof deck, mainly just running lighting and soak tests as we contemplate project design, and I have already watched the ZED_Rig_Mono object in the 2D tracking demo scene rotate itself a full 360 degrees on Y in 6-7 degree increments over the course of a few hours. This isn’t so bad if we are in a strictly frame-locked 2D context, but it would absolutely wreck a 3D experience, and feels so far removed from the minor jitter I might expect from accelerometer vibration that I’m wondering if something else might be up. The ZEDManager script itself is a black box, so it’s hard to get a clear picture of where error originates and propagates.
Hi @NNSkelly,
One option is to set the camera as static in Unity in the ZEDManager, which should prevent drifting issues.
That being said, to check if the IMU is at fault, you can use ZED Sensors Manager (in the tools folder of the SDK), put your camera on a stable surface, and check the position and rotation. If you see unexpected movement, you can reset the calibration of the camera, as described in this forum post.
Thanks for the reply. I’m not seeing ZED Sensors Manager in my tools folder. Per the linked thread, it looks like Zed Calibration is the correct tool - "ZED Calibration.exe" --cimu
Ran the recalibration, relaunched the Unity object tracking 2D demo; within 3 minutes, the camera Y rotation had jumped by 25 degrees. Going to try calibration one more time on an even more stable surface that’s even more level, but this might have made it worse…
Update: yep. Recalibrated a second time, on concrete, with the tripod removed. Now the Y rotation jumps 5-7 degrees every 10 seconds rather than every 15-30 minutes. The second run of the tool indicates that the first run set the gyro bias to 0 0 0. Is there a syntax to feed the old bias from before the first recalibration back in manually? It’s still quoted in the console.
I have since learned that the sensor we picked up might be an older model without an IMU, which would explain why the autocalibrate returned all 0s. So is there any way to do a manual setting of the gyro bias to specific values? The -h text on the calibration tool doesn’t list anything promising, or at least not with enough detail to use.
@NNSkelly
There is no way to set a fixed bias.
To be sure about your camera model, you can check what appears under Status → Camera Model in ZED Manager (at the bottom of the component) when the scene is running and the camera is open.
I’m not seeing ZED Sensors Manager in my tools folder
I gave the wrong tool name, sorry, I was thinking of ZED Sensor Viewer, not ZED Calibration. Please record or take a screenshot of the gyroscope and orientation data.
These are the results of the two runs of the calibration tool:
And this is from the first minute or so of a run of the 2D Object Detection demo:
The physical camera is static, sitting on the pack-in tripod.
Sensor data:
Sensor Viewer Accelerometer:
Gyro:
Orientation:
I’m not seeing the drift in the Sensor Viewer. Is it possible that e.g. lag spikes in Unity are causing improper accumulation of gyro values?
@NNSkelly
It’s possible the odometry is having issues then.
To investigate further, I will need an SVO recording reproducing the issue. You can record directly from Unity via a button in the ZED Manager once the scene is running.
Please send it either here or to support@stereolabs.com, through your hosting platform of choice and mentioning this post, I’ll find it.
Please also include a ZED Diagnostic file.
I’ll take a look as soon as possible.
Thank you for the files, and sorry for the delay. I showed it to the team, and it’s possibly an IMU edge-case whose handling is fixed in the gen 2 of ou Positional Tracking.
Can you set the “Positional Tracking Mode” to GEN 2 and test again? It improved the results a lot on your SVO on my side.
Thank you. I’ve been out for a week, and I think the department has already sent back the older-gen ZED 2 sensor anticipating receiving the current model. I’ll need to ask around for where that ended up and see what I can repro, but will keep an eye on the tracking mode if I see similar symptoms.
So we didn’t get a new sensor after all because it was too expensive relative to where the anticipated use-case project currently stands, but by the same token, we couldn’t return the old sensor because the return shipping was too expensive relative to the project. So I can confirm that GEN_2 is at least far more stable on the old sensor, even if I have yet to throw GEN_1 into an outright tailspin since getting it all set back up.
Alright… even with GEN_2, I just observed something really weird. Currently trying to reproduce it in-editor so I can grab another SVO, but it may be a unicorn. In a built single-scene app basically consisting of the 2D Object Detection demo with additional stuff bolted on, running in Objects mode with Floor as Origin but not Static, the background feed began wobbling out of control. A few users walking through the scene registered with different IDs every single frame, almost as if the entire stack was resetting constantly. I only managed to capture video of the wobble, not the detection bleed, but since the object-detection-on-camera-feed demo seemed entirely visually stable before, I was wondering if there are known conditions under which it… re-evaluates the intrinsics and re-applies the camera frustum??
wobble video: -
@NNSkelly
The image is indeed wobbly.
It doesn’t happen when you set the camera as static, right?
I don’t know why the IMU would change the image, but it looks more and more like it is the issue here.
If you can capture an SVO, it will help identify the issue, in the meantime, you can set it as static if it prevents the issue at all.
Can’t edit the old post, but I pulled the original video down to splat over some identifying readout. Reposted for posterity: wobbly_1.mp4 - Google Drive
Various runs in-editor in the last 48 hours haven’t shown the wobble; going back to a build to soak-run over the weekend.
I have the same drift issues.
I installed latest 4.1.2 sdk on Jetson ORIN with Zed X and Zed X Mini.
I opened the example sw from the positional Location.
I changed it to NEURAL and GEN2.
everytime I put the camera looking to outside and then moving it a little bit around for some seconds and turn it again on the same spot back. after this it always starts to drift away, sometimes with 1meter a second.
looking to the orientation in Zed sensor viewer there is no drift is visible.
so I suspect the visual implentation seems trigger this drift.
other test with mounting it on a tractor and driving for 1,5minute in a round and coming back to the start location, the Height of the machine has lowered by 1 meter!