I’m testing sl::Pose::pose_confidence on a ZED X Mini using SDK 5.3 on Jetson Orin NX, and I’m seeing behavior that seems inconsistent with the documentation / expected tracking quality reporting.
Setup: ZED X Mini on a Jetson Orin NX running SDK 5.3 with positional tracking enabled.
What I’m observing in a single-camera Camera::getPosition() test:
pose.pose_confidence stays locked at 100
POSITIONAL_TRACKING_STATE remains OK
timestamps continue updating normally
However, this occurs even when the camera is fully covered and moved around in space.
At the same time, the SDK prints internal warnings such as:
[ZED][WARNING] Degraded or noisy image(s) detected on X keyframe(s) in the last tracking interval. These keyframes will be removed from the map to maintain tracking quality.
So internally, the SDK seems to detect degraded image/tracking quality, but pose.pose_confidence still remains at 100.
I’m also seeing a related behavior when using Fusion. If I call fusion.getPosition() and inspect the returned pose confidence, the confidence value does not appear to update at all and remains at 0.
So I’m seeing two different behaviors:
Camera::getPosition() returns confidence fixed at 100
Fusion::getPosition() returns confidence fixed at 0
Questions:
Is pose.pose_confidence expected to remain fixed at 100 in GEN3 positional tracking?
Is pose.pose_confidence populated/meaningful when using Fusion::getPosition()?
Is this field currently recommended for runtime tracking-quality validation or failure detection?
Is there another recommended metric/API for detecting degraded visual tracking quality in real time?
Is confidence only updated under certain tracking-state transitions or tracking modes?
Right now, it seems difficult to use pose.pose_confidence as a sanity metric, since it does not appear to reflect degraded visual conditions in the direct camera case, and does not appear to be populated in the Fusion case.
sl::Pose::pose_confidence is not yet populated when using GEN_3, you can ignore it with this version of the SDK.
This information will be available with one of the next versions of the ZED SDK.
This is expected to appear only on heavily degraded images (typically low-light, noisy frames from the ZED X in dark conditions) and should not impact tracking quality.
In the meantime, the best way to monitor tracking quality at runtime is to use either:
Thanks for the reply! I had one more question about the covariance matrix.
The covariance behavior also seems unexpected on my setup. The first three diagonal terms (presumably X/Y/Z position covariance) rapidly grow to extremely large values, reaching on the order of billions, and do not appear to recover once tracking conditions improve.
The last three terms (presumably roll/pitch/yaw orientation covariance) remain near zero, but still exhibit a similar growth trend relative to their baseline.
This behavior seems inconsistent with the actual tracking quality in some cases, since positional tracking can remain visually stable while the covariance diverges significantly. Is this expected behavior, or is there a recommended interpretation of the covariance values in SDK 5.3 for positional tracking?
This behavior is not normal.
Are you reading the covariance from Fusion or from the Positional Tracking module?
In the SDK, the covariance is computed primarily by accumulating the odometry covariance over time, except when a loop closure occurs — in that case, we recompute it from the graph.
That said, a sanity check is applied to the reported covariance to prevent it from exploding.
If you confirm that this is coming from the Positional Tracking module, could we get an SVO to reproduce and fix the issue?
The last three orientation covariance terms match between Fusion and camera PT, but the first three position covariance terms diverge only on the Fusion side.
This was from sl::Fusion::getPosition(…, WORLD) versus sl:Camera:getPosition(…, WORLD) on the tracking camera.
Does this point to a Fusion-side covariance issue? Or is this expected behaviour? I can provide the SVO for this run, but its 270 MB and the max limit is 10 MB here. Is there a method you would like me to send it to you?
Thanks for confirming the Fusion covariance bug, that explains part of what we were seeing.
While debugging this, we also started looking at the latency between the tracking camera grab() timestamp and the fused pose timestamp returned from Fusion::getPosition(…, WORLD).
We typically see ~40–70 ms latency, but periodically there are spikes to ~180–300+ ms. The odd part is that they appear somewhat periodic, roughly every ~0.6 s in some runs and ~1 s in others.
This is on a drone with a downward-facing camera flying over a reasonably feature-rich scene (concrete with markings and scattered objects). Tracking remains valid during these events, and we do not observe obvious tracking failures.
Is this type of periodic latency expected for downward-facing UAV tracking, or does Fusion perform any periodic synchronization / optimization that could delay pose updates?
Also, during these delayed periods, is any form of prediction or dead reckoning used internally? In some cases we effectively lose fresh pose updates for ~300–400 ms, which becomes problematic for our controller.