Hello, thank you for the great products, these cameras have helped me a lot in my work. Currently, i’m researching the phenomenon of depth inaccuracies caused by temperature on the Zed Mini and Zed 2i cameras. Typically, I hold the camera in the same position and measure depth at a single point for several hours, and the measured depth gradually decreases over time (the illustration shows that, when measuring at a real distance of 5 meters, in 3 hours, the temperature of the Zed Mini camera increased from 25 to 45 degrees Celsius, and the measured depth decreased, with an inaccuracy of over 60 cm). At shorter distances, the inaccuracy is smaller. I’ve tried simulating this behavior using a pinhole model, assuming something inside expands due to the temperature increase, increasing the distance between the lens and the sensor, thus increasing the inaccuracy and reducing the depth. However, the simulation is not yet close to reality. Could you provide me with detailed information about the camera’s internal structure (lens placement, sensor placement, etc.), as well as the materials used for the camera housing, lens mount, and circuit board? I need this information to fine-tune the simulation input data. Please also provide information about the Zed Mini and Zed 2, as I am conducting experiments with them.
Hi @phan_hieu
Thank you for the interesting analysis.
Can you please add useful information?
- ZED SDK version
- Depth mode
- camera resolution
- target type
Thank you for your feedback.
I am using:
ZED SDK version 5.1
NEURAL_PLUS depth mode
2K camera resolution
Objective type: 1 Chessboard corner point
The temperature data I obtained is from the internal IMU temperature data of the camera.
I have conducted many experiments and concluded that the depth mode and resolution affect the rate at which the camera heats up. For example, when operating at 2K resolution, the camera heats up faster than when operating at HD720 resolution. But that’s not the most important thing. What’s important is that after dozens of measurements, each time the camera heats up, the depth value obtained gradually decreases, and it follows a non-linear pattern that we need to understand. The phenomenon of depth drift due to temperature has been demonstrated in numerous papers; it can increase or decrease depending on the camera’s structure and materials. Therefore, I would be very grateful if you could provide information or images of the camera’s internal structure, as well as the materials of the circuit board, lens mount, etc. This would help us analyze how the internal expansion of the camera occurs and the extent of that expansion. From there, we would be able to simulate the depth drift pattern of ZED cameras.
Hi @phan_hieu
The ZED SDK provides an API method to trigger a “self calibration” than can allow improving depth estimation if some conditions change.
The ZED depth estimation highly rely on optics stability, so you could trigger self-calibration when the temperature change reaches a specific value.
Please read this section of the documentation about this process:
Unfortunately, this is confidential information.
Hm. Very interesting discussion. My question centers on pre-recorded data. I suspect one way is to therefore record time-synchronized temperature data to ‘trigger’ recalibration. But that would create discretized jumps in depth est.which will be highly undesirable in our case, especially since we plan on fusing data from multiple cameras . Otherwise, this is a very large warmup period even worse than IR cameras I have used. Suppose, as a ‘hack’, we monitor calibration points (via grid etc.) is it possible to continuously update just the intrinsic parameters of each camera through the SDK?
thanks,t
This is already the case if you saved data in SVO2 format.
Currently, when you trigger a recalibration, only the extrinsic parameters are tuned. The intrinsic matrices are left unchanged.