ZED 2i Calibration and Skeleton Tracking

Hardware ZED2
1

Hi StereoLabs team and community,

I am working with multiple ZED 2i cameras for skeleton tracking (tracking distances between 2–10 m, testing one camera at a time under normal lighting conditions but for calibration I used a dark environment as advised). After evaluating five different units using the same procedure, I have observed some unexpected behaviour in both calibration and distance measurements. I would appreciate clarification before moving to the next stage of my experiments.

What I’ve observed

· As StereoLabs recommends, some cameras performed better with factory calibration (FC) compared to user calibration.

· However, for other units, user calibration sometimes resulted in better performance than FC.

These inconsistencies raised several questions:

My questions

1. Under what conditions does StereoLabs recommend performing user calibration instead of relying on factory calibration?

Are there specific indicators, symptoms, or accuracy thresholds that suggest a recalibration is necessary?

2. Can user calibration degrade performance if environmental conditions are suboptimal (e.g., poor lighting, monitor glare, low screen resolution, reflections)?

In other words, is the calibration process sensitive enough that suboptimal conditions may produce intrinsics worse than the factory defaults?

3. Does the ZED Calibration Tool provide a quality score, confidence measure, or any metric to help determine whether a calibration was successful or should be repeated?

4. Do calibration parameters drift over time, or should the factory calibration remain stable unless the camera experiences mechanical shock, temperature extremes, or physical damage?

5. What is the recommended robust procedure for user calibration?

The process appears susceptible to user-induced errors—particularly while moving the camera in front of the screen.

Is it preferable to minimize camera movement by mounting the camera during calibration?

Or is it acceptable to hand‑hold the camera and follow the on‑screen markers?

6. Do all calibration parameters (intrinsics, distortion coefficients, stereo baseline, etc.) contribute equally to the final calibration quality?

Or are certain parameters more critical for depth accuracy and downstream tasks such as skeleton tracking?

7. Across all five cameras tested, I observed systematic depth‑measurement biases: significant overestimation (6–14%) at approximately 2 m, improved accuracy between 4–8 m (typically <3% error), and increasing underestimation beyond 8 m. Could you explain the factors that may cause these distance‑dependent variations in measurement accuracy?

Any guidance or best practices would be greatly appreciated.

Thank you!

2

Hi @sachithwelarathna
Welcome to the Stereolabs community.

Normally, we recommend performing a manual calibration only if required:

  • Using the camera in non-standard conditions (underwater, with modified optics, behind glasses, etc)
  • The camera received a hard shock that could have invalidated the factory calibration

If you notice that the depth estimation is not as good as expected, then you can try to improve it by manual calibration.

Yes, you must perform the calibration always in optimal conditions.

No, it does not. You can use the ZED OpenCV Calibration tool that provides score metrics at the end of the calibration process:

Adverse environmental conditions can invalidate the factory calibration. Very low or very high temperatures and shocks can damage or slightly move the optics, causing a modification of the intrinsic values.

We recommend using the validated process described here:

Yes, they all affect the quality of the calibration. There are no parameters that are more important than others.

Please provide more information regarding:

  • the test setup
  • the camera configuration
  • the depth mode
  • etc

Please find detailed information regarding the expected accuracy in this section of the online documentation:

3

Thank you for the detailed and useful response. I would also like to know the following: I am particularly interested in using the ZED 2i camera in two different resolutions (HD1080 and 2K). My understanding is that the camera needs to be calibrated at the specific resolution in order to obtain optimal results.

I would like to ask whether it is acceptable to first calibrate the camera at the 2K resolution and then scale the calibration for other resolutions?

4

This is not true. Al the ZED stereo cameras are factory calibrated using a precise automated process. You do not need to manually calibrate them, unless you use them in a modified optical condition (behind non-homogeneous glasses, underwater, etc).

5

Thank you for the response. I am using the OpenCV method for camera calibration. I use HD1080 and 2K modes for my experiments most of the time. But I want to clarify: if I use the OpenCV method as you mentioned in your previous response, do I need to perform calibration separately for the HD1080 mode and the 2K mode (not the factory calibration)?

6

No, you can use the maximum resolution. The ZED SDK will automatically calculate the correct values for the lower resolutions.

7

Small clarification: I used the OpenCV method to calibrate the ZED 2i at 2K resolution and generated a calibration file in .yml format, which contains the intrinsic, distortion, and stereo parameters (except Baseline) for that specific resolution. Then, I replaced the corresponding values in the factory calibration to use it as a .conf file for the camera. Do you mean that the ZED SDK will automatically compute the correct calibration values for the lower resolutions during use?

8

Yes, that’s the behavior of the ZED SDK.

You are not required to perform this. You can pass the YAML file directly to the ZED SDK using the parameter optional_opencv_calibration_file of InitParameters.

9

Thank you for the clarification.

  1. Do you have any suggestions regarding the checkerboard and setup for the OpenCV calibration method?

  2. I tried using checkerboards with different square sizes in my experiments (25.4 mm, 48 mm, and 72 mm). I placed the camera (I am using multiple cameras for my experiments) at a fixed location with appropriate lighting and then moved the checkerboard to different positions for sample collection. However, most of the time the reprojection error exceeded 0.5 (stereo error was more prominent), and the calibration was not successful, so the .yml file was not generated (it only worked successfully a few times), even though I achieved 100% on all four metrics (X coverage, Y coverage, skew, and size).

  3. Do you also have any data or an opinion on the stability of the ZED 2i camera once it has been calibrated (i.e., stability of depth measurements over time)?