Hi,
I’m facing a calibration problem with a Virtual Stereo setup made of two DUAL ZED X One cameras connected to a ZED Box. I created the virtual stereo, then attempted calibration using the ZED Calibration Tool.
Problem
After calibration, when I visualize the depth on ZED Depth Viewer, the results are very poor (unstable / inaccurate).
Looking at the calibration report, I always get a high reprojection error, even when following the recommended steps.
Here are some examples:
Best attempt so far:
Reprojection error: Left 0.927619 | Right 0.633618 | Stereo 0.875887
!! Warning !!
The reprojection error looks too high, check that the lens are clean (sharp images) and that the pattern is printed/mounted on a strong and flat surface.
Worst attempts reach above 2.0 error on Left lens.
What I already tried
Lenses are clean and image sharpness looks good.
Tried multiple calibration patterns:
9x6, square size 22 mm (A4)
9x6, square size 34 mm (A3)
10x7, square size 22 mm (A4)
Patterns printed and mounted on a flat and rigid surface.
Calibration consistently gives a correct baseline.
Questions
Is there a known issue or extra step when calibrating a Virtual Stereo ZED X One setup?
Do the ZED X One lenses require a specific pattern size or board distance range to achieve <0.5 reprojection error?
I recommend you try to calibrate the cameras using the new Calibration tool that will be launched in a few days:
It uses the new Virtual Stereo API functions of the ZED SDK v5.1 to open the cameras in a virtual stereo setup.
The new calibration tool uses different methods to guarantee that the saved images create a valid database to generate a good calibration by checking the checkerboard position, orientation, and size.
You can use the command ZED_Opencv_Calibration -h to get all the parameters.
Normally, you can start a calibration by using the simple command ZED_Opencv_Calibration --zedxone --left_id 0 --right_id 1
I recommend you use ZED_Explorer --all or ZED_Studio to by sure that the left camera has index 0 and the right camera has index 1.
For more context: I am using 6 mm lenses mounted on ZED X One cameras. I followed your recommendations, and the individual left/right calibrations now give good RMS values. However, the stereo reprojection error remains high.
I also tried manually selecting only the sharpest image pairs (excluding any blurred samples), but the stereo error is still large. My best result so far is shown below:
Detecting the target corners on the images
....................................................................................................
* Valid samples: 100/100
Left camera calibration... * Intrinsic matrix K:
[2031.387635751969, 0, 948.8327363738646;
0, 2045.76617808517, 593.2323058293558;
0, 0, 1]
* Distortion coefficients D:
[4.999972729726103, 153.8422970189366, 0.0008581095932488858, 0.0001026804842450577, -13.69553280028204, 5.286452642560306, 155.7162203611177, 28.77089901773841, 0, 0, 0, 0, 0, 0]
* Re-projection error (RMS): 0.335
Done.
Right camera calibration... * Intrinsic matrix K:
[2020.894906475655, 0, 966.9231793212911;
0, 2034.234389000558, 545.9106006214496;
0, 0, 1]
* Distortion coefficients D:
[-14.29432326059899, 85.52423561768239, 0.000604907817032998, -0.0005144790728147929, 14.53356020939793, -13.98188566787328, 81.00926969375587, 41.86336916649651, 0, 0, 0, 0, 0, 0]
* Re-projection error (RMS): 0.363
Done.
Stereo calibration... * Rotation matrix R:
[0.9998531420266131, 0.01678223823826573, 0.003471434749981282;
-0.01681504655206682, 0.9998120662810941, 0.009648125630447681;
-0.003308865207455861, -0.009705081063194019, 0.9999474300244967]
* Rotation vector Rv:
[-0.009677228119420799;
0.003390368864699583;
-0.01679972700420818]
* Translation vector T:
[-131.807933999837;
1.863674568585843;
52.43177081729498]
* Re-projection error (RMS): 8.06
Done.
*** Calibration Report ***
* Reprojection errors:
* Left: 0.335 px -> GOOD
* Right: 0.363 px -> GOOD
* Stereo: 8.06 px !!! TOO HIGH !!!
!!! ERROR !!!
The max reprojection error looks too high (> 0.5 px). Check that the lenses are clean (sharp images) and that the calibration pattern is printed/mounted on a RIGID and FLAT surface.
As you can see, both cameras individually calibrate well, but the stereo reprojection error is consistently around 8 px, and sometimes it explodes to extremely large values such as:
* Left: 0.32559 px -> GOOD
* Right: 0.360614 px -> GOOD
* Stereo: 1.96173e+08 px !!! TOO HIGH !!!
This happens even after removing blurred images.
In addition to that, I tried the updated calibration branch from this PR:
However, I obtained the same results, and I have repeated the full calibration process several times with no significant improvement.
Do you have any additional guidance on what could cause good monocular RMS but very high stereo RMS?