Camera intrinsics and rectification for Zed X

Hardware
1

We are trying to perform hand eye calibration with a mounted Zed X and are getting small errors which we suspect may have to do with our use of the intrinsic calibration. To that end, a number of clarifying questions:

  1. When taking rectified RGB images with the Zed X, the intrinsics returned by get_camera_information().camera_configuration.calibration_parameters.left_cam have zero distortion parameters, but the focal lengths and center points change between camera invocations. We are using these intrinsic parameters “as is”, and trying to calibrate the extrinsics based on these numbers. If the rectification is performed based on some factory calibration, why aren’t these constant? Do the changes reflect an error in calibration or simply a choice made on what parameters to rectify the image with?
  2. Is some sort of RGB image stabilization being performed? there’s mention of depth stabilization, but not of RGB.
  3. If we use unrectified images, how do we get the factory calibration paramaters? it seems that get_camera_information().camera_configuration.calibration_parameters.left_cam again contains zero distortion, which can’t be right.
2

When the ZED SDK opens the camera, it performs a quick camera parameters tuning to correct little modifications due to temperature or vibrations over time.
You can disable this behavior to always have the same parameters by setting camera_disable_self_calib to false in sl::InitParameters:

No, there’s no filtering on RGB images.

This command

sl::CalibrationParameters calibration_params = zed.getCameraInformation().camera_configuration.calibration_parameters;

returns the rectified parameters.

This command

sl::CalibrationParameters calibration_params_raw = zed.getCameraInformation().camera_configuration.calibration_parameters_raw;

returns the raw image parameters.

Read more here:
https://support.stereolabs.com/hc/en-us/articles/27824782212119-Why-is-there-a-difference-in-camera-parameters-between-the-ZED-SDK-API-and-the-ZED-Explorer-tool

3

So which is more accurate? using the unrectified images with the factory calibrated intrinsics, or the rectified images with the “tuning” done when camera_disable_self_calib is false? In other words, do we expect the tuning to increase accuracy?

4

Of course, that’s the purpose of this feature. Otherwise, we wouldn’t set it to enabled by default. :slight_smile:

5

The unrectified intrinsics do not seem to correspond to reality - on the Zed X Mini, this:

camera_info=self.zed.get_camera_information()
calib_params = camera_info.camera_configuration.calibration_parameters_raw

Returns for fx, cx, fy, cy & distortion parameters:

  • 737.052978515625
  • 978.52197265625
  • 736.5230102539062
  • 643.114013671875
  • 1.1389000415802002
  • 1.6283799409866333
  • 0.0002682750055100769
  • 2.426269929856062e-05
  • 0.17436400055885315

The first two distortion parameters (k1 and k2) are huge, and of course to not correspond well to the unrectified image from the Zed X Mini. Am I missing something?

6

What do you mean? Unrectified images have distortion values all set to 0.
The distortion parameters above are used internally by the ZED SDK to generate the rectified images.

7

That is, if I capture an unrectified image, I should be able to use the calibration_parameters_raw parameters to rectify it. However, these parameters seem too large and indeed do not give correct rectification - the Zed X mini unrectified images aren’t that distorted.

8

Please print a checkerboard and point the camera at it, then save one rectified and one unrectified image to compare them.