Improving 3D Reconstruction and Distance Accuracy with ZED X Camera

[MaelCap2]

Hello, I am reaching out to request your assistance regarding an issue we are experiencing with the ZED X camera, which we are currently using in a mid-range detection project.

We have observed that certain objects, although clearly visible in the 2D image, appear poorly reconstructed in the 3D point cloud — particularly in terms of shape and depth consistency.

Additionally, the distance measurements returned by the camera seem to be significantly less accurate than expected, especially beyond a few meters. For example, according to the camera documentation, we should have an error of nearly 2% at 12 meters, but we are experiencing an error of nearly 10% at 6 meters.

Despite checking various parameters (depth settings, calibration, etc.), we have not yet identified the source of the issue.
Would you have any recommendations or guidance that could help improve the 3D reconstruction quality and distance reliability in such situations?

[Myzhar]

Hi @MaelCap2
Welcome to the StereoLabs community.

@MaelCap2

We have observed that certain objects, although clearly visible in the 2D image, appear poorly reconstructed in the 3D point cloud — particularly in terms of shape and depth consistency.

What objects? Please add more details concerning what you want to achieve and what you get instead.

@MaelCap2

Additionally, the distance measurements returned by the camera seem to be significantly less accurate than expected, especially beyond a few meters. For example, according to the camera documentation, we should have an error of nearly 2% at 12 meters, but we are experiencing an error of nearly 10% at 6 meters.

What ZED SDK? What camera resolution? What depth mode?
Please add any relevant information to better understand your test setup.

@MaelCap2

Despite checking various parameters (depth settings, calibration, etc.), we have not yet identified the source of the issue.
Would you have any recommendations or guidance that could help improve the 3D reconstruction quality and distance reliability in such situations?

Recording an SVO file with ZED Explorer and sharing it can help us test your operational conditions an provide good tuned values.

[MaelCap2]

Of course, here are some additional details.

I am working on detecting a ball on a field with reduced dimensions. The field is 6 meters by 4 meters, and the ball has a diameter of approximately 15 cm. First, I perform ball detection, and then I obtain the distance to the ball from the depth map using zed.retrieve_measure(depth_zed, sl.MEASURE.DEPTH) followed by the function depth_zed.get_value(x, y)[1] on the pixel at the center of the ball. However, this is where the problem arises: the measured distances are not accurate enough (nearly 10% error at 6 meters), and the error becomes even more significant as the ball moves away from the center of the image.

In terms of parameters, I am currently using resolution=sl.RESOLUTION.HD1200 and depth_mode=sl.DEPTH_MODE.NEURAL.

[Myzhar]

@MaelCap2

I obtain the distance to the ball from the depth map using zed.retrieve_measure(depth_zed, sl.MEASURE.DEPTH) followed by the function depth_zed.get_value(x, y)[1] on the pixel at the center of the ball.

We do not recommend taking a single-pixel depth measure. This is surely affected by noise. You can obtain better results by performing median filtering around the central pixel.

Furthermore, please consider that you are not calculating an Euclidean distance, but you are retrieving the perpendicular distance of the ball from the camera sensor plane. To obtain a distance, you must calculate it from the point cloud:

stereolabs/zed-sdk/blob/master/tutorials/tutorial 3 - depth sensing/python/depth_sensing.py#L69-L73