I am facing an issue with, 3 ZedX where the clouds all appear to be quite different in terms of depth even for the same location. I would expect some differences between them due to the nature of stereo vision, but this seems more extreme than what I expect. Interestingly they all appear to be clear and fairly consistent on geometry, but scale of objects varies quite a bit. Could this be a calibration issue? Is there something I look at in the settings? This issue appears in all NEURAL depth modes. The cameras are connected to a Jetson Orin, which opens and streams them to an Ubuntu 22 box. Both computers are on SDK 5.1
Please find attached some images of the clouds and the URDF for their transforms. They are about 30 cm apart and all facing forward with some small pitch angle. They are all as close to the same alignment as I can get them.
Please let me know if this is something that can fixed or if this is the expected variance.
The red and purple clouds are from ZedX 2.2mm. The green cloud is ZedX 4mm. All are polarized lenses. The wall/doorway in the cloud is about 3m away. I will try to get a better estimate later today. These clouds were taken with NEURAL_PLUS.
So I replaced the 4mm camera with another 2.2mm camera, but I still see the same pattern. Is there something wrong in the way I have defined my transforms? They appear correct in Foxglove.
@Myzhar yes, relative to the base_link. If you notice, the three cameras in question all share the same x and z offset. And they are differing in y (all sitting next to each other). The camera_links in the image do reflect their positions relative to the base_link and each other correctly. But they report almost 1.1 meter difference in depth for the same location. I am placing them according to (x forward, y left, z up).
I noticed that there’s also a slight tilt offset between the point clouds. That could contribute.
Have you performed a custom manual calibration using the ZED Calibration Tool?
Can you verify if the problem is visible also when using ZED Depth Viewer?
I have not done a calibration. I read the documentation and it sounds like I might not be able to it as well as the factory calibration so I wanted to reach out before going down that road.
As for the depth viewer. This is also problematic. For some reason the depth viewer will not connect to the cameras. The first one 40483530 does connect but within a few seconds the stream dies and the it says camera not initialized. I have to restart the jetson to recover. The other two cameras won’t even open at all in depth viewer.
They do however all open just fine in zed explorer. I am not sure why the difference in behavior.
rzr-admin@rzr-admin:~/auto-rzr$ ZED_Depth_Viewer -r HD1200 [2025-11-12 12:32:09 UTC][ZED][INFO] Logging level INFO [Streaming] No backward compatibility required. [Streaming] Warning: Corrupted frame chunk received (recv: 196 / expect 65016) from ip : 192.168.16.175 at ts 1762968729942(ms) [2025-11-12 12:32:10 UTC][ZED][INFO] [Init] Serial Number: S/N 40483530 [2025-11-12 12:32:10 UTC][ZED][INFO] [Init] Depth mode: NEURAL [2025-11-12 12:32:19 UTC][ZED][ERROR] CAMERA NOT INITIALIZED in sl::ERROR_CODE sl::grab(sl::RuntimeParameters) [2025-11-12 12:32:27 UTC][ZED][ERROR] CAMERA NOT INITIALIZED in sl::ERROR_CODE sl::grab(sl::RuntimeParameters) [2025-11-12 12:32:36 UTC][ZED][ERROR] CAMERA NOT INITIALIZED in sl::ERROR_CODE sl::grab(sl::RuntimeParameters) [2025-11-12 12:32:44 UTC][ZED][ERROR] CAMERA NOT INITIALIZED in sl::ERROR_CODE sl::grab(sl::RuntimeParameters) [2025-11-12 12:32:52 UTC][ZED][ERROR] CAMERA NOT INITIALIZED in sl::ERROR_CODE sl::grab(sl::RuntimeParameters) ^C rzr-admin@rzr-admin:~/auto-rzr$ ZED_Depth_Viewer -r HD1200 [2025-11-12 12:33:24 UTC][ZED][INFO] Logging level INFO [Streaming] Metadata timeout. the size is equal to 196 instead of 2904. Skipping. [Streaming] Metadata timeout. the size is equal to 196 instead of 2904. Skipping. [Streaming] Error: Stream Opening returned TIMEOUT (350) [Streaming] Metadata timeout. the size is equal to -1 instead of 2904. Skipping. [2025-11-12 12:33:35 UTC][ZED][ERROR] FAILURE in sl::ERROR_CODE sl::open(sl::InitParameters) [2025-11-12 12:34:35 UTC][ZED][INFO] Logging level INFO [Streaming] Metadata timeout. the size is equal to 196 instead of 2904. Skipping. [Streaming] Metadata timeout. the size is equal to 196 instead of 2904. Skipping. [Streaming] Error: Stream Opening returned TIMEOUT (350) [2025-11-12 12:34:45 UTC][ZED][ERROR] FAILURE in sl::ERROR_CODE sl::open(sl::InitParameters) [Streaming] Metadata timeout. the size is equal to -1 instead of 2904. Skipping.
So I recorded a short SVO with each and opened those in depth viewer. These open just fine. However the difference in depth still appears to be present. I am not sure if there is a way to measure in depth viewer, but I set the max depth for each camera to 5 m and they all show slightly different results.
Depth Viewer was over a remote connection. Same for Zed Explorer, but Depth Viewer would not connect, while Zed Explorer had no problem. I recorded the SVO using the remote stream and Zed Explorer and then opened the SVO in Depth Viewer this works fine.
I ran a test again this morning to try clicking on different parts of image. I set up some cones out in front of the cameras at 5.4(m). I then recorded a short SVO for each and measured the distance to the cones. There is also a black paw print sticker on the door behind the cones. For each of the cameras in question I got:
40483530: 5.35-5.5 paw 6.32
44036609: 6.05-6.2 paw 7.13
49440015: 5.05-520 paw 5.8
So 40483530 seems to be reading pretty accurately, while the other two are over and under estimating. The images/point clouds don’t seem warped in any way though so it seems that at least from the warp perspective they are calibrated correctly. Its almost like there is a scaling factor in each direction (x,y,z). If you want I can run a more scientific test. I really don’t want to risk making it worse by doing a bad calibration, but I am not sure what options I have left. Is there a way to apply some kind of scaling factor to the point cloud?
Hi Benjamin,
A useful thing that you could do to validate the calibration of your cameras is to print a checkerboard and record SVO files while you move it in front of them.
Hey @Myzhar I will try to do this today or tomorrow. I am trying to see if I can print something larger than A4 on campus.
Is there any particular procedure/ distance from the cameras that would be most helpful? Can I leave them mounted to the robot or should they all be on a consistent mount?
You can leave them mounted, I only need you to manually move the checkerboard.
Finally, I also recommend you try to recalibrate the camera that has the bigger depth estimation error by using the ZED Calibration tool.
This procedure is not “invasive”. You can return to the factory calibration at any moment by simply deleting the custom calibration file.
