Point Cloud Alignment Issue in Multi-Camera Fusion with ZED2i (ZED360 + Fusion API)

[priyamgupta]

Hi,

I’m working on a multi-camera setup using two ZED2i cameras. My goal is to fuse the point cloud data from both cameras into a single, well-aligned cloud so I can calculate the volume of an object placed within the shared space.

Here’s what I’m doing:

1. Calibration:

• I used ZED360 for calibration.
• Flip mode: Auto
• Depth mode: Neural (instead of Ultra)
• Both cameras are connected locally via USB 3.0 on the same machine.

2. Fusion code:

• I’m using the body tracking fusion sample from the official ZED SDK GitHub:
  zed-sdk/body tracking/multi-camera/cpp at master · stereolabs/zed-sdk · GitHub
• Resolution: 1080p @ 30 FPS on both cameras.
• Depth mode: **NEURAL_PLUS

Problem:
The point clouds from the two cameras are not aligning correctly after fusion. The fused cloud seems disjointed or misaligned, making volume estimation unreliable.

I’ve attached an image showing the issue (see below).

---

Questions:

• Is this the correct approach for creating a unified point cloud for volume estimation?
• Could this issue be caused by:
  • Improper calibration?
  • USB bandwidth limits?
  • Using Neural instead of Ultra depth mode?
  • Camera placement or orientation?
• Are there additional steps or settings I should be using to improve the accuracy of fusion?

Any suggestions or troubleshooting tips would be appreciated!

In the above image you can see that the box made by tape should match or align but it is not.

[priyamgupta]

zed2i_zed2i_new19.json (1.5 KB)
ClientPublisher.cpp (3.3 KB)
main.cpp (8.1 KB)

these are the code files which I am using for the reference.

[BenjaminV]

Hi,

During the calibration, make sure to cover the whole area slowly.

If you still can not get an acceptable calibration, you can use this sample : zed-aruco/multi-camera at master · stereolabs/zed-aruco · GitHub to calibrate using aruco markers

[priyamgupta]

Yes, I tried this today but the calibration is still not perfect.
I’ve attached three images: two from the individual cameras and the third showing the fused point cloud result. I’m using a 6x6 ArUco marker (ID = 5) with a marker length of 200 mm.

The main issue now seems to be with the depth alignment — the floor plane is not aligning properly along the Z-axis between the two cameras. The fused point cloud shows noticeable misalignment in elevation.

Is there any way to improve the Z-axis calibration accuracy? Should I be using a marker board or multiple markers instead of just one?
Also the alignment issues are still there.

[BenjaminV]

You can try to use a bigger Aruco.
Using multiple markers can help; however, this is not supported in the current version of the code I shared. You will need to implement it yourself.