Getting depth image from stereo images

Software
,
1

Hi all,

I recorded my dataset with stereo images from a ZED 2i camera (not a wise idea but unfortunately the data recording has already finished). Is there anyway that I can use ZED SDK to recreate the depth image? I can use OpenCV for this but I assume the depth quality from the SDK would be better.

Thanks in advance

2

The ZED 2i is a stereo camera, which means it uses both lenses to estimate depth by comparing the left and right images—much like human vision.The ZED SDK computes depth by finding disparities between corresponding points in the left and right images after stereo rectification.

  • 2 images (two lens): If your recording includes both the left and right images for each frame you can absolutely use the ZED SDK to recreate the depth map, and the sdk is better optimised than cv2.
  • Only single images (one lens): No, you cannot use the ZED SDK for stereo depth; you’d need to use monocular AI models, which are less accurate.**
3

Hi Adrian, thank you so much for replying. I have the rectified left and right images recorded. I initially thought this would be doable with ZED SDK as well, however, after looking at the tutorials, I didn’t see how can I do this. Could you please point some directions for me? Anything information is appreciated!

4

Hi,
The ZED SDK can’t input rectified images (like PNG files). It requires SVO files recorded from the ZED SDK.
The SVO files are proprietary and optimized for the ZED SDK. They contain mainly raw side-by-side images, just like the camera USB/GMSL stream. They can’t be recreated from images.

5

Hi,
Thanks for your reply. If ZED SDK is not an option, would you recommend any method or packages to recreate the depth images then? Thanks in advance!

6

You can use OpenCV, or more modern AI methods like FoundationStereo or IGEV, RAFT etc
This will generate the disparity map, then you’ll need to use your factory camera calibration parameters to get the depth and point cloud data

7

I just tried FoundationStereo and it worked quite well (honestly the depth images seem even better than with ZED SDK). Thank you so much for your help!

8

FoundationStereo is indeed very impressive. However, I must highlight that it is a foundation model and is not designed to be deployed in a production environment. It’s also several orders of magnitude slower than our SDK on Jetson.