We are using the ZED 2i for our autonomous vehicles, but we are getting some strange results in our depth maps.
I took some screenshots of the ZED Depth Viewer (version 4.1.2) playing this SVO recording: recording.svo (3.5 MB)
Camera image:
Corresponding depth map with ‘Neural’ depth mode:
We have mostly been using the Neural depth mode, as it seemed to give us the best results in most cases. However, as shown in this example, it sometimes detects far away reflections as being really close by.
I recently tried the new ‘Neural+’ depth mode, where the results seem to be a lot better but the problem is not gone completely:
For comparison, I also tried the ‘Ultra’ depth mode. It didn’t have the same problem with the reflections, but it did detect parts of the sky a lot closer than they really are:
We would like to use the depth information for obstacle detection, but our current depth results can cause our vehicle to detect obstacles that aren’t really there. What can we do to improve these results?
We’re using the ROS 2 wrapper, where the ‘remove_saturated_areas’ parameter was already enabled:
The screenshots I sent last week were taken from the ZED Depth Viewer, where this setting was enabled:
I’m assuming this is the same setting? It was already turned on by default, and for this recording it does not seem to make a lot of difference whether it is switched on or off.
Am I doing something wrong? Or is there anything else we can try?
For the neural depth mode lowering the confidence threshold does not help, the depth map has a lot more missing values and the problem is still there:
(confidence threshold set to 10%)
However, for the neural+ depth mode it does help, decreasing the problem a lot while only causing a few more missing values:
(confidence threshold set to 10%)
Lowering the confidence threshold even further makes the problem go away for this frame but it’s still there for some other frames:
(confidence threshold set to 1%)
(confidence threshold set to 1%)
While we still can’t get rid of the problem completely this is probably good enough for us. For now we’ll use the neural+ depth mode and experiment with the confidence thresholds in different scenarios.
Can you please record an SVO with ZED Explorer and share it with us (you can send an email to support@stereolabs.com) so we can use it to improve our NEURAL Depth AI networks?
