Issues with ZEDX on Isaac SIM: glitches and wrong depth

Software
1

Hi, I encountered two issues while using the ZEDX camera extension (V1.1.0) using Isaac SIM 2023.1.1.
Both the issues have been noticed both using ZED_Depth_viewer and using zed-ros2-wrapper.

The first issue is encountered when the resolution of the camera is set to something different than HD1080 from the ZEDX OmniGraph Node. If I set the resolution to something different the depth measures are incorrect. (e.g. 1.5m instead of 1.1m when using HD720, 3m if using SVGA)

The second issue is encountered if the ZEDX is not moved for some time. Some artifacts begin to appear on both the image and the pointcloud and after some time the depth quality also decreases.
The pointcloud as it appear at the start of the simulation.

PointcloudBefore
PointcloudBefore694×568 26.1 KB

The Pointcloud after three minutes where the ZEDX is not moving.

PointCloudAfter
PointCloudAfter694×568 39.7 KB

The camera has the same artifacts.
Image

2

Hi @Alessio-Parmeggiani,

Regarding your two points:

  • Thank you for reporting this, our team will take a look if we are able to reproduce. From the look of it, this seems to be a calibration issue.
  • The second issue of artifacts in the image seems to be a limit of the Isaac Sim camera in low light conditions. Can you confirm that these issues are not present in a well-lit environment? If you can confirm that this comes from the Isaac Sim Camera sensors, I would suggest getting in touch with the Isaac Sim team to see if this is known on their end
1 Like
3

Thanks for the answer.

Regarding the second issue I don’t think the problem come from Isaac Sim, the artifact do not appear in the Isaac Sim camera viewport and they reset if a change depth type from ZED_Depth_viewer.

I can also confirm that they appear also in a well-lit environment.

This is the isaac sim viewport (camera set to the left lens of the zedx)

IsaacSim viewport
IsaacSim viewport832×439 46.3 KB

This is the image from ZED_Depth_viewer (after around two minutes)
ZED depth viewer image

Some more informations:

  • I am using ZED SDK 4.1.1
  • The artifacts also appear when looking at only the images with ZED_Explorer.
  • The artifacts starts to appear very gradually only when I start some ZED software, it does not matter when I start it.
  • By publishing on ROS2 an image from the same lens of the ZEDX the artifacts do not appear on the image but they appear on the image from the zed ros wrapper node.
  • The artifacts only appear when the ZEDX does not move for some time
4

Hi @Alessio-Parmeggiani,

Thank you for the detailed information, we will look into this as this is definitely not intended behavior and seems to be an SDK streaming issue.

1 Like