Zed robot integration and visualising using the camera in isaac sim

Software ROS2
1

I have integrated zedx camera into robot using the documentation provided in this link ZED integration on a Robot - Stereolabs

When i try to import the urdf into isaac sim and try to stream the data using that camera, i am only getting a top plane view from the origin of the enviroment the camera is in,. The camera view stays there even if the robot moves.
I am adding the zed_camera_link into the zed camera streamer in the camera prim while building action graph.

Screenshot from 2025-01-03 20-17-54
Screenshot from 2025-01-03 20-17-54399×220 46.4 KB

2

Hi @Anirudhgudi,

Once you’ve imported the URDF into Isaac Sim, have you included the ZED Camera extension using this guide in our documentation? Setting up ZED Camera in Isaac Sim - Stereolabs

Are you able to stream the ZED camera data to our other tools such as ZED_Explorer or ZED_Depth_Viewer?

3

Hi sorry for the delay… yes i can stream the data… the image is the same as i attached in my previous post. Once i do the integration of robot in the urdf as per documentation, the cameras get added with individual prims and its visible this way in isaac sim ( image ). which prim should i add in the zed streamer node in omnigraph?

image
image402×218 8.6 KB

4

Hi @Anirudhgudi,

Currently, we do not support importing the zed camera directly from an urdf file, the ZED X model still requires to be imported as described here: Setting up ZED Camera in Isaac Sim - Stereolabs

I would recommend placing the ZED_X USD model under the zed_back_camera_link prim, and then following the tutorial to add the streamer node.

5

Got it. Thank you for the support!

6

Hi @mattrouss ,
I configured both of the zed cameras in my isaac sim using the zed camera extension using two different action graphs. I have specified two different serial numbers in the zed camera streamer but i am not able to stream data from even one camera. As soon as hit the play sim button, i get this error “2025-01-20 09:43:55 [Error] [sl.sensor.camera.ogn.nodes.SlCameraStreamer] Failed to initialize the ZED SDK streamer with serial number 40976320.”
Any suggestions on how to configure two cameras in isaac sim?

7

Hi @mattrouss ,

I was able to rectify the error by keeping the serial number same and different sim ports in the zed cam streamer action graph.

Thank you!