Hi there,
I am using ZED 2i, and I have mounted ZED Cam on my robot head to view inside the simulated environment of ZED Tools. I was wondering, can I see the real view while I am doing robot simulation (robot head movement but instead of ZED Tools showing the inside of the simulation environment, it can show the real view of my desk).
Additionally, how can I utilize this view (fetching data) to be used by the LLM or further Computer Vision applications?
Any leads will be appreciated.
Hi @iamshakeelsindhu,
Welcome to the Stereolabs forums!
From what I understand, you wish to replicate the real world in simulation inside Isaac Sim, creating a “digital twin” of the world. This is unfortunately not available in our integration of Isaac Sim at the moment, where we have focused more on simulating the ZED camera to be able to create any type of scenario and still test the ZED SDK and its algorithms.
We currently offer the integration you mention in Unity and Unreal Engine, you can find more information on this in our documentation: Key Concepts & Scripts - Stereolabs
To use the data streamed from the simulated camera in Isaac Sim, you can use the camera streaming module: Local Video Streaming - Stereolabs
Once you specify the IP and port, you can use the camera the same way as a physical camera with the SDK.
1 Like