I currently am using the ROS2 integration with a Jetson Orin AGX and a Zed2 camera, and a separate integration with a Meta Quest 3, using running a unity app with a ROS2 integration, allowing me to transmit the controller pose and button presses to the Jetson.
I now want to transfer the feed from the zed camera to the quest 3, such that the user visualizes the scene from the perspective of the camera. I have seen the Unity vr integration with zed tutorial, but this seems to involve using pcvr. AFAIK, I cannot run unity on the jetson, due to it being ARM architecture. I have access to the camera feed, (albeit in a high latency fashion) via ros topic. I can display the image data in the unity environment by subscribing to the topic and setting the texture of a plane to the image, but ideally I want to use the depth information and get a similar vr experience as the unity vr integration demo.
Has anyone had a similar setup, and/or what would be a good way to proceed?
To have the best VR integration, you want to use our Unity plugin but that means the app needs to be installed on a computer, not directly in the headset.
Indeed, our plugin requires the ZED SDK to run, which can’t be installed on the headset itself.
Yeah, the plugin can’t be used directly. I was wondering if there was any guidance on dveloping a standalone unity vr app for the zed. I found a similar forum post Stream Stereo Images Only to Standalone VR device built with Unity 3D which seem to refer to a couple unity scripts. If no one has succeeded at this yet it’s fine, I just wanted to know if there’s a better starting point or direction to take.
On a somewhat related note, how is the binocular disparity handled with the Zed2 vs the Zedm? The zedm has lenses approximately the human eye’s distance apart, but the zed2 is about twice as far apart. In the 2nd link you send, it says
Note : A ZED mini is required to fully enjoy this tutorial, the usage of a ZED will introduce visual discomfort due to wide baseline.
Was this problem somehow fixed/alleviated in the newer VR integration?