ZED 2 as a passthrough camera for the Quest 2 via Unity

Hardware ZED2
1

Hi!

I’m trying to get my Zed2 cameras to work as a passthrough camera for my Quest 2.
To achieve this, I followed the usual tutorials that I could find on the StereoLabs documentation page, created a 3D ver. 2020.3.21f1 blank unity project, and imported the following packages: Oculus Integration, XR Plugin Management, XR Interaction Toolkit (with Starter Assets), ZED unity 3.7.1.

Proceeded to add to a blank scene a ZED_rig_stereo (tracking and spatial memory are both enabled) object, a sphere, and adjusted the Directional Light as specified. My sample scene is made up of only these 3 objects.
Adjusted display aspect ratio to 16:9 and scale to 1x.

Unfortunately, the problem is that, when I click run, the following happens:

  • The application runs on the Quest 2, but everything is pitch black. Nothing shows up when I look or move around.
  • On the computer screen however, on both the game and scene tabs the ZED_Rig_Stereo does show the camera footage and its correct interaction with the sphere object in unity. So I believe the ZED camera works as expected.
    Processing: ZEDtest.png…

So the problem is just that all this doesn’t want to show on the Quest.
Android seems to be set up fine too, and indeed when I add an XR origin object (which evidently supersedes the ZED_Rig_Stereo object) and I run the application, I can see the blank scene space, ZED “camera screen” (a black rectangle) and the sphere in the Quest.
No errors or warnings are thrown in the console, only the following messages (repeatedly):

ZEDtest2
ZEDtest21920×1080 271 KB

Do you happen to know how I can fix this issue? How would I get the Quest and the ZED to work correctly together? I am happy to provide more info if it helps.

PS: I understand that the ZED mini is better suited for passthrough, but I don’t have one unfortunately.

Update: I have now made the project into an URP unity project rather than a 3D one and it works way better! Still seems messed up but at least now the quest doesn’t just show black, it does actually show both the camera feed and the scene objects! Unfortunately, it only shows the intended output on the Left_eye, but not the right one, where only the scene shows but not the camera feed… for whatever reason… I’ll post more updates if this ends up working.

Update2: Ok, now it shows on both eyes. To achieve so I went to Edit>Project Manager>XR Plugin Management>OpenXR and set Render Mode to Multi Pass.

2

Hi,

  • Do you manage to run basic Oculus sample (without using the ZED camera and plugin ) ?

  • Can you make sure you enabled the “Oculus link” feature in your headset ? Note that you don’t have to change the platform to Android as the Oculus link will basically emulate an Oculus Rift.

Benjamin Vallon

Stereolabs Support

3

Hi,

  • Yes I can run the basic Oculus sample without issues.
  • Yes Oculus Link is always enabled, as it’s prompted when I put on the headset and of course I accept.

I have added some updates in the main post by the way, which may be useful.

Kind regards

4

Hello @ddrago,

I’m trying to do the same on the Quest 3. As more of a beginner, I’d be very interested in getting more information on the complete Unity project setup, as I’m struggling quite a bit!
Would you be open to share it ?

Thanks a lot!
Arnaud