Camera stream stops in Unity project

Software
1

Hello. I am using ZED Mini camera for an Augmented Reality research project. Here is a quick summary of the setup:
Graphics Card: NVidia Quadro P4000
Displays connected to the PC: 1x Samsung 1080P + 1x Fujitsu1080P
HMD: HTC Vive Cosmos
Stereo Camera: ZED Mini
ZED Camera settings: 720p 60FPS
ZED SDK Version: v3.8.2
Unity Version: 2018.4.17f1

The problem is pretty easy to reproduce.
I start my project with SteamVR, everything seems perfect in the HMD including both the real world visual from the ZED camera and the virtual objects that I put in the Unity project.
Everything goes perfect until I click another app in the PC and get back to the Unity again. When I click the Unity window back again, the ZED Camera stream is gone. Virtual objects are still seen in the HMD, but real world video streaming is not seen from the ZED Mini.

Steps to reproduce:

  1. Start the project, everything seems okay in the Augmented Reality (both the camera stream and the Virtual Objects on top of it)
  2. Click another application in the Windows (i.e. Google Chrome window) so that the focus is changed from Unity to that application
  3. Click the Unity window again
  4. Camera stream stops here and a completely black background is seen behind the virtual objects
2

Hi,

If possible, can you try to update your Unity version to 2019? I think this issue is only happening with certain Unity versions.

Stereolabs Support

3

Actually the project is quite big and it might be extremely buggy to update the Unity version.
In fact I have no idea how to move all stuff from this version to 2019.

4

First, I would recommend you to use Unity hub (https://docs.unity3d.com/2021.1/Documentation/Manual/GettingStartedInstallingHub.html). It is very useful when using multiple projects and Unity version.

And if you plan to change the unity version used by your project, make a backup first.

Stereolabs Support

5

I am already using the Unity Hub.

Currently I am working on it to update the project to the 2021.3.24f1 LTS version.
In addition I am updating ZED SDK to v4.0.2 and the ZED Unity Plugin to v4.0.2 as well.

Let us see if I can succeed.

6

I have updated the Unity version and the ZED SDK, imported the ZED Unity as well.

After I import the ZED_Rig_Stereo and start the project, I started seeing “Unable to open the camera” error on the Game view.
What should I do in this case?

7

NEW EDIT:

I managed to have the ZED Mini camera working in the Unity 2021.3.24f1 after I installed the CUDA 12.1 as well.
However I can surely say that the same problem still continues in the Unity version too.

The visual is completely gone now.

8

I’m here searching for an answer to this. I have the CUDA 12.1 SDK 4 Windows 11. If the Unity project (Ver. 2022.3.16fi) loses focus, then I go back, the camera feed will start masking everything in black. It’s not a perfect black. If I put my arm in front, I can see it as pure black. It’s a strange thing. I’m not an expert in Unity or StereoLabs SDK, just searching for an answer. I think it has something to do with masking and depth. I think the feed stays up.

Did you ever have any luck?

9

I found something. There’s a setting in the ZED_Rig_Mono manager called, “Don’t Destroy on Load”. If the feed goes black while you are running your project from the editor, you can check “Don’t Destroy on Load” on and off and the feed will come back.

I found that I only lose the video feed when changing focus when running from the Unity editor. If running as an executable .exe, it works fine and doesn’t go dark.