ZED Mini + HTC Vive Cosmos in Unity: which tracking to use?

Hardware ZED Mini
1

Hi all!
I recently started using ZED Mini for a research project. Before that I was using SRWorks SDK with the HTC Vive Cosmos, and had no issues except the camera resolution.

To resolve this, I am now trying out the ZED Mini camera and I disabled the SRWorks in Unity. Yesterday I installed the ZED SDK and imported its plugin in my Unity project.
After I imported the ZedCamera Prefab, I started seeing the ZED Mini video on my HTC Vive Cosmos HMD. This was the first time and excellent to see it is working.

However I noticed after some time that, my virtual objects start sliding in the scene. This is a random process, sometimes I see my virtual object at the correct position, sometimes I see it flying in the HTC device view.
I believe this is because of the ZED Mini tracking system. Sometimes it seems it gets problematic.

In this case, how do you use tracking in this kind of scenes?
Do you keep using the tracking of the HMD, or do you use the tracking of the ZED Mini?

In case I would like to use the tracking method of my HMD, how do I do so with the ZED SDK?
At the end, I would like to use the tracking of my HTC Vive Cosmos, and I would like to use the all visual methods (camera images, depth etc.) from the ZED Mini device.

2

Hi,

For the tracking, we use the data from the headset and the zed camera.

You can either try to follow this documentation to see if it helps or completely disable the camera’s tracking (in the ZED Manager script, uncheck “enable tracking”) to only use the headset’s tracking data.
However, without the tracking enabled in the SDK ,you won’t be able to use our Spatial mapping or Object detection module.

Best,
Benjamin Vallon

Stereolabs Support

1 Like
3

Thanks for your reply, @BenjaminV

I am going to utilize the occlusion effects and the depth feeling of the camera as well. So I mean, whenever a virtual object is behind the real object, it should be hidden as expected.
To do so, do I need the ZED Mini tracking activated? or does it work without the tracking (only with the tracking of the HTC Vive Cosmos HMD)?

4

This feature works without the tracking enabled.

Stereolabs Support

5

This is perfect to know.
Thanks again for your help!