Hello. I have an Augmented Reality research project including ZED Mini and HTC Vive Cosmos, but I have a problem such as placing the virtual objects correctly and avoiding any flickering in occlusion effects. First of all, here is my setup:
Graphics Card: NVidia Quadro P4000
HMD: HTC Vive Cosmos
Stereo Camera: ZED Mini
ZED Camera settings: 720p 60FPS
ZED SDK and Unity Plugin Versions: v4.0.2
Unity Version: 2021.3.24f1
I have virtual objects placed in Unity, and when I run it, the subject sees both the virtual objects and the real world video coming from the ZED Mini, these all are shown in the VR device (HTC Vive Cosmos).
The problem is some flickering, while the software tries to block a virtual object when it gets behind with a real world object (depth sensing). At some parts it is pretty problematic, especially when the object distance is higher than 1.5 meter so to say.
The other issue is that I could use the positional tracking of the ZED Mini camera, as I can not locate the virtual objects at the correct position in the real world. They either look drifted, or go at another location which I do not want them to. When I deactivate the “Enable Tracking” settings in the ZED_Rig_Stereo game object, it uses the HTC Vive Cosmos as the tracking device, and VR device knows the room pretty well. In this case, virtual objects placement wrt the real world is pretty accurate; but now when the subject moves his head fast, virtual objects respond lately so there is no spatial relationship in between virtual and real world. This decreases the “real” feeling.
Do you have any idea or any advise how to resolve these? Maybe I am using the software wrong. But for now, the best use case is just deactivating the “Enable Tracking” method of the ZED Mini, as the virtual objects get placed at correct locations once the Unity project runs.