Hi All,
I’d like to use the ZED Mini as a tracker together with unity to render a stereo output for presentation on a see-through display. I have a couple of questions on this topic that hopefully someone might be able to comment on - thanks!:
1). Firstly, is there a way within the ZED SDK to compensate for the fact that the ZED Mini and the display are positionality and angularly offset? ie. the ZED Mini might be located a few cm above the eye location and therefore I would need to provide some form of compensation. I would have thought I could do that with some matrix maths but wondered whether the SDK had any ability of this type.
2). My second query concerns whether to use the Stereo or Mono rig. If not using the ZED mini camera video at all (other than for tracking purposes), do I need a stereo rig? Can I just take a mono rig and attach two cameras to it?
3). Finally, when the documentation talks about ‘AR Mode’ and Pass-thru mode, what does that mean?
I don’t fully understand the setup you’re trying to achieve. Is it AR in a HMD, or a real see-through display connected in some way to the ZED Mini movements?
In any case:
You will have to go through some matrix maths, the SDK & the Unity plugin can only provide the location of the camera either relative to the world or to the previous frame. It can however use the tracking of a HMD instead of the ZED camera, but that does not seem what you want here.
The only difference between stereo and mono rig is that stereo will compute the views from both camera sensors so they can be displayed in a headset, using Unity’s XR features. It’s necessary when doing AR in a VR headset, but that’s all. If you don’t use the view from the ZED, you can do without it. In the background, it creates some tools for the XR to work correctly, you can check out around this line of the script for more technical details.
These refer to one, if not the ZED Mini main use-case that is adding video passthrough with depth to a VR headset.