Hello,
We just acquired a ZED mini to try to implement mixed reality for a research experiment.
We use Unity (2019.2.9f1) with an htc Vive pro. For this experiment we need accurate overlapping of real and virtual objects.
To make it work we have just put the zed_rig_stereo object inside the scene and we have forced through code the Camera_eyes object transform to be identical to the htc vive headset transform.
It’s working but it’s not enoughly accurate for what we are trying to do. When we try to overlap a real object (htc vive tracker) with the virtual equivalent it is off of about a few centimeters. Offset that varies depending if the object is near or far, and if it is on the sides or the center of the zed mini field of view.
To try to reduce these effects we looked at how was implemented the offset between the zed mini reference frame and the htc vive pro reference frame.
We found this github issue : ZEDMixedRealityPlugin zed2eye_distance does not match hmdtozedCalibration · Issue #43 · stereolabs/zed-unity · GitHub
Based on that we tried to modify the hmdtozedCalibration.translation values. By testing we found some values that made the overlapping much more accurate at close range with tracker in the center of the field of view.
If we understand correctly the hmdtozedCalibration.translation values correspond to the offset between the center of the hmd (the htc vive pro headset reference frame) and the two cameras reference frame. Our problem to try to put these values right is that we don’t have the zed mini reference frames origin, we know that the two cameras are spaced of 0.063 meters, and we don’t care about the vertical component because it’s placed in front of the headset at th center but what we are missing is the depth / z component. At which distance are the origins of the reference frames of the camera measured from the back of the ZED mini ?
We would also be grateful if you have any advices of how to diminish the variations depending of whether the object is near or far, or on the sides or the center of the zed mini field of view. Or whether there is nothing to do about it.
Thank you!