Hi,
I’m trying to create an application for tracking ARUCO markers in Unity using a ZED Mini and following the example on the StereoLabs website:
https://www.stereolabs.com/blog/how-to-detect-markers-with-opencv-in-zed-unity-plugin/
However, following the guidance, it appears there are some incompatibilities between the ‘OpenCV For Unity’ package and the ZED unity package such that they can’t co-exist in the same Unity project (as suggested) within loads of errors. The errors are primarily to do with the Aruco class and the absence of marker dictionary entries as follows:
Assets\ZED\Examples\OpenCV ARuco Detection\Scripts\Core\ZEDArUcoDetectionManager.cs(236,29): error CS0117: ‘Aruco’ does not contain a definition for ‘DICT_5X5_50’
In fact, there are loads of these. I’ve tried being selective about which parts of the packages I import but this makes no difference. Anyone got any ideas?
We are using the following s/w versions:
Unity Editor 2021.3.27f1
ZED Unity Plugin v3.7.1
OpenCVForUnity 2.5.4 (trial version)
Thanks for any pointers,
Dave
OK, so it looks like the ZED SDK version makes a big difference. Updating to 4.0.3 reveals a single error in VideoWriterAsyncExample.cs line 236. I’ve commented this line out in the short term and now the demo runs - but doesn’t track anything! - the investigation continues…
Hi,
Indeed the compatibility to the latest version of Opencv has been added recently, you need to use the v4.0.x of the ZED SDK (and Unity), or copy paste the script into your project if you absolutely need to use an older version.
The VideoWriterAsyncExample.cs comes from the OpenCV plugin, I think you can safely comment it if you are not using it.
1 Like
Thanks. Have updated to 4.0.3. However, whilst the example runs, it doesn’t actually track any markers! Has anyone got this example to work correctly recently?
I note that the example in the 4.0.3 is using depreciated OpenCV functions on occasions (like Aruco.EstimatePoseSingleMarkers() in ZEDArUcoDetectionManager.cs). This might not be a show stopper but perhaps needs updating. In the meantime, I’m after any fix to be the basic claimed functionality working.
Thanks for any pointers,
Dave
I tested it quite recently and I was working as expected.
Make sure to follow this step by step documentation : Using OpenCV in Unity | Stereolabs
1 Like
Thanks Benjamin,
I’ve recreated the Unity project from scratch again and followed the instructions with similar results.
Having put in some Debug into ZEDArUcoDetectionManager.cs, I can see that aurcoDetector.detectMarkers() always returns 0 results, even with a marker ‘0’ (4x4) in clear view of both ZED Mini cameras (although I understand it uses the left camera only).
Is there any way to check the imagery being passed to the OpenCV functions to ensure that it is getting a valid input?
Also, I wonder whether there is an issue with the trial version of the OpenCV For Unity library? I had thought the only restriction was the inability to build/deploy an executable.
We are using the following s/w versions:
Unity Editor 2021.3.27f1
ZED Unity Plugin v4.0.3
OpenCVForUnity 2.5.4 (trial version)
Thanks for any pointers,
Dave
Similar non-working situation with Unity Editor 2022.2.2f1.
OK, I’ve taken a closer look at this and there’s a bug in the ZEDToOpenCVRetriever.cs file. This could never work the way it is.
On line 217 the current line reads:
MatUtils.copyFromMat(buffermat, zedmat.GetPtr());
This should be replaced by the following line:
MatUtils.copyToMat(zedmat.GetPtr(), buffermat);
There’s a commented out line above this in the code, so it looks like the function call was changed but then nobody actually checked to see if it worked! The code as it is, is trying to move the data contents of buffermat to zedmat but in fact it should be the other way around. Therefore, the video image is not transferred and the OpenCV calls don’t detect any markers as a result.
Regards,
Dave
Hi Dave,
Thanks for the catch, you are completely right.
I apologize for that.
I’ll update the plugin with the fix soon.
Best regards.
1 Like