ZED 2i + OpenCV, Marker Examples Not Work in Unity

Software Unity
1

Hi,
I’m trying to create an application for tracking ARUCO markers in Unity using a ZED 2i and following the example on the StereoLabs website:

After following the guide, I click the play button in unity, ZED 2i can output video to unity,but when I put a marker in front of the ZED, the virtual cube does not appear on the marker as expected.

I tried the solution below: ARUCO Marker Examples and Unity - Software / Unity - Stereolabs Community, but it does not work. I also opened the scene “Simple Marker Placement” in ZED/Examples/OpenCV ArUco Detection/Scenes, but encountered the same problem.

I’m using the following s/w versions:
Unity 2021.3.26f1c1
ZED SDK 4.0.3 with CUDA 11.8
ZED Unity Plugin 4.0.3
OpenCV for Unity 2.5.3

Thanks for any pointers,
James

2

Hi,

Can you try the solution suggested in this post ?

It will be fixed with the next version ( coming today or tomorrow).

3

I already tried this solution,but it does not work.

4

Can you make sure the correct size and dictionary have been set in the scene? Usually, the issues are coming from that.
https://www.stereolabs.com/docs/unity/using-opencv-with-unity/#measure-your-markers

Stereolabs Support

5

Thanks for your reply.
I have set the dictionary with “DICT_4X4_50” which is set as defult, and change the marker size to 0.05m which I display on my cellphone screen, but unfortunately it does not work.
This is my setting detail as follow:

maker setting
maker setting1229×540 118 KB

6

Hi @JamesLee,

If you applied the fix mentioned above, can you try this SVO: aruco_test.svo (WeTransfer) ?
I tried it with a brand new unity project, the free trial openCV for Unity from here (enoxsoftware.com) and the scene Simple Marker Placement with all default settings, and it worked. I’m on the just out 4.0.4 version of the SDK and Unity plugin however, but there are no change concerning aruco tracking except the one already mentioned.

This is just to check that everything is set up correctly.
If it does not work, can you try creating a new project with the 4.0.4 and see if it works?

Best,
Jean-Loup

7

Also, make sure the aruco code is not on a black background, it will not be detected otherwise.

1 Like
8

Thanks for your reply.
Unfortunately, I still have the same problem after trying the SVO you provided.
Here are my steps of creating Unity project with ZED below, I’ll try to describe every detail as much as possible and maybe you can help me see where the problem is.

  1. Creat a Unity(2021.3.26) project using “High Definition RP” template.
  2. In HDRP Wizard, choose tag “HDRP+VR” in Configuration Checking, and click “Fix all” button.
  3. Import the “Zed_Unity_Plugin” unitypackage.
  4. Upgrade all the prefabs to HDRP: click on “ZED” → “Upgrade ZED Plugin to HDRP”.
  5. The compiler reported two errors here:
    ①Assets\ZED\SDK\Helpers\Scripts\MR\ZEDMixedRealityPlugin.cs(662,32): error CS0246: The type or namespace name ‘ScriptableRenderContext’ could not be found (are you missing a using directive or an assembly reference?)
    ②ArgumentException: Path: Assets/ZED/Examples/GreenScreen/Prefabs/ZED_GreenScreen.prefab, does not exist
    UnityEditor.PrefabUtility.LoadPrefabContents (System.String assetPath) (at <3febd34c7754471c9a6f2c0dc5b747bb>:0)
    EditPrefabAssetScope…ctor (System.String assetPath) (at Assets/ZED/Editor/Scripts/UpgradePluginToSRP.cs:15)
    UpgradePluginToSRP.UpgradeToHDRP () (at Assets/ZED/Editor/Scripts/UpgradePluginToSRP.cs:186)
  6. Try to solve these two errors:
    ①Go to the script “ZEDMixedRealityPlugin.cs” and add the namespace “UnityEngine.Rendering”
    ②Go to the script “UpgradePluginToSRP.cs” and delete all “greenscreen” related codes by searching the keyword “green” in VisualStudio.(I check the path “Assets/ZED/Examples/GreenScreen/Prefabs/ZED_GreenScreen.prefab”, it exists indeed, I don’t know why unity can not find this prefab.)
  7. Import the “OpenCVForUnity2.5.4TrialVersion” unitypackage.
  8. Follow the guide on the StereoLabs webside: How to Detect Markers with OpenCV and ZED in Unity – Stereolabs
  9. The same problem reappears: virtual cube does not appear on the marker.

Here are the errors I mentioned above:

errors
errors1416×501 88.3 KB

9

Hi @JamesLee,

Concerning the 2 errors, they are “normal”: the XR management plugin needs to be installed, and there is a typo in the updater script, there is a space in “Green Screen” that we added only for the URP updater when we changed it. :sweat: It will be fixed soon.

  • Instead of following the tuto, can you directly launch the Simple Marker Placement scene?
  • Does it work normally when you don’t use the HDRP/URP pipeline? Both the sample scene and the one you create with the tuto?

Best,
Jean-Loup

10

Thanks for your reply.
I directly launch the Simple Marker Placement scene after importing "Zed_Unity_Plugin” unitypackage, but the problem still exists.Then I tried to launch it without HDRP, but it didn’t work as well. :cry:
I also tried SVO input, still not work.
This is my Unity interface below, running the Simple Marker Placement scene without HDRP.

ZED-markernotwork
ZED-markernotwork1810×1148 324 KB

11

To be sure, what OS are you on?

I’ll run some more tests on my side, I have not successfully reproduced your problem yet.
Is it possible you try the setup on another computer?

12

Thanks for your reply.
I have already solved this problem by upgrading to the latest version of ZED SDK and CUDA, although I still have no idea why it not work on ZED SDK v4.0.3 with CUDA11.8.
By the way, I suggest adding a feature in the Unity plugin to check if its version matches the ZED SDK. My unity project will crash immediately after I install XR management plugin and import the “Zed_Unity_Plugin” unitypackage, just because my ZED SDK version is v4.0.5 which does not match the “Zed_Unity_Plugin v4.0.4", and this has caused me to waste some time searching for problems.