ZED CustomDetection.cs not working with openCVforUnity plugin

Software Unity
1

OS: Win 10
ZED SDK: 4.0.8
Unity zed plugin:4.0.7
OpenCVforUnity:2.5.9
cuda: 12.1

I try the sample of the ZED Custom Object Detection scene. But the unity can’t load the ZEDcustomobjdetection.cs in the 3D object visualizer.
The define ‘ZED_OPENCV_FOR_UNITY’ not working, even activated the define code in the OpenCVpackageDetector.cs.

Someone delete the define code ’ #if ZED_OPENCV_FOR_UNITY’, so I try the way.
But the code still miss something, print error code like ‘Empty path name is not legal’.

Empty path name is not legal.
UnityEngine.Debug:LogError (object)
ZEDCustomObjDetection:readClassNames (string) (at Assets/ZED/Examples/Object Detection/Scripts/ZEDCustomObjDetection.cs:333)
ZEDCustomObjDetection:OnValidate () (at Assets/ZED/Examples/Object Detection/Scripts/ZEDCustomObjDetection.cs:102)

2

We just ignore the define code, and manually write the paths(model, config, classes). After that, show me no error and not working 3D visualization, just streaming screen of the zed camera. In this point, maybe you check the paths are wrong. Because the openCVForUnity plugin change files position in installation processing . The developers of the code, need to re-check. The scene is not working.

[Modified Paths in ZEDCustomObjDetection.cs]

model: Assets\StreamingAssets\OpenCVForUnity\dnn\yolov4-tiny.weights
config: Assets\StreamingAssets\OpenCVForUnity\dnn\yolov4-tiny.cfg
classes: Assets\StreamingAssets\OpenCVForUnity\dnn\coco.names

3

Hi @kdsss,

Thanks for the report, and the correct paths! :slight_smile:
I think moving the files is not mandatory in the OpenCV plugin, it’s a click of a button if I’m not mistaken. It’s probably why it didn’t make the cut into the (already short) description of the custom OD scene.
It is kind of an advanced feature, so thanks for clarifying it here!

4

Hello, JPlou.
Thanks for the reply.
They instruct the moving file via clicking button after downloading sample assets in the setup_dnn_module.pdf of OpenCV plugin.

Although we fixed the path and modified the define code, the example scene (CustomObjectDetection) was not working. Just show me screen of camera. No letter box for AI recognition. I also tested 3D Object Detection scene of ZED and yolo4 example scene of openCVForUnity plugin. The scenes are working.

5

Hello @kdsss

I think the Object Detection is not started automatically in this scene on the ZED SDK side, so to see the detections you need to select the ZED Manager, and in the “Object Detection” part, click “Start Object Detection” once the scene is running.

6

Hello, JPlou.

I checked the program as your guidance. But the detection function was started automatically.
The ZED program printed the log, “Object Detection module started in 0 seconds.”
After 5~10 seconds, the program show me video of the zed camera. The frame of video is low (15 frame). It seems something wrong. Also, I restarted the zed object detection via clicking the button manually, but still no letter box for yolo4 AI in the unity screen.

7

@kdsss

Ok, I think I found the problem. It will be fixed in the next release.
In the meantime, to get it working, please look for this line in ZEDManager.cs:

od_param.instanceModuleID = objectDetectionInstanceID;

Just before it, add the line:

objectDetectionImageSyncMode = true;

So that you get a block looking like:

            sl.ObjectDetectionParameters od_param = new sl.ObjectDetectionParameters();
            objectDetectionImageSyncMode = true;
            od_param.instanceModuleID = objectDetectionInstanceID;
            od_param.imageSync = objectDetectionImageSyncMode;
            od_param.enableTracking = objectDetectionTracking;
            od_param.enableSegmentation = objectDetection2DMask;
            od_param.detectionModel = objectDetectionModel;
            od_param.maxRange = objectDetectionMaxRange;
            od_param.filteringMode = objectDetectionFilteringMode;

This should do the trick, sorry for the inconvenience.

1 Like
8

Hello, JPlou.

Thanks for the solution.
The customOD with yolo4 is working!

But I concern other errors due to the added code.
Because we are handling many functions with multiple zed cameras.
I will report a problem if we encounter another error.

Thank you for your quick and accurate responses!!

1 Like