Object Detection Model "Loads" in 0 seconds

Software Unity
1

Hey! I’m trying to use the Zed Mini camera with Unity to run a custom-trained computer vision model. To test, we installed OpenCV for Unity (paid version) and tried running the included example with yolov4. However, the object detection model does not load, and the Unity message says “object detection model loaded in 0 seconds”. I did not make any changes. All I did was install the necessary packages and run the example demo.

Do you have any suggestions as to how to fix this? Thank you!

2

Hey! For anyone running into the same issue, please look at this post:

Custom Object Detection Unity SDK - Stereolabs Forums

It still says “Loaded in 0 seconds”, but the custom model actually runs :))

Happy Coding!

3

Hey! Okay so for the yolov4-tiny model, the example scene works. However, for .onnx file type, the example still cannot load the model… I’m trying to load the yolox_tiny.onnx file provided by OpenCV for Unity package…

Any advice? Thanks!

4

Hi @blueming,

Welcome to the Stereolabs forums :wave:

Glad to hear you were able to run the custom object detector. Our sample does not support .onnx files, I would suggest using weights files in any of the following formats: .caffemodel (Caffe), .pb (TensorFlow), .t7 or .net (Torch), .weights (Darknet).

I will log this improvement to be added in the future.

5

Hi! Thank you for the clarifications! I’ll try to convert the model to one of these file formats. We originally have our model as a .pth pytorch model. We then converted it to .onnx hoping we can call DNN.readNetFromONNX function… what’s stopping the ZED example from calling this function?

Thanks again!

6

Additionally, based on this post: Unity Custom Object Detection with *.pt file

Can we use a .pt Torch file instead…? Thanks!

7

Hi @blueming,

There should be no particular issue with using the readNetFromONNX function.
You can update this script and test it, we are happy to receive any pull requests on the ZED-Unity repo.

The sample supports all model types that the OpenCV for Unity package DNN module supports. I have not found a reference to .pt files being usable in the OpenCV documentation though.

8

Thanks again for the reply! I’ll give it a try :))

Any starting point / prior work you’re aware of? No worries if not. Thanks!

9

I don’t know too much about the OpenCV repository features for DNN, you can get a better idea by looking into their issues on GitHub.