I have a ZED Mini setup with the Python API and latest SDK (4.0.7), all looks good on the Diagnostic tool, and I’ve installed Ultralytics. But when I run example tutorial:
python detector.py --weights yolov8m.pt
the following happens in the terminal:
Intializing Network...
Initializing Camera...
Downloading https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8m.pt to 'yolov8m.pt'...
Initialized Camera
and the ZED object detection window opens.
However, the window is a black screen and eventually becomes unresponsive, and the program needs to be interrupted to close.
I’m not sure if the pt file is being downloaded, but I turned off my firewall in case it was that, but it did not fix it.
I solved this by running the detector from my D: drive.
When I tried to record an SVO from the Explorer it wouldn’t work due to admin or storage issues. My C: drive was a bit low, so I cleared out some space, but it still won’t record an SVO to C:, so I switched to D: and it records fine.
I have run the Explorer as admin and restarted my computer after clearing space, but it still won’t save to C:.
Although I have them both working from my D: drive, how can I fix the issue so that they’ll work from my C: drive again?
Also, the custom detector is not working in Unity. When I run the custom detector.py file, the custom detection model runs in its own window, then I set the model in Unity to CUSTOM_BOX_OBJECTS, and then play the Unity scene, but it says the camera can’t be loaded.
As soon as I quit the custom detector running and it frees up the camera, the camera is detected in the Unity scene.
But how to I get these to work together so that the camera and custom detector run inside my Unity project/scene?
There is an example scene that uses the custom detector feature with Opencv and yolo as detector : https://github.com/stereolabs/zed-unity/tree/master/ZEDCamera/Assets/ZED/Examples/Object%20Detection
I think you might take a lot at the "ZEDCustomObjDetection.cs" script. In this script, the custom detector runs on each frame from the ZED Camera, and the 2D output is ingested inside the ZED SDK and gets transformed into 3D data.
I’ve run through those scripts and documentation, and got it setup in Unity, cleared all the exceptions so it plays without any errors; however, the camera image has no detections/bounding boxes appearing in the scene (see attached image).
I’ve cross-checked with the provided Custom Object Detection scene and ensured all was set up correctly. Without any errors/tracebacks I’m stumped as what to try next. Can you see what’s missing?
Is it something to do with the Image Retriever? There is nothing attached to it, but when I play the scene, the Zed to OpenCV Retriever appears. Is that behaviour expected?
Great thanks! Please let me know when the proper fix is uploaded.
Also segmentation doesn’t seem to work. I’ve changed the model to yolov7-tiny and checked ‘Enable Segmentation’, but nothing happens. yolov7 can perform segmentation, so how do I get the 2D segmentation visualised? Do I need to use ‘Mask2DDisplay’ in some way?
