3D object detection not working

Software Python
1

Hello,

I am trying to run a custom 3D object detection with the ZED 2 camera.

This is my setup:
Windows 10
Python: 3.8.5
ZED SDK: 3.8
yolov5

With the 3D object detection, I am trying to detect production modules. I followed your documentation in this link and the linked github code.

For the custom object detection, I trained a yolov5 network with my data. An example of the inference is shown in the picture below:

ZED inference
ZED inference1920×1080 119 KB

For the inference, I’m running this prompt:
python3 custom_object_detection_v2.py --weights ...\weights\best.pt --img_size 640

custom_object_detection_v2.py is this code, which you provided.

If I run the inference via the prompt the yolo documentation provides (python3 detect.py --weights ...\weights\best.pt --source 0) I get the following results:

Screenshot (322)
Screenshot (322)1651×482 78.6 KB

You can see the images of both cameras of the ZED 2 camera and all the right classifications.

So, I have the following questions:

  1. What am I missing, that I get excellent results when running the prompt of the yolo documentation and nearly no object detections when running the same model on the ZED script?
  2. Where do I have to change the class names, that they show up on the bbox of the inference?

Thanks in advance!

2

Hi,

  1. You can set the filtering mode to None in the Object detection parameters. We definitely need to add this in the documentation.
  2. You cannot display the class names now, we have a bug and labels are not forwarded correctly. Sorry for that :slight_smile:
3

Thanks a lot,

  1. How is the parameter called exactly which I need to change?
    I found this in the documentation and added obj_param.filtering_mode = None line to my script at this position:
obj_param = sl.ObjectDetectionParameters()
obj_param.detection_model = sl.DETECTION_MODEL.CUSTOM_BOX_OBJECTS
obj_param.enable_tracking = True
obj_param.filtering_mode = None
zed.enable_object_detection(obj_param)

But this results in the following error message:

  File "pyzed\sl.pyx", line 2533, in pyzed.sl.ObjectDetectionParameters.filtering_mode.__set__
TypeError

Can you point me at what to ingest exactly?

  1. Alright. is there any plan on fixing this bug?
4

It’s OBJECT_FILTERING_MODE.NONE from here : OBJECT_FILTERING_MODE Class Reference | API Reference | Stereolabs

5

Thanks, that already helped a little bit. Are there further options to get better results on the prediction from the ZED camera?
The amount of detected objects and the precision is still not comparable to the results of the yolo inference.

To elaborate on what I tried so far:

  1. I tried setting the OBJECT_FILTERING_MODE to NONE

  2. I set the detection confidence threshold in ObjectDetectionRuntime to 10:

obj_runtime_param = sl.ObjectDetectionRuntimeParameters(detection_confidence_threshold = 10)

  1. I tried to run depth_mode Ultra and Neural

  2. I tried obj.is_grounded with False and True

  3. I tried obj_param.enable_tracking with False and True

  4. I ran yolov5 n/s and m

So far with no satisfactory result