Custom Detector for Object Detection not Working in With ROS 2 Wrapper

Software ROS2
1

I want to use the ZED ROS 2 wrapper’s object detection capabilities with a custom detector. Following the docs I was creating an ONNX file from a standard YOLO 8 (yolov8n.pt) model.

yolo export model=yolov8n.pt format=onnx simplify=True dynamic=False imgsz=512

Using the exported ONNX I could successfully run the ZED SDK samples (without ROS). However, if I run the ROS wrapper (even on the same SVO file) and try to visualize the results (using the zed_display_rviz2 package) no objects are detected at all.

I’m using the following parameters:

general:
          grab_resolution: 'HD1200' 
          pub_frame_rate: 30.0
        depth:
          depth_mode: 'NEURAL_PLUS' 
        object_detection:
          od_enabled: true
          model: 'CUSTOM_YOLOLIKE_BOX_OBJECTS'
          custom_onnx_file: '/path/to/yolov8n.onnx'
          custom_onnx_input_size: 512

I’ve seen you working on the ROS2 wrapper for the custom detector @Myzhar, do you have any suggestions on how to get this up and running?

2

Hi @birneamstiel
we are working on an improvement of the ZED ROS2 Wrapper concerning the Custom Detection support, but it should normally work as it is now.

Can you please test your model with the ZED SDK native code and let me know if it works with it?

3

Hi @Myzhar,

thanks for the quick reply. I have already tested my model with the tensorrt_yolov5-v6-v8_onnx_internal example you linked to (and with the same SVO file I used with the ROS2 wrapper) and it works fine.

Could you maybe list the steps you took to get a working model ONNX file along with a working list of object_detection parameters for the wrapper?

Thanks in advance!

4

You can use the same ONNX model and the same parameters of the tensorrt_yolov5-v6-v8_onnx_internal example.

For example:
custom_onnx_input_size: 512 is not the same because the example uses 320

Did you set this path to point to your ONNX file?

5

Yes I’ve set the right path, the wrapper does also find the model and optimizes successfully. However, no objects are detected…

Thanks for the hint regarding the input size. I’ve now created another ONNX file using a dynamic input size (yolo export model=yolov8n.pt format=onnx dynamic=True) and tried setting the custom_onnx_input_size parameter to 320 and 640, both did not work. The tensorrt_yolov5-v6-v8_onnx_internal sample works also with this ONNX file though.

Are you aware of any other ways of debugging the ROS wrapper to get closer to the root of the problem?

6

Are you allowed to share the ONNX model and an SVO file to test it?
You can send them to support@stereolabs.com

7

Yes, I’ve just sent the files. I look forward to hearing from you!

8

I suppose you are using the ZED SDK v5.0 because I was able to replicate the problem with the latest version of the ZED ROS2 Wrapper and the new SDK.

A fix will be pushed next week.

9

The ROS 2 Wrapper has been fixed to correctly work with Custom ONNX files with the ZED SDK v5.0 EA

10

Thanks @Myzhar, updating the SDK to v5 and pulling today’s latest changes did the trick, the custom ONNX files are working!

However, I’ve experienced some issues with launching the ROS wrapper with SVO files with the latest SDK and ROS wrapper version, maybe you could give it a look: Problems Launching the ROS2 Wrapper with a SVO File on SDK 5.0 EA

1 Like