Issue with neural network using detectron2


I am testing two famous machine learning packages with ZED2i, such as detectron2 and yolov7.
yolov7 model works pretty well without any issue. But I had an issue when I use detectron2.

I trained my custom model by detectron2 and tried to predict objects based on the ZED camera image using OpenCV. Simultaneously, I tried to display a 3D point cloud in the OpenGL viewer. Unfortunately, I faced the error below.

RuntimeError: CUDA error: invalid argument
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.

This error was not shown when I passed my model through CPU. So, this error only occur when both ZED and Detectron use CUDA.

I was wondering if anyone succeed to integrate Detectron2 with ZED SDK.
Should I refer zed-pytorch (GitHub - stereolabs/zed-pytorch: 3D Object detection using the ZED and Pytorch)?
Any comments would be appreciated.

Thank you.


Hi @jlee
I think that this Github issue can contain useful information regarding your problem:

@jlee I am using the yolov5 sample and know of the custom detection documentation: Using the Object Detection API with a Custom Detector | Stereolabs

How did you get yoloV7 and therefore maybe yoloV8 running? I am still working my way through the example code and would also like to better understand the relation between the yolo object info and the SDK objects[] array. I would like to log the distance in vector space and the yolo object class.

If you are able to help me/ point me in a good direction to start, I would really appreciate that :slight_smile:

Have you tried to adapt the GitHub repository code to the new inference engine?

I am in the process of trying that but if someone has successfully done that, it would safe a lot of time as I am not a native computational engineer.

Hi! Did you manage to run the yolov7 model inference on the Zed camera?
I’m facing the same issue on a Zed2 camera and I don’t really understand how to adapt the existing example of object detection (using yolov5) to yolov7.
If anyone could point me to the right direction I would greatly appreciate it.