Querying sl.ERROR_CODE.SUCCESS takes too long. Solution possible via multiprocessing?

Phyrokar · January 17, 2024, 8:27am

Hello, I still have the problem that code part
zed.grab(self.srt) == sl.ERROR_CODE.SUCCESS
takes a long time (up to 30ms) to process. In a thread I was presented with a solution using multiprocessing that works, but does not actually run multithreaded due to the Python GIL.

Is there a solution how I can skip or bypass this query? Unfortunately the zed object is not pickleable which is why multiprocessing doesn’t work either.

JPlou · January 17, 2024, 9:54am

Hello @Phyrokar,

The grab duration depends mainly on your depth mode and resolution. Can you try playing around with these settings and see the outcome?
Also, if you limit the camera’s fps in the initParameters, since the grab is blocking, it will take longer to match the desired fps.

If you can give more context on your hardware and the thread you mention, I may be able to give more insight, but as it is, I just see a grab running at 30 fps.

alassagne · January 17, 2024, 12:21pm

Hi,

Actually, 30ms wait of the grab function can be totally normal. If your camera runs at 30 frame/s, it cannot have the next frame until 33ms have passed. The grab() function will wait until the next image is available.
Can you share your program so that we can have a better idea if there is a problem or not ?

Phyrokar · January 17, 2024, 12:35pm

Thanks for your quick answer!

I pretty much use a modified version of the custom detector:

github.com

stereolabs/zed-sdk/blob/master/object detection/custom detector/python/pytorch_yolov8/detector.py

#!/usr/bin/env python3

import sys
import numpy as np

import argparse
import torch
import cv2
import pyzed.sl as sl
from ultralytics import YOLO

from threading import Lock, Thread
from time import sleep

import ogl_viewer.viewer as gl
import cv_viewer.tracking_viewer as cv_viewer

lock = Lock()
run_signal = False
exit_signal = False

This file has been truncated. show original

The problem is how it is processing the code. It grabs the images and then does all the detection. When it’s finished it loads another image and that takes a full 30ms again. It would simply be smart if it loaded the next image parallel to the detection.

alassagne · January 17, 2024, 4:19pm

Hi,

Thank you for these information. From what you explain, the behavior is not as expected.
Let’s imagine that your detection takes 33ms and you asked for 30 FPS. Then, a grab() call will not wait at all, because the next frame is already available. This is what you should get.

You are right that the GIL may be responsible for your performance issue. There are no reason that the grab would take so much time if the image is already available, unless it is blocked. You mentioned that you tried with multiprocess and it does not work either?

Can you send the report of ZED_Diagnostics so that we can have a better idea of your hardware ? If you are trying to run a NEURAL depth on a very slow hardware, you can encounter bad performance too.