Object detection just stops after a while


I’m currently building an Python application that uses a ZED 2 to detect and track people over a long period of time.
I’m using the object detection module with the HUMAN_BODY_FAST detection model since I need some of its features. The code is running on a Jetson AGX Xavier with Jetpack 4.5.1 (Ubuntu 18.04) and ZED SDK 3.5.

It’s normally working fine, but the problem is that after a non-specific amount of time, object detection just stops working. I can still grab video frames and get object detection data, but there is no new data inside the structure (I only ever get the same data over and over again for each new frame).

If I restart the script, it works again. If I run the camera at 30fps, this occurs pretty slowly (it generally occurs after over 8 hours of uptime). However, if I change the limit to 60fps, this occur much quicker and I generally lose object tracking in less than an hour (I can only get around 45fps on this system). CPU load never exceeds 70%.

I haven’t encountered this problem when working with an earlier SDK, but I didn’t test for longer periods of time back then.

Has anybody else seen this? Is there a way to prevent this or at least detect this condition and reset object tracking?

Here’s the part of my code that grabs the video frame and object data. The code is multiprocessed and this specific process only grabs the frame and data then shares them to other subprocesses.

    ### Init
        # Init memoire
        shm_buffer = sm.SharedMemory(name='sm_zed_img')
        img_output = np.ndarray(SH_zed, dtype=np.uint8, buffer=shm_buffer.buf)
        # Code d'initialisation du processus
        # Initialisation de la caméra ZED
        zed = sl.Camera()   # Création de l'objet caméra ZED

        # Configuration initiales avec objet initparameters de la ZED
        # Set configuration parameters
        init_params = sl.InitParameters()
        init_params.camera_resolution = sl.RESOLUTION.HD720 # Mode HD720 (FPS par defaut: 60Hz)
        #init_params.camera_resolution = sl.RESOLUTION.VGA # Mode HD720 (FPS par defaut: 60Hz)
        #init_params.coordinate_system = sl.COORDINATE_SYSTEM.RIGHT_HANDED_Y_UP # Systeme coordonnees
        init_params.coordinate_system = sl.COORDINATE_SYSTEM.RIGHT_HANDED_Z_UP_X_FWD # Systeme coordonnees
        init_params.camera_fps = 30                  # Limite de FPS a 30
        init_params.coordinate_units = sl.UNIT.METER # Unites en metres

        # Parametres runtime
        runtime_parameters = sl.RuntimeParameters()
        runtime_parameters.sensing_mode = sl.SENSING_MODE.STANDARD

        # Ouverture de la camera
        err = zed.open(init_params)
        if err != sl.ERROR_CODE.SUCCESS:
            raise Exception("Erreur d'ouverture de la camera ZED")

        # Activation du module de detection d'objet
        obj_param = sl.ObjectDetectionParameters()

        # Determine si le suivi des objets se fait au travers du flux d'images
        obj_param.enable_tracking = True       # Active le tracking positionel
        obj_param.image_sync = False           # Multithread async
        obj_param.detection_model = sl.DETECTION_MODEL.HUMAN_BODY_FAST
        obj_param.enable_body_fitting = True   # Le body fitting compute tous les joints, meme ceux qui n'ont pas ete detecte. Permet d'ameliorer la stabilite du modele.
        if obj_param.enable_tracking:
        camera_info = zed.get_camera_information()

        # Configure object detection runtime parameters
        # Config des parametres runtime du module de detection d'objets
        obj_runtime_param = sl.ObjectDetectionRuntimeParameters()
        obj_runtime_param.detection_confidence_threshold = zed_ConfidenceLevel

        obj_runtime_param.object_class_filter = [sl.OBJECT_CLASS.PERSON]    # Detecter seulement les objets "Person"

        # Creer les objets ZED a remplir dans la boucle principale
        objects = sl.Objects()
        image = sl.Mat()

    except Exception as errMsg:
        exceptStr = exceptStr+"_ INIT_"+str(errMsg)
        fl_RUN = False
    # Boucle principale du processus
    while (fl_RUN):    
        # Bloc donnees
            if zed.grab(runtime_parameters) == sl.ERROR_CODE.SUCCESS:   # Si grab reussi
                zed.retrieve_image(image, sl.VIEW.LEFT)                 # Obtenir l'image de la camera gauche
                zed.retrieve_objects(objects, obj_runtime_param)        # Obtenir les infos objets
                if shm_zed_lck.acquire(timeout=0.005):
                    img_output[...] = image.get_data()
                    pipe_zed_data.send((True, getObjectDict(objects)))        # On envoie les dernieres donnees
                # On envoie le FPS actuel de la boucle
                pipe_zed_ctrl.send(('F', zed.get_current_fps()))   # Valeur FPS rafraichie tout les Grab()

        except Exception as errMsg:
            exceptStr = exceptStr+"_ LOOP_"+str(errMsg)
            fl_RUN = False

@Numitron thank you for reporting this issue.
We are going to investigate it to provide a valid solution.

1 Like

is it solved? I also connected the ZED2 camera to the NANO board and is streaming with TX2. During streaming, the object module detection is in progress, the camera is turned on and the object is detected,
zed.retrieveObjects(objects, detection_parameters_rt); I always get a hang in this code.

Strangely, after connecting the ZED2 camera to the TX2 board, it works well when detecting objects. Why doesn’t it work while streaming?

Objects objects

int detection_confidence = 35; 
ObjectDetectionRuntimeParameters detection_parameters_rt(detection_confidence);