Sometimes process was stucked in sl::Camera::retrieveObjects

Software C++
1

i really appreciate the amazing product you provide us. while using it, i came up with some issues. one of them is, while i called the function sl::camera::retrieveObjects to ingest custom bboxes into zed 3d bboxes, process stucked in this function, and for thread safety, i used zed instance with lock, so when it stucked, other thread cannot acquired lock and cannot use zed instance. so is there anything wrong? do i use is with wrong actions? and i have set image_sync to false to avoid sync call.

  1. following is part of my code.
    sl::PositionalTrackingParameters zedPositionalTrackingPara;
    zedPositionalTrackingPara.set_as_static = true;
    zedCamInst->enablePositionalTracking(zedPositionalTrackingPara);
    sl::ObjectDetectionParameters obj_param;
    obj_param.image_sync = false;
    obj_param.detection_model = sl::DETECTION_MODEL::CUSTOM_BOX_OBJECTS;
    obj_param.enable_tracking = true;
    zedCamInst->enableObjectDetection(obj_param);
  2. // Retrieve colored point cloud. Point cloud is aligned on the left image.
    auto rm_ret = zed->retrieveMeasure(point_cloud, MEASURE::XYZRGBA);
    // Update pose data (used for projection of the mesh over the current image)
    sl::POSITIONAL_TRACKING_STATE tracking_state = zed->getPosition(cam_w_pose, REFERENCE_FRAME::WORLD);
    sl::CameraParameters calibration_params_left = zed->getCameraInformation().camera_configuration.calibration_parameters.left_cam;
    …
  3. std::vectorsl::CustomBoxObjectData detections = DetectionsToCustomBox(ai_result_objects);
    zed->ingestCustomBoxObjects(detections);
    auto ro_ret = zed->retrieveObjects(objects, obj_runtime_param);

product: ZED2i

OS: ubuntu18.04. aarch64
GPU: NVIDIA Jetson AGX Xavier
CUDA: 10.2.89
cuDNN: 8.0.0.180
TensorRT: 7.1.3.0
OpenCV: 4.1.1 compiled CUDA: YES
ZED version: ZED_SDK_Tegra_L4T32.7_v3.8.2.zstd

2

additionally, here is process stack.
(gdb) bt
#0 0x0000007f7f969844 in step_four(double**, int, int, int*, int*, double**, int&, int&, int&) () at /usr/local/zed/lib/libsl_zed.so
#1 0x0000007f7f96a7b4 in zed_tracking::Munkres(slutils::cv::Mat&, bool) () at /usr/local/zed/lib/libsl_zed.so
#2 0x0000007f7f872448 in std::vector<BoxMeasurement*, std::allocator<BoxMeasurement*> > HungarianAssociator<BoxTrack<BoxStateEstimator > >::associate<ASStep1BoxAppearance<BoxTrack<BoxStateEstimator >, (OBJECT_MOTION_TYPE)1>, 0>(std::vector<BoxTrack<BoxStateEstimator >, std::allocator<BoxTrack<BoxStateEstimator >> >&, std::vector<BoxMeasurement*, std::allocator<BoxMeasurement*> >&, float) () at /usr/local/zed/lib/libsl_zed.so
#3 0x0000007f7f874cec in BoxAssociationBehavior<BoxTrack<BoxStateEstimator >, HungarianAssociator<BoxTrack<BoxStateEstimator > > >::associate(std::vector<BoxTrack<BoxStateEstimator >, std::allocator<BoxTrack<BoxStateEstimator >> >&, std::vector<BoxMeasurement*, std::allocator<BoxMeasurement*> >&) () at /usr/local/zed/lib/libsl_zed.so
#4 0x0000007f7f87537c in BoxTracker<BoxAssociationBehavior<BoxTrack<BoxStateEstimator >, HungarianAssociator<BoxTrack<BoxStateEstimator > > >, BoxTrack<BoxStateEstimator >, 4>::associate_imp() () at /usr/local/zed/lib/libsl_zed.so
#5 0x0000007f7f876964 in std::vector<sl::Bbox2D, std::allocatorsl::Bbox2D > ai::ObjectDetector::track<(OBJECT_DETECTION_SOURCE_TYPE)1>(sl::Mat&, sl::Mat&, sl::Timestamp, sl::Transform, sl::Transform, sl::Transform, std::vector<BBoxOut, std::allocator >&) () at /usr/local/zed/lib/libsl_zed.so
#6 0x0000007f7fa0e2c4 in sl::ObjectsDetectorHandler::computeObjectDetection(sl::Camera*) () at /usr/local/zed/lib/libsl_zed.so
#7 0x0000007f7f9af7e8 in sl::camera::retrieveObjects(sl::Objects&, sl::ObjectDetectionRuntimeParameters) () at /usr/local/zed/lib/libsl_zed.so

3

Hi,

We are investigating this issue on our side.

Can you just try to verify if the 2d bounding boxes you ingest in the SDK are not of size 0?