Inconsistancy of detections after using ingest_custom_box_objects function, what are the reasons?

Software Python
1

Using
CUDA 11
ZED SDK 3.7

I am trying to figure out how does ingest_custom_box_objects works.
Having some inconsistent results of detections drops after ingesting.

the following code is itterating over a SVO file:

  if settings.detection_model == "tensorrt":
       detections, length = self.detect_trt(image_left[:, :, :3], need_resize=True)
                        objects_in = []
                        for i in range(length):
                            try:
                                tmp = sl.CustomBoxObjectData()
                                # Fill the detections into the correct SDK format
                                tmp.unique_object_id = sl.generate_unique_id()
                                tmp.label = FruitClass.get_label_idx(detections[i].name.decode("ascii"))
                                tmp.probability = detections[i].prob
                                b = detections[i].bbox
                                tmp.bounding_box_2d = self.xywh2xyxy((b.x,b.y,b.w,b.h))
                                tmp.is_grounded = True
                                objects_in.append(tmp)

self.cam.ingest_custom_box_objects(objects_in)
self.cam.retrieve_objects(self.objects)

Here is a comparison between the length of objects_in and self.objects.object_list

frame: 0 before ingest: 2 after ingest: 2
frame: 1 before ingest: 2 after ingest: 1
frame: 2 before ingest: 2 after ingest: 1
frame: 3 before ingest: 2 after ingest: 1
frame: 4 before ingest: 2 after ingest: 3
frame: 5 before ingest: 2 after ingest: 1
frame: 6 before ingest: 4 after ingest: 2
frame: 7 before ingest: 3 after ingest: 1
frame: 8 before ingest: 3 after ingest: 2
frame: 9 before ingest: 2 after ingest: 2
frame: 10 before ingest: 5 after ingest: 5
frame: 11 before ingest: 3 after ingest: 3
frame: 12 before ingest: 6 after ingest: 4
frame: 13 before ingest: 8 after ingest: 5
frame: 14 before ingest: 7 after ingest: 7
frame: 15 before ingest: 8 after ingest: 7
frame: 16 before ingest: 5 after ingest: 2
frame: 17 before ingest: 6 after ingest: 9
frame: 18 before ingest: 6 after ingest: 5
frame: 19 before ingest: 19 after ingest: 10
frame: 20 before ingest: 19 after ingest: 10
frame: 21 before ingest: 13 after ingest: 10
frame: 22 before ingest: 16 after ingest: 14
frame: 23 before ingest: 25 after ingest: 19
frame: 24 before ingest: 31 after ingest: 19
frame: 25 before ingest: 26 after ingest: 17
frame: 26 before ingest: 26 after ingest: 21
frame: 27 before ingest: 20 after ingest: 24
frame: 28 before ingest: 11 after ingest: 15
frame: 29 before ingest: 13 after ingest: 16
frame: 30 before ingest: 12 after ingest: 15
frame: 31 before ingest: 7 after ingest: 15
frame: 32 before ingest: 11 after ingest: 5
frame: 33 before ingest: 9 after ingest: 8
frame: 34 before ingest: 11 after ingest: 11
frame: 35 before ingest: 17 after ingest: 11
frame: 36 before ingest: 25 after ingest: 15

What could be the reason for the differences?

2

Hi Mark,

There are few reasons that may explain the difference :

  • The SDK applies a 3d NMS by default on inputs. If two objects are too close, we remove the background object and only keep the foreground.
    To disable this behavior, you have to use ObjectDetectionParameters::filtering_mode = OBJECT_FILTERING_MODE::NONE.

  • When tracking is enabled, to prevent detection jittering, we apply a prediction on detection so that the number of input detection will be less than the output detection.
    To disable, you can set ObjectDetectionParameters::prediction_timeout_s = 0.

  • Also, SDK keeps in memory all recently detected objects that are not visible anymore and output them with ObjectData::OBJECT_TRACKING_STATE = OBJECT_TRACKING_STATE::SEARCHING if tracking is enabled. By using only Objects with OBJECT_TRACKING_STATE::OK (stable tracked objects) or OBJECT_TRACKING_STATE::OFF (recently tracked objects or not tracked objects), you will have less objects.

Hope that it will help you. Let us know if there is still some difference after these changes.

Ludovick

2 Likes