Displaying object detection confidence scores with the custom object detector

a11s · December 14, 2023, 5:27pm

How can I display the confidence score for the detected objects alongside the bounding box visualisation and object class ID and distance with the custom detector?

github.com

stereolabs/zed-sdk/blob/512cb4b2589dc3789a14ba9caf5c1dc4c234be16/object detection/custom detector/python/pytorch_yolov8/detector.py#L58


      
          
          def detections_to_custom_box(detections, im0):
              output = []
              for i, det in enumerate(detections):
                  xywh = det.xywh[0]
          
                  # Creating ingestable objects for the ZED SDK
                  obj = sl.CustomBoxObjectData()
                  obj.bounding_box_2d = xywh2abcd(xywh, im0.shape)
                  obj.label = det.cls
                  obj.probability = det.conf
                  obj.is_grounded = False
                  output.append(obj)
              return output
          
          
          def torch_thread(weights, img_size, conf_thres=0.2, iou_thres=0.45):
              global image_net, exit_signal, run_signal, detections
          
              print("Intializing Network...")

JPlou · December 18, 2023, 9:13am

Hello @a11s,

You can mimic how we display the class ID, in the render2D call in the main loop.

It’s a call to cv2.putText, placing some text at some coordinates.
You can add the confidence display here.

a11s · May 30, 2024, 10:22pm

Hi @JPlou, thanks for the pointer.

I have tried replacing the following lines in the render_2D function of tracking_viewer.py:

text = "class " + str(obj.raw_label)
text_color = (255, 255, 255, 255)
cv2.putText(left_display, text, text_position, cv2.FONT_HERSHEY_COMPLEX_SMALL, 0.5, text_color, 1)

With the following lines:

if isinstance(obj, sl.CustomBoxObjectData):
label = obj.label
probability = obj.probability
text = f"class {obj.raw_label} ({probability:.2f})"
else:
text = "class " + str(obj.raw_label)

text_color = (255, 255, 255, 255)
cv2.putText(left_display, text, text_position, cv2.FONT_HERSHEY_COMPLEX_SMALL, 0.5, text_color, 1)

The code runs fine, but the confidence/probability score is still not showing. Where am I going wrong?

JPlou · June 5, 2024, 5:08pm

@a11s

By the time it reaches the viewer, the obj that is displayed is ingested by the SDK, and processed into an ObjectData.

You should try displaying the confidence of ObjectData instead of the probability of CustomBoxObjectData.