Concurrent custom Object Detection and Body Tracking

Software Python
1

Hello,
using some of your example programs, I’m trying to do custom object detection and body tracking.
This is how I set up both in python.

   positional_tracking_parameters = sl.PositionalTrackingParameters()
    # If the camera is static, uncomment the following line to have better performances and boxes sticked to the ground.
    positional_tracking_parameters.set_as_static = True
    zed.enable_positional_tracking(positional_tracking_parameters)

    body_param = sl.BodyTrackingParameters()
    body_param.enable_tracking = True                # Track people across images flow
    body_param.enable_body_fitting = False            # Smooth skeleton move
    body_param.detection_model = sl.BODY_TRACKING_MODEL.HUMAN_BODY_FAST 
    body_param.body_format = sl.BODY_FORMAT.BODY_18  # Choose the BODY_FORMAT you wish to use

    # Enable Object Detection module
    zed.enable_body_tracking(body_param)

    body_runtime_param = sl.BodyTrackingRuntimeParameters()
    body_runtime_param.detection_confidence_threshold = 40

    obj_param = sl.ObjectDetectionParameters()
    obj_param.detection_model = sl.OBJECT_DETECTION_MODEL.CUSTOM_BOX_OBJECTS
    obj_param.enable_tracking = True
    zed.enable_object_detection(obj_param)

    objects = sl.Objects()
    obj_runtime_param = sl.ObjectDetectionRuntimeParameters()
    bodies = sl.Bodies()

The body tracking part is working, but custom object detection model is not.
Please note that when I comment the body tracking detection, custom object detection is working.

Is custom object detection supposed to work concurrently with body tracking?
Is it working in C++ only?

The only example I found is C++ based and it’s not using custom object detection: zed-sdk/object detection/concurrent detections/cpp at master · stereolabs/zed-sdk · GitHub

Thank you,
Giorgio

1 Like
2

There seems to be a bug. You should be able to run body tracking and custom object detection concurrently. We’ll make a fix.

Antoine

3

Hi again,

To use concurrent object detection you need to set different instance_id, like in our sample zed-sdk/object detection/concurrent detections at master · stereolabs/zed-sdk · GitHub

Otherwise it returns an invalid error code. Can you check the error codes ?

4

Hello,
I set up the instance_module_id for both detector, in this way:

    body_param = sl.BodyTrackingParameters()
    body_param.enable_tracking = True 
    body_param.instance_module_id = 0
    obj_param = sl.ObjectDetectionParameters()
    obj_param.detection_model = sl.OBJECT_DETECTION_MODEL.CUSTOM_BOX_OBJECTS
    obj_param.instance_module_id = 1

I’ve got success in enabling both detectors:

    # Enable Object Detection module
    status = zed.enable_body_tracking(body_param)
    print(f'zed.enable_body_tracking status: {status}')
    if status != sl.ERROR_CODE.SUCCESS:
        print(repr(status))
        exit()
    status = zed.enable_object_detection(obj_param)
    print(f'zed.enable_object_detection status: {status}')
    if status != sl.ERROR_CODE.SUCCESS:
        print(repr(status))
        exit()

this is what I see printed:

zed.enable_body_tracking status: SUCCESS
zed.enable_object_detection status: SUCCESS

I still see the same behaviour:
when both detectors are enabled, just bodies are detected.
If I comment status = zed.enable_body_tracking(body_param) and obj_param.instance_module_id = 1, objects only (because bodies are disabled) get detected.

Giorgio

5

Hey @alassagne ,
can you confirm if the issue is caused by a bug in zed sdk, or is it something wrong in the way i set up the concurrent detection?

Thanks,
Giorgio

6

Hi, sorry for the delay. It seems so, but I did not catch it yet. Investigating.