Hi,
I was wondering if there is a way to access the human body tracking capability while also running the ZED ROS 2 wrapper.
I tried running the ZED SDK and the ROS 2 wrapper together, and it did not work, and suggestions on how this can be achieved?
Thanks in advance!
Hi @bd16
Welcome to the Stereolabs community.
The ZED ROS 2 Wrapper provides you with full Body Tracking support:
Can you add details to better explain your requirements?
Hi @Myzhar,
Thanks for the response, I did not realize that the ROS 2 interface exposed this feature at the time. It is exactly what I wanted!
However, I noticed one issue when using the human tracking feature. When the human is in the field of view, the ID of that human is fixed. However, when the human goes outside of the view and then comes back the ID of that same human would be different (at least that is what I have observed). Is there a way to resolve this issue using the ROS 2 interface?
It seems like the functionality of human re-identification has been introduced in v3.5: ZED SDK 3.5 - Download | Stereolabs, and there is an example using this feature with the ZED SDK: zed-sdk/object detection/birds eye viewer at master · stereolabs/zed-sdk · GitHub. I wonder how/if I can achieve the same thing using the ROS 2 wrapper?
Thanks in advance!
Try to raise the value of this parameter to keep the skeleton “alive”.
Hi,
@Myzhar Thanks for the response!
I was playing around with this parameter, and after setting it to 30 seconds, I still find it that once I get outside of the camera field of view and re-enter I get assigned a new ID. Are there any other parameters that I need to set?
My current body tracking and object detection parameters are:
object_detection:
od_enabled: false # True to enable Object Detection
model: "MULTI_CLASS_BOX_MEDIUM" # 'MULTI_CLASS_BOX_FAST', 'MULTI_CLASS_BOX_MEDIUM', 'MULTI_CLASS_BOX_ACCURATE', 'PERSON_HEAD_BOX_FAST', 'PERSON_HEAD_BOX_ACCURATE'
allow_reduced_precision_inference: true # Allow inference to run at a lower precision to improve runtime and memory usage
max_range: 20.0 # [m] Defines a upper depth range for detections
confidence_threshold: 50.0 # [DYNAMIC] - Minimum value of the detection confidence of an object [0,99]
prediction_timeout: 0.5 # During this time [sec], the object will have OK state even if it is not detected. Set this parameter to 0 to disable SDK predictions
filtering_mode: 1 # '0': NONE - '1': NMS3D - '2': NMS3D_PER_CLASS
mc_people: true # [DYNAMIC] - Enable/disable the detection of persons for 'MULTI_CLASS_X' models
mc_vehicle: true # [DYNAMIC] - Enable/disable the detection of vehicles for 'MULTI_CLASS_X' models
mc_bag: true # [DYNAMIC] - Enable/disable the detection of bags for 'MULTI_CLASS_X' models
mc_animal: true # [DYNAMIC] - Enable/disable the detection of animals for 'MULTI_CLASS_X' models
mc_electronics: true # [DYNAMIC] - Enable/disable the detection of electronic devices for 'MULTI_CLASS_X' models
mc_fruit_vegetable: true # [DYNAMIC] - Enable/disable the detection of fruits and vegetables for 'MULTI_CLASS_X' models
mc_sport: true # [DYNAMIC] - Enable/disable the detection of sport-related objects for 'MULTI_CLASS_X' models
body_tracking:
bt_enabled: true # True to enable Body Tracking
model: "HUMAN_BODY_MEDIUM" # 'HUMAN_BODY_FAST', 'HUMAN_BODY_MEDIUM', 'HUMAN_BODY_ACCURATE'
body_format: "BODY_38" # 'BODY_18','BODY_34','BODY_38','BODY_70'
allow_reduced_precision_inference: false # Allow inference to run at a lower precision to improve runtime and memory usage
max_range: 20.0 # [m] Defines a upper depth range for detections
body_kp_selection: "FULL" # 'FULL', 'UPPER_BODY'
enable_body_fitting: false # Defines if the body fitting will be applied
enable_tracking: true # Defines if the object detection will track objects across images flow
prediction_timeout_s: 30.0 # During this time [sec], the skeleton will have OK state even if it is not detected. Set this parameter to 0 to disable SDK predictions
confidence_threshold: 50.0 # [DYNAMIC] - Minimum value of the detection confidence of skeleton key points [0,99]
minimum_keypoints_threshold: 5 # [DYNAMIC] - Minimum number of skeleton key points to be detected for a valid skeleton
I also tried setting od_enabled to true and prediction_timeout under object_detection to 30. However, the same behavior is observed where I get assigned a new ID when I re-enter the FOV.
I also played with the object_detection_birds_view.py script in the zed-sdk/object detection/birds eye viewer/python module, and got the following output:
During last batch processing: 8 Objets were detected: 58 72 74 75 76 77 78 79
While 3 different only after reID: 0 1 7
During last batch processing: 4 Objets were detected: 78 72 76 74
While 5 different only after reID: 0 1 3 7 8
During last batch processing: 1 Objets were detected: 72
While 1 different only after reID: 3
During last batch processing: 2 Objets were detected: 72 80
While 1 different only after reID: 3
During last batch processing: 2 Objets were detected: 72 81
While 1 different only after reID: 3
During last batch processing: 3 Objets were detected: 81 72 82
While 2 different only after reID: 0 3
During last batch processing: 3 Objets were detected: 81 72 83
While 2 different only after reID: 0 3
During last batch processing: 5 Objets were detected: 81 72 84 85 86
While 2 different only after reID: 0 3
During last batch processing: 2 Objets were detected: 81 84
While 3 different only after reID: 0 3 7
During last batch processing: 1 Objets were detected: 81
While 1 different only after reID: 0
During last batch processing: 3 Objets were detected: 81 87 88
While 1 different only after reID: 0
I assume the line While 1 different only after reID: 0 shows the ids of the objects in the FOV after reID?
Can you please test if this behavior is also present when using the zed-sdk repository Body Tracking examples?
Hi @Myzhar,
When using the body tracking test code, if I move outside of the frame and then come back I noticed that sometimes the color of my skeleton changes, I assume that means the ID got changed?
Yes, you are right. It can happen that the ReId algorithm fails to associate the same person to the same ID