Question about Object Detection with zed2

hgutze · October 24, 2022, 3:01pm

I am wondering which object detector is used in the zed-ros-examples and zed-ros-wrapper frameworks when using the example zed_display_rviz and also if it is possible to change this detector to a custom detector like it is shown here with the ZED SDK?

Thanks in advance!

Myzhar · October 24, 2022, 3:32pm

Hi @hgutze
the ZED ROS Wrapper allows you to use all the AI models available in the ZED SDK simply by changing the value of a parameter:

github.com

stereolabs/zed-ros-wrapper/blob/master/zed_wrapper/params/zed2.yaml#L22

    
      
          pos_tracking:
              imu_fusion:                 true            # enable/disable IMU fusion. When set to false, only the optical odometry will be used.
          
          
sensors:
              sensors_timestamp_sync:     false           # Synchronize Sensors messages timestamp with latest received frame
              max_pub_rate:               200.            # max frequency of publishing of sensors data. MAX: 400. - MIN: grab rate
              publish_imu_tf:             true            # publish `IMU -> <cam_name>_left_camera_frame` TF
          
          
object_detection:
              od_enabled:                 false           # True to enable Object Detection [not available for ZED]
              model:                      0               # '0': MULTI_CLASS_BOX - '1': MULTI_CLASS_BOX_ACCURATE - '2': HUMAN_BODY_FAST - '3': HUMAN_BODY_ACCURATE - '4': MULTI_CLASS_BOX_MEDIUM - '5': HUMAN_BODY_MEDIUM - '6': PERSON_HEAD_BOX
              confidence_threshold:       50              # Minimum value of the detection confidence of an object [0,100]
              max_range:                  15.             # Maximum detection range
              object_tracking_enabled:    true            # Enable/disable the tracking of the detected objects
              body_fitting:               false           # Enable/disable body fitting for 'HUMAN_BODY_X' models
              mc_people:                  true            # Enable/disable the detection of persons for 'MULTI_CLASS_BOX_X' models
              mc_vehicle:                 true            # Enable/disable the detection of vehicles for 'MULTI_CLASS_BOX_X' models
              mc_bag:                     true            # Enable/disable the detection of bags for 'MULTI_CLASS_BOX_X' models
              mc_animal:                  true            # Enable/disable the detection of animals for 'MULTI_CLASS_BOX_X' models
              mc_electronics:             true            # Enable/disable the detection of electronic devices for 'MULTI_CLASS_BOX_X' models
              mc_fruit_vegetable:         true            # Enable/disable the detection of fruits and vegetables for 'MULTI_CLASS_BOX_X' models

The custom object detector feature is not yet available in the ZED ROS Wrapper because of all the problems reported in this Github issue:

github.com/stereolabs/zed-ros2-wrapper

CUSTOM_BOX_OBJECTS support

opened 07:06PM - 26 Apr 22 UTC

osu-uwrt-bot

feature_request

### Preliminary Checks - [X] This issue is not a duplicate. Before opening a ne…w issue, please search existing issues. - [X] This issue is not a question, bug report, or anything other than a feature request directly related to this project. ### Proposal Support for the CUSTOM_BOX_OBJECTS from the ZED object detection module. The ZED SDK has a parameter which allows users to submit custom detections to it and it is able to track the objects as well as output the position and 3d bounding boxes for the objects. Currently the wrapper doesn't expose that part of the SDK and it isn't possible to access that part of the SDK through another ROS node while running the wrapper due to the wrapper having control of the camera. Would it be possible to enable that as an option within the launch file and then accept the objects as a message to which the wrapper outputs a Detections3Darray (or even a custom message with the info from the SDK). ### Use-Case I would be able to use a custom detector in ROS with 3D bounding boxes ### Anything else? _No response_

hgutze · October 25, 2022, 10:58am

okay, thank you very much for the information!

The two object detectors of the zed SDK “birds eye viewer” and “image viewer” differ from the object detector of the zed ROS wrapper/zed ROS examples, is that correct?

Do I understand correctly that I can also use the models MULTI_CLASS_BOX, MULTI_CLASS_BOX_ACCURATE, HUMAN_BODY_FAST, HUMAN_BODY_ACCURATE , MULTI_CLASS_BOX_MEDIUM, HUMAN_BODY_MEDIUM, PERSON_HEAD_BOX for the two object detectors “birds eye viewer” and “image viewer” axactly like in the case of the object detector for ROS?

Myzhar · October 25, 2022, 2:54pm

Yes, the index of the models matches the index of the ZED SDK.