Custom Object Detection Flickering

Software ROS2
1

I am currently using the ROS2 wrapper with a custom detector for object detection (for tracking pose of robots). I noticed that there is often a “flickering” phenomenon. Here is a video of the flickering, where the object jumps between locations, even when sitting still. I tested my detector that I trained using YOLO v11 model and it seems to be working well so I don’t think the detector is the problem. I did, however, notice that the ObjectsStamped message has two objects even when there is only one object in the scene:

header:
  stamp:
    sec: 1751656557
    nanosec: 446281903
  frame_id: zed_left_camera_frame
objects:
- label: jackal
  label_id: 97
  sublabel: '0'
  confidence: 58.81842803955078
  position:
  - 3.7580249309539795
  - -0.5921962857246399
  - -0.36878183484077454
  position_covariance:
  - 0.0004960558726452291
  - -7.787102367728949e-05
  - -4.884915688307956e-05
  - 2.4928054699557833e-05
  - 7.668357284273952e-06
  - 1.8098267901223153e-05
  velocity:
  - -7.6945390701293945
  - 4.107448101043701
  - 0.8168067336082458
  tracking_available: true
  tracking_state: 1
  action_state: 1
  bounding_box_2d:
    corners:
    - kp:
      - 671
      - 355
    - kp:
      - 756
      - 355
    - kp:
      - 756
      - 454
    - kp:
      - 671
      - 454
  bounding_box_3d:
    corners:
    - kp:
      - 3.481393575668335
      - -0.28705349564552307
      - 0.014070898294448853
    - kp:
      - 4.0811052322387695
      - -0.28705349564552307
      - -0.04846164584159851
    - kp:
      - 4.080885887145996
      - -0.8900126218795776
      - -0.05056571960449219
    - kp:
      - 3.4811739921569824
      - -0.8900126218795776
      - 0.011966824531555176
    - kp:
      - 3.408859968185425
      - -0.284599632024765
      - -0.6815544366836548
    - kp:
      - 4.008571624755859
      - -0.284599632024765
      - -0.7440869808197021
    - kp:
      - 4.008352279663086
      - -0.8875588178634644
      - -0.7461910247802734
    - kp:
      - 3.4086406230926514
      - -0.8875588178634644
      - -0.6836584806442261
  dimensions_3d:
  - 0.6029629111289978
  - 0.6994009613990784
  - 0.6029630303382874
  skeleton_available: false

- label: jackal
  label_id: 96
  sublabel: '0'
  confidence: 69.107177734375
  position:
  - 6.663793087005615
  - -1.2240804433822632
  - -0.668341875076294
  position_covariance:
  - 0.004975060001015663
  - -0.0009138746536336839
  - -0.0004989710287190974
  - 0.00020810244313906878
  - 9.165657684206963e-05
  - 9.212526492774487e-05
  velocity:
  - 51.88490676879883
  - -10.522818565368652
  - -5.447213172912598
  tracking_available: true
  tracking_state: 2
  action_state: 1
  bounding_box_2d:
    corners:
    - kp:
      - 671
      - 354
    - kp:
      - 756
      - 354
    - kp:
      - 756
      - 454
    - kp:
      - 671
      - 454
  bounding_box_3d:
    corners:
    - kp:
      - 6.299466133117676
      - -0.8111989498138428
      - -0.1710924208164215
    - kp:
      - 7.122864723205566
      - -0.8111989498138428
      - -0.2569490373134613
    - kp:
      - 7.12256383895874
      - -1.6390562057495117
      - -0.25983792543411255
    - kp:
      - 6.29916524887085
      - -1.6390562057495117
      - -0.17398130893707275
    - kp:
      - 6.2056779861450195
      - -0.8080260753631592
      - -1.070559024810791
    - kp:
      - 7.02907657623291
      - -0.8080260753631592
      - -1.156415581703186
    - kp:
      - 7.028775215148926
      - -1.6358833312988281
      - -1.1593044996261597
    - kp:
      - 6.205376625061035
      - -1.6358833312988281
      - -1.073447823524475
  dimensions_3d:
  - 0.8278623819351196
  - 0.9043486714363098
  - 0.8278626799583435
  skeleton_available: false

Could I get some help in figuring out why this is happening and how I could fix it? Thank you!

2

The flickering is caused by the tracking that switches between the two identified objects.
Please send the configuration file

3

Hello Myzhar,

Thanks for the response. I have attached common_stereo.yaml and custom_object_detection.yaml configuration files.

custom_object_detection.yaml (5.4 KB)

common_stereo.yaml (17.8 KB)

4

Hello Myzhar,

Thanks for the response. I have attached common_stereo.yaml and custom_object_detection.yaml configuration files.

custom_object_detection.yaml (5.4 KB)
common_stereo.yaml (17.8 KB)

[Discourse post]

5

It seems like commenting out the following lines from custom_object_detection.yaml fixes the problem:

is_grounded: true # Provide hypothesis about the object movements (degrees of freedom or DoF) to improve the object tracking
is_static: false # Provide hypothesis about the object staticity to improve the object tracking
tracking_timeout: -1.0 # Maximum tracking time threshold (in seconds) before dropping the tracked object when unseen for this amount of time
tracking_max_dist: -1.0 # Maximum tracking distance threshold (in meters) before dropping the tracked object when unseen for this amount of meters. Only valid for static object
max_box_width_normalized: -1.0 # Maximum allowed width normalized to the image size
min_box_width_normalized: -1.0 # Minimum allowed width normalized to the image size
max_box_height_normalized: -1.0 # Maximum allowed height normalized to the image size
min_box_height_normalized: -1.0 # Minimum allowed height normalized to the image size
max_box_width_meters: -1.0 # Maximum allowed 3D width
min_box_width_meters: -1.0 # Minimum allowed 3D width
max_box_height_meters: -1.0 # Maximum allowed 3D height
min_box_height_meters: -1.0 # Minimum allowed 3D height
object_acceleration_preset: 'DEFAULT' # Object acceleration preset. Possible values: 'DEFAULT', 'LOW', 'MEDIUM', 'HIGH'
max_allowed_acceleration: 100000.0 # If set with a different value from the default [100000], this value takes precedence over the selected preset, allowing for a custom maximum acceleration. Unit is m/s^2.

May I ask why these lines were problematic? Thanks!

6

You could try to tune the parameters instead of commenting on them.
They normally use the default values of the ZED SDK, so if you comment them you should assign the same values that you see in the YAML file