Custom Object Detection Flickering

I am currently using the ROS2 wrapper with a custom detector for object detection (for tracking pose of robots). I noticed that there is often a “flickering” phenomenon. Here is a video of the flickering, where the object jumps between locations, even when sitting still. I tested my detector that I trained using YOLO v11 model and it seems to be working well so I don’t think the detector is the problem. I did, however, notice that the ObjectsStamped message has two objects even when there is only one object in the scene:

header:
  stamp:
    sec: 1751656557
    nanosec: 446281903
  frame_id: zed_left_camera_frame
objects:
- label: jackal
  label_id: 97
  sublabel: '0'
  confidence: 58.81842803955078
  position:
  - 3.7580249309539795
  - -0.5921962857246399
  - -0.36878183484077454
  position_covariance:
  - 0.0004960558726452291
  - -7.787102367728949e-05
  - -4.884915688307956e-05
  - 2.4928054699557833e-05
  - 7.668357284273952e-06
  - 1.8098267901223153e-05
  velocity:
  - -7.6945390701293945
  - 4.107448101043701
  - 0.8168067336082458
  tracking_available: true
  tracking_state: 1
  action_state: 1
  bounding_box_2d:
    corners:
    - kp:
      - 671
      - 355
    - kp:
      - 756
      - 355
    - kp:
      - 756
      - 454
    - kp:
      - 671
      - 454
  bounding_box_3d:
    corners:
    - kp:
      - 3.481393575668335
      - -0.28705349564552307
      - 0.014070898294448853
    - kp:
      - 4.0811052322387695
      - -0.28705349564552307
      - -0.04846164584159851
    - kp:
      - 4.080885887145996
      - -0.8900126218795776
      - -0.05056571960449219
    - kp:
      - 3.4811739921569824
      - -0.8900126218795776
      - 0.011966824531555176
    - kp:
      - 3.408859968185425
      - -0.284599632024765
      - -0.6815544366836548
    - kp:
      - 4.008571624755859
      - -0.284599632024765
      - -0.7440869808197021
    - kp:
      - 4.008352279663086
      - -0.8875588178634644
      - -0.7461910247802734
    - kp:
      - 3.4086406230926514
      - -0.8875588178634644
      - -0.6836584806442261
  dimensions_3d:
  - 0.6029629111289978
  - 0.6994009613990784
  - 0.6029630303382874
  skeleton_available: false

- label: jackal
  label_id: 96
  sublabel: '0'
  confidence: 69.107177734375
  position:
  - 6.663793087005615
  - -1.2240804433822632
  - -0.668341875076294
  position_covariance:
  - 0.004975060001015663
  - -0.0009138746536336839
  - -0.0004989710287190974
  - 0.00020810244313906878
  - 9.165657684206963e-05
  - 9.212526492774487e-05
  velocity:
  - 51.88490676879883
  - -10.522818565368652
  - -5.447213172912598
  tracking_available: true
  tracking_state: 2
  action_state: 1
  bounding_box_2d:
    corners:
    - kp:
      - 671
      - 354
    - kp:
      - 756
      - 354
    - kp:
      - 756
      - 454
    - kp:
      - 671
      - 454
  bounding_box_3d:
    corners:
    - kp:
      - 6.299466133117676
      - -0.8111989498138428
      - -0.1710924208164215
    - kp:
      - 7.122864723205566
      - -0.8111989498138428
      - -0.2569490373134613
    - kp:
      - 7.12256383895874
      - -1.6390562057495117
      - -0.25983792543411255
    - kp:
      - 6.29916524887085
      - -1.6390562057495117
      - -0.17398130893707275
    - kp:
      - 6.2056779861450195
      - -0.8080260753631592
      - -1.070559024810791
    - kp:
      - 7.02907657623291
      - -0.8080260753631592
      - -1.156415581703186
    - kp:
      - 7.028775215148926
      - -1.6358833312988281
      - -1.1593044996261597
    - kp:
      - 6.205376625061035
      - -1.6358833312988281
      - -1.073447823524475
  dimensions_3d:
  - 0.8278623819351196
  - 0.9043486714363098
  - 0.8278626799583435
  skeleton_available: false

Could I get some help in figuring out why this is happening and how I could fix it? Thank you!