ZED Gstreamer fps is very low than expected

Software Python
1

Hi,

I am using the following GStreamer pipeline:

gst-launch-1.0 zedsrc camera-resolution=3 camera-fps=30 stream-type=0 od-enabled=true od-detection-model=1 ! queue max-size-buffers=32 max-size-time=0 ! zedodoverlay ! queue max-size-buffers=32 max-size-time=0 ! autovideoconvert ! fpsdisplaysink sync=false

Setup Details:

Device: Jetson AGX Orin

JetPack Version: 6

ZED SDK Version: 4.2

GStreamer Version: 1.20

Observations:

  1. Without object detection enabled, the pipeline achieves 30 FPS. However, enabling object detection reduces the FPS to 15.

  2. If sync=true is set:

The FPS drops to 2, with dropped frames = 14.

  1. If sync=false is used:

The FPS is capped at 15, with dropped frames = 0, despite setting camera-fps=30.

Questions:

  1. Why does enabling object detection result in reduced FPS, even when camera-fps=30 is specified? Is there a way to optimize this pipeline for higher FPS with object detection enabled, what have to do in order to get 30 ?

  2. In your documentation, it is stated that camera-resolution=2 corresponds to HD720. However, during my testing, I achieved HD720 resolution with camera-resolution=3. Could you please clarify this discrepancy?

2

Hi @ashfaq
What Object Detection model are you using?
What is the power mode of your Jetson?

Object Detection consumes processing power according to the selected OD model the host cannot provide enough processing power to guarantee full FPS

From gst-inspect:

  camera-resolution   : Camera Resolution
                        flags: readable, writable
                        Enum "GstZedSrcRes" Default: 6, "Default value for the camera model"
                           (0): HD2K (USB3)      - 2208x1242
                           (1): HD1080 (USB3/GMSL2) - 1920x1080
                           (2): HD1200 (GMSL2)   - 1920x1200
                           (3): HD720 (USB3)     - 1280x720
                           (4): SVGA (GMSL2)     - 960x600
                           (5): VGA (USB3)       - 672x376
                           (6): Default value for the camera model - Automatic

Can you send me the link to the documentation to be fixed?

3

Thank u for replying

Object detection model : I tried these and was same results

(0): Object Detection Multi class FAST - Any objects, bounding box based

(1): Object Detection Multi class MEDIUM - Any objects, bounding box based, compromise between accuracy and speed

(2): Object Detection Multi class ACCURATE - Any objects, bounding box based, more accurate but slower than the base model

Power mode of my jetson is MAXn

  1. The camera-resolution link here : GStreamer - ZED Camera Source - Stereolabs

camera-resolution : Camera Resolution
flags: readable, writable
Enum “GstZedsrcResolution” Default: 7, “Default value for the camera model”
(1): HD2K (USB3) - 2208x1242
(3): HD1200 (GMSL2) - 1920x1200
(2): HD1080 (USB3/GMSL2) - 1920x1080
(4): HD720 (USB3) - 1280x720
(5): SVGA (GMSL2) - 960x600
(6): VGA (USB3) - 672x376
(7): Default value for the camera

*** also I want to ask u, because I’m new with zed gstreamer. Can I make the detection only for persons and ignore any other objects direct with gstreamer pipeline ?

4

I’m going to update the documentation to match the latest GStreamer package modifications.

  od-conf-animal      : Animal Detection Confidence Threshold
                        flags: readable, writable
                        Float. Range:              -1 -             100 Default:              35 
  od-conf-bag         : Bag Detection Confidence Threshold
                        flags: readable, writable
                        Float. Range:              -1 -             100 Default:              35 
  od-conf-electronics : Electronics Detection Confidence Threshold
                        flags: readable, writable
                        Float. Range:              -1 -             100 Default:              35 
  od-conf-fruit-vegetables: Fruit/Vegetables Detection Confidence Threshold
                        flags: readable, writable
                        Float. Range:              -1 -             100 Default:              35 
  od-conf-people      : People Detection Confidence Threshold
                        flags: readable, writable
                        Float. Range:              -1 -             100 Default:              35 
  od-conf-sport       : Sport Detection Confidence Threshold
                        flags: readable, writable
                        Float. Range:              -1 -             100 Default:              35 
  od-conf-vehicle     : Vehicle Detection Confidence Threshold
                        flags: readable, writable
                        Float. Range:              -1 -             100 Default:              35 
  od-confidence       : Minimum Detection Confidence
                        flags: readable, writable
                        Float. Range:               0 -             100 Default:              50

Set the confidence value of the types that you do not need to -1

Concerning the FPS issue, can you please try to remove the zedodoverlay element from the pipeline to check if it’s a data rendering issue?

5

  1. While object detection is true , with/without zedoverlay I got same result

  2. I want to give u an update : i did this and fps acheived with average 21
    “Sudo nvpmodel -m 0
    Sudo jetson_clocks”

  3. Also if i put paper on front of cam lens the fps beinv 29 , if there is objects the fps mainly 21

  4. I want to ask after a while mainly 15 minutes the stream stopping suddenly ?

6

This means that is not a rendering problem, but a processing power limitation

This means that you were not working in full power mode

Please add more information, a log, video. It’s not easy to solve this with no information.

7

Ok how to solve this ?

[/quote]
Please add more information, a log, video. It’s not easy to solve this with no information.
[/quote]
Once I go to the office, I will provide u all info.

I have another question , when use th object detection, the depth mode must be neural ?

8

No, it works with each depth mode option