FPS of Depth/VIdeo compressed topics are not stable at 30 FPS when subscribing 3 of them.

Software ROS2
, , ,
1

Hardware : Zed Mini , Zed Box Orin.
Using 3.0 cable.

Software: ROS2 Humble,
zed_ros2_wrapper.
Using HD720 resolution
At 30 FPS.

Subscringing left, right, depth compressed topics, and writing them into storage using GitHub - berndpfrommer/rosbag2_composable_recorder: ROS2 package that allows recording without interprocess communication
Average fps from all 3 topics are same 25.4, 26,5 something like this not 30fps.
Used debug log for it and detected that ,ros2 wrapper is publishing at 15 hz every 5-6 frames.
and it is making fps drop for average.
Using jetson in full power mode , jetson_clocks activated.
In jtop 30% cpu load and 50-60 % gpu.
Using openni mode 24-bitrate.

What can be cause of it?
Any suggestion could help me to solve issue.

Thanks

2

Hi @rikhsitlladeveloper
Welcome to the StereoLabs community.

Using compressed topics requires a huge amount of CPU processing that could cause an overload and then lower FPS.

How are you measuring the FPS?
Have you checked the node diagnostic to confirm that the internal processing frequency is reduced too?

3

Hi @Myzhar , I am writing to mcap using rosbag2 composable recorder. and mcap info showing that it is 24 - 25 fps. Also while running ros2_zed_wrapper , using debug for visualize HZ of publishing images. It is showing 15HZ after 4-5 30 HZ . How to check internal processing frequency?
Also checked with using uncompressed images same error happened and mean fps is 27-28.

4

You can use the Robot Monitor in rqt:

Please send a screenshot.

This is another highly demanding task that adds load to the CPU.
Also the disk write speed can affect the final results.

5

Is any way writing/recording depth& rgb images in real time with HD720 at 30 fps into MCAP format ?

6

Many factors can affect this process. Please check the internal processing frequency, then we can proceed further with the analysis.

7 
header:
  stamp:
    sec: 1769707513
    nanosec: 757880008
  frame_id: ''
status:
- level: "\0"
  name: 'zed_node: ZED-M'
  message: Camera grabbing
  hardware_id: 'Stereolabs ZED-M - ''zed'' - S/N: 11178431'
  values:
  - key: Uptime
    value: 0 days, 0 hours, 1 min, 50.6137 sec
  - key: IPC Enabled
    value: 'YES'
  - key: Camera Grab rate
    value: 30 Hz
  - key: Data Capture
    value: 'Mean Frequency: 23.5 Hz (78.3%)'
  - key: Data Capture
    value: 'Tot. Processing Time: 0.042404 sec (Max. 0.033 sec)'
  - key: Frame Drop rate
    value: 721/3377 (21.3503%)
  - key: Input mode
    value: Live Camera
  - key: Video/Depth
    value: 'Mean Frequency: 23.7 Hz (79.0%)'
  - key: Video/Depth
    value: 'Processing Time: 0.012558 sec (Max. 0.033 sec)'
  - key: Depth status
    value: ACTIVE
  - key: Depth mode
    value: NEURAL LIGHT
  - key: Point Cloud
    value: Topic not subscribed
  - key: GNSS Fusion
    value: DISABLED
  - key: Odometry tracking status
    value: OK
  - key: Spatial Memory status
    value: OK
  - key: TF Odometry
    value: DISABLED
  - key: TF Pose
    value: DISABLED
  - key: 3D Mapping
    value: INACTIVE
  - key: Object Detection
    value: INACTIVE
  - key: Body Tracking
    value: INACTIVE
  - key: TF IMU
    value: DISABLED
  - key: IMU
    value: 'Mean Frequency: 89.9 Hz'
  - key: Magnetometer Sensor
    value: N/A
  - key: Barometer Sensor
    value: N/A
  - key: Left CMOS Temp.
    value: N/A
  - key: Right CMOS Temp.
    value: N/A
  - key: SVO Recording
    value: NOT ACTIVE
  - key: Streaming Server
    value: NOT ACTIVE
---
header:
  stamp:
    sec: 1769707514
    nanosec: 757870568
  frame_id: ''
status:
- level: "\0"
  name: 'zed_node: ZED-M'
  message: Camera grabbing
  hardware_id: 'Stereolabs ZED-M - ''zed'' - S/N: 11178431'
  values:
  - key: Uptime
    value: 0 days, 0 hours, 1 min, 51.6137 sec
  - key: IPC Enabled
    value: 'YES'
  - key: Camera Grab rate
    value: 30 Hz
  - key: Data Capture
    value: 'Mean Frequency: 23.2 Hz (77.2%)'
  - key: Data Capture
    value: 'Tot. Processing Time: 0.042979 sec (Max. 0.033 sec)'
  - key: Frame Drop rate
    value: 728/3408 (21.3615%)
  - key: Input mode
    value: Live Camera
  - key: Video/Depth
    value: 'Mean Frequency: 23.7 Hz (78.9%)'
  - key: Video/Depth
    value: 'Processing Time: 0.012458 sec (Max. 0.033 sec)'
  - key: Depth status
    value: ACTIVE
  - key: Depth mode
    value: NEURAL LIGHT
  - key: Point Cloud
    value: Topic not subscribed
  - key: GNSS Fusion
    value: DISABLED
  - key: Odometry tracking status
    value: OK
  - key: Spatial Memory status
    value: OK
  - key: TF Odometry
    value: DISABLED
  - key: TF Pose
    value: DISABLED
  - key: 3D Mapping
    value: INACTIVE
  - key: Object Detection
    value: INACTIVE
  - key: Body Tracking
    value: INACTIVE
  - key: TF IMU
    value: DISABLED
  - key: IMU
    value: 'Mean Frequency: 98.1 Hz'
  - key: Magnetometer Sensor
    value: N/A
  - key: Barometer Sensor
    value: N/A
  - key: Left CMOS Temp.
    value: N/A
  - key: Right CMOS Temp.
    value: N/A
  - key: SVO Recording
    value: NOT ACTIVE
  - key: Streaming Server
    value: NOT ACTIVE

here is output of /diagnostics topic

8

This confirms that the CPU is overloaded and cannot handle all the tasks (compression, recording, etc) simultaneously.

When the CPU is not overloaded, the Tot. Processing Time: is lower than the capture period.
In your case, it’s exceeding the capture period by a ~28.5% rate.

You have a few methods to reduce the load:

9

Disabled Point cloud.
Can not decrease grab or publishing rate from 30 FPS.
How to get continues 30 fps ?
I need rgb, depth at 30 HZ

10

Have you tried to connect another device to the ZED Box with ethernet cable and to perform the recording on it to reduce the amount of CPU load while capturing the data?

Also read this section of the documentation that can allow you to reduce the amount of load caused by the ROS 2 middleware communication:

11

It is happening same within local device. without remote device.

I did all DDS tuning with guide.
Did not help for this issue.

12

@Myzhar Is it possible to get 30 FPS and record into mcap in HD720 with ZED Box Orin and Zed Mini camera?

13

You can record an SVO at 30 FPS, but I cannot confirm that it’s possible to record a bag in mcap without hardware compression support.