Somehow zed ROS2 wrapper is not able to keep desired frequency when used in ROS2 foxy and Jetson Xavier AGX. Even when using 15 Hz output rate as well as disabling object detection and mapping the problem still occurs. What may cause that?
Hi @JereK
please add more information:
- resolution
- frame rate
- depth mode
- AI modules active
- other
How are you measuring the “output rate”? What topic?
What is the power mode of the Jetson module?
Are you running the jetson_clocks script?
- resolution: 1080 but I also tested lower and same happened at some level also.
- frame rate: 15
- depth mode: Performance
- AI modules active: if you mean that if I have disabled object detection and other functionalities of zed then yes. If you mean if I have nvidia AI modules then also yes, they should be active.
- other: -
I am using ros2 topic hz with regular image topics (not pointcloud). Jetson is using maxn mode. And yes I have ran once jetson_clocks earlier.
Another point is that i have been using same camera with regular laptop without problems.
How are you measuring the “framerate”?
Are you running rviz on the Jetson?
Please use rqt and the diagnostic information (Runtime Monitor plugin) to have an idea of the real performance of the wrapper.
ros2 topic hz does not always provide correct information.
This is what jetson_clocks gives:
SOC family:tegra194 Machine:Jetson-AGX
Online CPUs: 0-7
cpu0: Online=1 Governor=schedutil MinFreq=1190400 MaxFreq=2265600 CurrentFreq=2265600 IdleStates: C1=1 c6=1
cpu1: Online=1 Governor=schedutil MinFreq=1190400 MaxFreq=2265600 CurrentFreq=2265600 IdleStates: C1=1 c6=1
cpu2: Online=1 Governor=schedutil MinFreq=1190400 MaxFreq=2265600 CurrentFreq=2265600 IdleStates: C1=1 c6=1
cpu3: Online=1 Governor=schedutil MinFreq=1190400 MaxFreq=2265600 CurrentFreq=2265600 IdleStates: C1=1 c6=1
cpu4: Online=1 Governor=schedutil MinFreq=1190400 MaxFreq=2265600 CurrentFreq=2265600 IdleStates: C1=1 c6=1
cpu5: Online=1 Governor=schedutil MinFreq=1190400 MaxFreq=2265600 CurrentFreq=2265600 IdleStates: C1=1 c6=1
cpu6: Online=1 Governor=schedutil MinFreq=1190400 MaxFreq=2265600 CurrentFreq=1497600 IdleStates: C1=1 c6=1
cpu7: Online=1 Governor=schedutil MinFreq=1190400 MaxFreq=2265600 CurrentFreq=2265600 IdleStates: C1=1 c6=1
GPU MinFreq=114750000 MaxFreq=1377000000 CurrentFreq=318750000
EMC MinFreq=204000000 MaxFreq=2133000000 CurrentFreq=2133000000 FreqOverride=0
DLA0_CORE MinFreq=0 MaxFreq=1395200000 CurrentFreq=1395200000
DLA0_FALCON MinFreq=0 MaxFreq=844800000 CurrentFreq=844800000
DLA1_CORE MinFreq=0 MaxFreq=1395200000 CurrentFreq=1395200000
DLA1_FALCON MinFreq=0 MaxFreq=844800000 CurrentFreq=844800000
PVA0_VPS0 MinFreq=0 MaxFreq=1088000000 CurrentFreq=1088000000
PVA0_VPS1 MinFreq=0 MaxFreq=1088000000 CurrentFreq=1088000000
PVA0_AXI MinFreq=0 MaxFreq=844800000 CurrentFreq=844800000
PVA1_VPS0 MinFreq=0 MaxFreq=1088000000 CurrentFreq=1088000000
PVA1_VPS1 MinFreq=0 MaxFreq=1088000000 CurrentFreq=1088000000
PVA1_AXI MinFreq=0 MaxFreq=844800000 CurrentFreq=844800000
CVNAS MinFreq=0 MaxFreq=1356800000 CurrentFreq=1356800000
FAN Dynamic Speed control=active hwmon2_pwm=0
NV Power Mode: MAXN
rqt seems to indicate that it is running in 15 Hz, but why is the ros2 topic hz so different?
Also could you provide help in the regard how do I subsribe those topics in python with correct QOS profile. Since I am seeing the same behaviour in my python node when subscribing these image topics that at some time instances it does not get data that fast as it should. In github page there is only examples related to c++.
edit: I am kind a sure that this problem is related to qos profiles in ros2. When I am using RELIABLE and VOLATILE I am able to subscribe all the image messages however at some point node stop subscribing. I have no idea what might cause that.
Also with this launch file:
# config/common_yaml
# Common parameters to Stereolabs ZED and ZED mini cameras
#
# Note: the parameter svo_file is passed as exe argumet
---
/**:
ros__parameters:
general:
svo_file: "" # usually overwritten by launch file
svo_loop: false # Enable loop mode when using an SVO as input source
svo_realtime: true # if true SVO will be played trying to respect the original framerate eventually skipping frames, otherwise every frame will be processed respecting the `pub_frame_rate` setting
camera_timeout_sec: 5
camera_max_reconnect: 5
camera_flip: false
zed_id: 0 # usually overwritten by launch file
serial_number: 0 # usually overwritten by launch file
grab_resolution: 'HD1080' # The native camera grab resolution. 'HD2K', 'HD1080', 'HD720', 'VGA'
pub_resolution: 'HD1080' # The resolution used for output. 'HD2K', 'HD1080', 'HD720', 'MEDIUM', 'VGA', 'LOW'
sdk_verbose: 1
grab_frame_rate: 15 # ZED SDK internal grabbing rate
pub_frame_rate: 15.0 # [DYNAMIC] - frequency of publishing of visual images and depth images
gpu_id: -1
region_of_interest: '' # A polygon defining the ROI where the ZED SDK perform the processing ignoring the rest. Coordinates must be normalized to '1.0' to be resolution independent.
#region_of_interest: '[[0.25,0.33],[0.75,0.33],[0.75,0.5],[0.5,0.75],[0.25,0.5]]' # A polygon defining the ROI where the ZED SDK perform the processing ignoring the rest. Coordinates must be normalized to '1.0' to be resolution independent.
#region_of_interest: '[[0.25,0.25],[0.75,0.25],[0.75,0.75],[0.25,0.75]]' # A polygon defining the ROI where the ZED SDK perform the processing ignoring the rest. Coordinates must be normalized to '1.0' to be resolution independent.
#region_of_interest: '[[0.5,0.25],[0.75,0.5],[0.5,0.75],[0.25,0.5]]' # A polygon defining the ROI where the ZED SDK perform the processing ignoring the rest. Coordinates must be normalized to '1.0' to be resolution independent.
video:
extrinsic_in_camera_frame: false # if `false` extrinsic parameter in `camera_info` will use ROS native frame (X FORWARD, Z UP) instead of the camera frame (Z FORWARD, Y DOWN) [`true` use old behavior as for version < v3.1]
brightness: 4 # [DYNAMIC]
contrast: 4 # [DYNAMIC]
hue: 0 # [DYNAMIC]
saturation: 4 # [DYNAMIC]
sharpness: 4 # [DYNAMIC]
gamma: 8 # [DYNAMIC] - Requires SDK >=v3.1
auto_exposure_gain: true # [DYNAMIC]
exposure: 80 # [DYNAMIC]
gain: 80 # [DYNAMIC]
auto_whitebalance: true # [DYNAMIC]
whitebalance_temperature: 42 # [DYNAMIC] - [28,65] works only if `auto_whitebalance` is false
qos_history: 1 # '1': KEEP_LAST - '2': KEEP_ALL
qos_depth: 1 # Queue size if using KEEP_LAST
qos_reliability: 1 # '1': RELIABLE - '2': BEST_EFFORT -
qos_durability: 2 # '1': TRANSIENT_LOCAL - '2': VOLATILE
depth:
quality: 0 # '0': NONE, '1': PERFORMANCE, '2': QUALITY, '3': ULTRA - '4': NEURAL - Note: if '0' all the modules that requires depth extraction are disabled by default (Pos. Tracking, Obj. Detection, Mapping, ...)
sensing_mode: 0 # '0': STANDARD, '1': FILL
depth_stabilization: true # Forces positional tracking to start if 'true'
openni_depth_mode: false # 'false': 32bit float [meters], 'true': 16bit unsigned int [millimeters]
point_cloud_freq: 15.0 # [DYNAMIC] - frequency of the pointcloud publishing (equal or less to `grab_frame_rate` value)
depth_confidence: 50 # [DYNAMIC]
depth_texture_conf: 100 # [DYNAMIC]
remove_saturated_areas: true # [DYNAMIC]
qos_history: 1 # '1': KEEP_LAST - '2': KEEP_ALL
qos_depth: 1 # Queue size if using KEEP_LAST
qos_reliability: 1 # '1': RELIABLE - '2': BEST_EFFORT -
qos_durability: 2 # '1': TRANSIENT_LOCAL - '2': VOLATILE
pos_tracking:
pos_tracking_enabled: false # True to enable positional tracking from start
imu_fusion: true # enable/disable IMU fusion. When set to false, only the optical odometry will be used.
publish_tf: true # [usually overwritten by launch file] publish `odom -> base_link` TF
publish_map_tf: true # [usually overwritten by launch file] publish `map -> odom` TF
publish_imu_tf: true # [usually overwritten by launch file] enable/disable the IMU TF broadcasting
base_frame: "base_link" # usually overwritten by launch file
map_frame: "map"
odometry_frame: "odom"
area_memory_db_path: ""
area_memory: true # Enable to detect loop closure
depth_min_range: 0.0 # Set this value for removing fixed zones of the robot in the FoV of the camerafrom the visual odometry evaluation
set_as_static: false # If 'true' the camera will be static and not move in the environment
set_gravity_as_origin: false # If 'true' align the positional tracking world to imu gravity measurement. Keep the yaw from the user initial pose.
floor_alignment: false # Enable to automatically calculate camera/floor offset
initial_base_pose: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0] # Initial position of the `base_frame` in the map -> [X, Y, Z, R, P, Y]
init_odom_with_first_valid_pose: true # Enable to initialize the odometry with the first valid pose
path_pub_rate: 2.0 # [DYNAMIC] - Camera trajectory publishing frequency
path_max_count: -1 # use '-1' for unlimited path size
two_d_mode: false # Force navigation on a plane. If true the Z value will be fixed to "fixed_z_value", roll and pitch to zero
fixed_z_value: 0.00 # Value to be used for Z coordinate if `two_d_mode` is true
qos_history: 1 # '1': KEEP_LAST - '2': KEEP_ALL
qos_depth: 1 # Queue size if using KEEP_LAST
qos_reliability: 1 # '1': RELIABLE - '2': BEST_EFFORT
qos_durability: 2 # '1': TRANSIENT_LOCAL - '2': VOLATILE
transform_time_offset: 0.0 # The value added to the timestamp of `map->odom` and `odom->base_link`` transform being generated
mapping:
mapping_enabled: false # True to enable mapping and fused point cloud pubblication
resolution: 0.1 # maps resolution in meters [0.01f, 0.2f]
max_mapping_range: 20.0 # maximum depth range while mapping in meters (-1 for automatic calculation) [2.0, 20.0]
fused_pointcloud_freq: 0.5 # frequency of the publishing of the fused colored point cloud
clicked_point_topic: "/clicked_point" # Topic published by Rviz when a point of the cloud is clicked. Used for plane detection
qos_history: 1 # '1': KEEP_LAST - '2': KEEP_ALL
qos_depth: 1 # Queue size if using KEEP_LAST
qos_reliability: 1 # '1': RELIABLE - '2': BEST_EFFORT -
qos_durability: 2 # '1': TRANSIENT_LOCAL - '2': VOLATILE
sensors:
sensors_image_sync: false # Synchronize Sensors messages with latest published video/depth message
sensors_pub_rate: 200. # frequency of publishing of sensors data. MAX: 400. - MIN: grab rate
qos_history: 1 # '1': KEEP_LAST - '2': KEEP_ALL
qos_depth: 1 # Queue size if using KEEP_LAST
qos_reliability: 1 # '1': RELIABLE - '2': BEST_EFFORT -
qos_durability: 2 # '1': TRANSIENT_LOCAL - '2': VOLATILE
object_detection:
od_enabled: false # True to enable Object Detection [only ZED 2]
confidence_threshold: 50.0 # [DYNAMIC] - Minimum value of the detection confidence of an object [0,100]
prediction_timeout: 0.5 # During this time [sec], the object will have OK state even if it is not detected. Set this parameter to 0 to disable SDK predictions
model: 0 # '0': MULTI_CLASS_BOX - '1': MULTI_CLASS_BOX_ACCURATE - '2': HUMAN_BODY_FAST - '3': HUMAN_BODY_ACCURATE - '4': MULTI_CLASS_BOX_MEDIUM - '5': HUMAN_BODY_MEDIUM - '6': PERSON_HEAD_BOX
filtering_mode: 1 # '0': NONE - '1': NMS3D - '2': NMS3D_PER_CLASS
mc_people: true # [DYNAMIC] - Enable/disable the detection of persons for 'MULTI_CLASS_X' models
mc_vehicle: true # [DYNAMIC] - Enable/disable the detection of vehicles for 'MULTI_CLASS_X' models
mc_bag: true # [DYNAMIC] - Enable/disable the detection of bags for 'MULTI_CLASS_X' models
mc_animal: true # [DYNAMIC] - Enable/disable the detection of animals for 'MULTI_CLASS_X' models
mc_electronics: true # [DYNAMIC] - Enable/disable the detection of electronic devices for 'MULTI_CLASS_X' models
mc_fruit_vegetable: true # [DYNAMIC] - Enable/disable the detection of fruits and vegetables for 'MULTI_CLASS_X' models
mc_sport: true # [DYNAMIC] - Enable/disable the detection of sport-related objects for 'MULTI_CLASS_X' models
body_fitting: true # Enable/disable body fitting for 'HUMAN_BODY_FAST' and 'HUMAN_BODY_ACCURATE' models
body_format: 1 # '0': POSE_18 - '1': POSE_34 [Only if `HUMAN_BODY_*` model is selected]
qos_history: 1 # '1': KEEP_LAST - '2': KEEP_ALL
qos_depth: 1 # Queue size if using KEEP_LAST
qos_reliability: 1 # '1': RELIABLE - '2': BEST_EFFORT
qos_durability: 2 # '1': TRANSIENT_LOCAL - '2': VOLATILE
debug:
debug_mode: false
debug_sensors: false
How can I get the following message since both should be the same:
[zed_wrapper-2] [WARN] [1713933239.166988713] [zed2.zed_node]: The publishing resolution (2208x1242) cannot be higher than the grabbing resolution (1920x1080). Using grab resolution for output messages.
Because topics subscription and data communication add underneath processing which slows down the final message receiving.
I recommend you tune the ROS 2 Middleware to obtain the maximum performance for your configuration:
- https://docs.ros.org/en/humble/How-To-Guides/DDS-tuning.html
- https://autowarefoundation.github.io/autoware-documentation/main/installation/additional-settings-for-developers/
- 15.4. Large Data Rates — Fast DDS 2.14.0 documentation
- https://docs.ros.org/en/galactic/Tutorials/Advanced/FastDDS-Configuration.html
Please read the documentation I linked above to obtain more information concerning QoS and the relative tuning.
Are you using the latest version of the wrapper and the latest compatible ZED SDK v4.0.8?