Hi,
I am using a ZED 2 camera with the ROS2 humble wrapper on Nvidia Jetson Orin NX (arm64) with JP 6.0/ Ubuntu 22.04
I have a ROS2 node that subscribes to the left camera image and depth topics. I have an object detector through which I find the center co-ordinate of the bounding box and I try to query the depth value of that center pixel in the depth image. I have noticed that no matter what depth mode I chose, I can’t get the depth values beyond 10 m. After that, its either nan or inf.
Below is the relevant code where it queries for the depth from the depth image topic
def depth_image_callback(self, msg):
"""
Callback function to process a 32FC1 depth image message.
Parameters:
msg (sensor_msgs.msg.Image): The depth image message.
"""
# Extract metadata
width = msg.width
height = msg.height
step = msg.step
is_bigendian = msg.is_bigendian
data = msg.data
# The depth image is a single-channel float32 image
# the values is the distance in mm in z axis
cv_image = self.bridge.imgmsg_to_cv2(msg, "32FC1")
# Convert the depth image to a Numpy array since most cv2 functions
# require Numpy arrays.
cv_image_array = np.array(cv_image, dtype = np.float32)
self.depth_image_query = cv_image_array
I then query the depth at that pixel using self.depth_image_query[centery, centerx]
It returns me either a number or np/.nan or np.inf
These are the parameters from clearpath’s robot.yaml that essentially calls the zed ros2 wrapper
sensors:
camera:
- model: stereolabs_zed
urdf_enabled: true
launch_enabled: true
parent: base_link
xyz: [0.0, 0.0, 0.0]
rpy: [0.0, 0.0, 0.0]
ros_parameters:
stereolabs_zed:
general.grab_frame_rate: 60 #30 earlier
general.serial_number: 0
general.camera_model: 'zed2'
general.grab_resolution: 'HD720'
depth.depth_mode: 'ULTRA' #'NONE', 'PERFORMANCE', 'QUALITY', 'ULTRA', 'NEURAL', 'NEURAL_PLUS'
depth.depth_stabilization: 0
pos_tracking.pos_tracking_enabled: False
video.brightness: 2
video.contrast: 4
I haven’t changed anything inside the zed2.yaml
and common.yaml
that comes with the wrapper.
FYI, this is the content of zed2.yaml
/**:
ros__parameters:
general:
camera_model: 'zed2'
camera_name: 'zed2' # usually overwritten by launch file
grab_resolution: 'HD720' # The native camera grab resolution. 'HD2K', 'HD1080', 'HD720', 'VGA', 'AUTO'
grab_frame_rate: 30 # ZED SDK internal grabbing rate
depth:
min_depth: 0.3 # Min: 0.3, Max: 3.0
max_depth: 15.0 # Max: 40.0
Any idea what’s happening here?