Can't get depth more than 10 m

Software ROS2
,
1

Hi,

I am using a ZED 2 camera with the ROS2 humble wrapper on Nvidia Jetson Orin NX (arm64) with JP 6.0/ Ubuntu 22.04

I have a ROS2 node that subscribes to the left camera image and depth topics. I have an object detector through which I find the center co-ordinate of the bounding box and I try to query the depth value of that center pixel in the depth image. I have noticed that no matter what depth mode I chose, I can’t get the depth values beyond 10 m. After that, its either nan or inf.

Below is the relevant code where it queries for the depth from the depth image topic

def depth_image_callback(self, msg):
        """
        Callback function to process a 32FC1 depth image message.

        Parameters:
            msg (sensor_msgs.msg.Image): The depth image message.
        """
        # Extract metadata
        width = msg.width
        height = msg.height
        step = msg.step
        is_bigendian = msg.is_bigendian
        data = msg.data

        # The depth image is a single-channel float32 image
        # the values is the distance in mm in z axis
        cv_image = self.bridge.imgmsg_to_cv2(msg, "32FC1")
        # Convert the depth image to a Numpy array since most cv2 functions
        # require Numpy arrays.
        cv_image_array = np.array(cv_image, dtype = np.float32)
        self.depth_image_query = cv_image_array

I then query the depth at that pixel using self.depth_image_query[centery, centerx] It returns me either a number or np/.nan or np.inf

These are the parameters from clearpath’s robot.yaml that essentially calls the zed ros2 wrapper

sensors:
  camera:
    - model: stereolabs_zed
      urdf_enabled: true
      launch_enabled: true
      parent: base_link
      xyz: [0.0, 0.0, 0.0]
      rpy: [0.0, 0.0, 0.0]
      ros_parameters:
        stereolabs_zed:
          general.grab_frame_rate: 60 #30 earlier
          general.serial_number: 0
          general.camera_model: 'zed2'
          general.grab_resolution: 'HD720'
          depth.depth_mode: 'ULTRA' #'NONE', 'PERFORMANCE', 'QUALITY', 'ULTRA', 'NEURAL', 'NEURAL_PLUS'
          depth.depth_stabilization: 0
          pos_tracking.pos_tracking_enabled: False
          video.brightness: 2
          video.contrast: 4

I haven’t changed anything inside the zed2.yaml and common.yaml that comes with the wrapper.

FYI, this is the content of zed2.yaml

/**:
    ros__parameters:
        general:            
            camera_model: 'zed2'
            camera_name: 'zed2' # usually overwritten by launch file
            grab_resolution: 'HD720' # The native camera grab resolution. 'HD2K', 'HD1080', 'HD720', 'VGA', 'AUTO'
            grab_frame_rate: 30 # ZED SDK internal grabbing rate

        depth:
            min_depth: 0.3 # Min: 0.3, Max: 3.0
            max_depth: 15.0 # Max: 40.0

Any idea what’s happening here?

2

Hi @you_know_who

can you please check that ZED Depth Viewer works as expected for longer distances?

3

Hi @Myzhar
Yes, I tested using the ZED Depth Viewer, clicked on a pixel (essentially me), moved away, and it worked as expected for longer distances. Btw, where do we specify the depth mode in the viewer? I couldn’t find that option.

4

In this case it’s possible that you are using a different configuration in ROS.
Please compare the depth settings.

Furthermore, when getting the depth value for a single point, we recommend to perform a median averages of the surrounding depth values filtering out not valid inf and nan values.

5

Is that also how the ZED SDK/ Depyh viewer calculates/ shows the depth values?

6

Yes, you cannot expect that all the depth map values are valid for many possible reasons: occlusions, light, saturations, reflexes, etc

7

I tried to debug further by checking the depth values by hovering the mouse and found that it doesn’t exist after ~10m. Also, a lot of area in the sky too reads the depth values that seems incorrect to me.

image
image716×431 70.4 KB