Object Detection issue with Ros2

Hello zed community
I am working on Jetson Orin. I am running zed_wrapper zed2.launch.py inside docker container.
I want to run object detection, I don’t see /zed/zed_node/obj_det topic when runnning the above command.
I changed the object_detection.od_enabled to true in config.yaml and can see it is true on console output but still don’t see object_detection topic published.

Lastly I ran service call for enable_object_detection.

I face this issue
ros2 service call /zed/zed_node/enable_obj_det std_srvs/srv/SetBool ‘{data: true}’
requester: making request: std_srvs.srv.SetBool_Request(data=True)

std_srvs.srv.SetBool_Response(success=False, message=‘Error occurred starting Object Detection. See log for more info’)

I am not sure what is causing it.

Looking forward to be helped on this.

Hi @Shubham_23
welcome to the StereoLabs community.

The first time you start the object detection module in a Docker image the ROS 2 Wrapper must download and optimize the AI model, so until this task is completed you cannot see any topic published.

I recommend following this guide to add persistence to the Docker image, otherwise, the optimization task must be performed each time the image is started:

Is it possible that I record the zed node (in rosbag) while in docker and I have the AI models already downloaded on my local system and then run the data (Rosbag) and get the object detection results?

This is not possible. The rosbag cannot be used as an input for the ZED ROS 2 Wrapper.
You can instead record SVO files and use them as an input for the wrapper.

Thanks for replying with that information. That saves a ton of time.
While recording SVO file, I am not able to record & ran into the error:

[zed_wrapper-2] [ERROR] [1705265044.295613586] [zed.zed_node]: Error saving frame to SVO

Full info Message:

     [zed_wrapper-2] [INFO] [1705263566.490740691] [zed.zed_node]: ** Start SVO Recording service called **
    [zed_wrapper-2] [INFO] [1705265044.295321868] [zed.zed_node]: SVO Recording started: 
    [zed_wrapper-2] [INFO] [1705265044.295464303] [zed.zed_node]:  * Bitrate: 0
    [zed_wrapper-2] [INFO] [1705265044.295506288] [zed.zed_node]:  * Compression: LOSSLESS
    [zed_wrapper-2] [INFO] [1705265044.295531216] [zed.zed_node]:  * Framerate: 0
    [zed_wrapper-2] [INFO] [1705265044.295549265] [zed.zed_node]:  * Input Transcode: FALSE
    [zed_wrapper-2] [INFO] [1705265044.295564657] [zed.zed_node]:  * Filename: zed.svo
    [zed_wrapper-2] [ERROR] [1705265044.295613586] [zed.zed_node]: Error saving frame to SVO

I am running it on Jetson Orin, creating my docker container on it. Inside it I was running zed_wrapper and recording svo.

Are you using a Jetson Orin Nano?

I am using Jetson Orin Nano

I recommend not running RVIZ2 on a Jetson because it’s a high demanding software and can make the overall system unstable.
Can you verify if the crash happens even if you launch RVIZ2 on another PC on the same network?
You can install only the zed-ros2-examples repository on the other PC and use this command to start RVIZ2 without starting the ZED node:

ros2 launch zed_display_rviz2 display_zed_cam.launch.py camera_model:=zed2i start_zed_node:=False

on the Jetson you can start the node without visualization:

ros2 launch zed_wrapper zed_camera.launch.pi camera_model:=zed2i

As per your recommendation I am trying to start object detection on zed2 (which is inside docker) running on Jetson Orin Nano.

I am using:

ros2 service call /zed/zed_node/enable_obj_det std_srvs/srv/SetBool “{data: true}”

requester: making request: std_srvs.srv.SetBool_Request(data=True)

std_srvs.srv.SetBool_Response(success=False, message='Error occurred starting Object Detection. See log for more info')

This is the console output of

ros2 launch zed_wrapper zed_camera.launch.py camera_model:zed2

[zed_wrapper-2] [INFO] [1706203681.389394168] [zed.zed_node]: ** Enable Object Detection service called **
[zed_wrapper-2] [INFO] [1706203681.389516571] [zed.zed_node]: Starting Object Detection

I don’t see /zed/zed_node/obj_det topic being published

The first time that you start the Object Detection module with a new AI model the host device must download and optimize it.
On the Jetson Orin Nano this task can require many minutes.

I recommend you manually optimize the AI models that you want to use manually before running the ROS node:

Hi @Myzhar

Wanted to run something through you.
I was previously using

 wget -q --no-check-certificate -O ZED_SDK_Linux_JP.run \
  https://download.stereolabs.com/zedsdk/4.0/l4t35.4/jetsons && \
  chmod +x ZED_SDK_Linux_JP.run ; ./ZED_SDK_Linux_JP.run silent skip_tools && \
  rm -rf /usr/local/zed/resources/* && \
  rm -rf ZED_SDK_Linux_JP.run && \
  rm -rf /var/lib/apt/lists/*


  1. To get the AI models, I think I should remove skip_tools and rm -rf /usr/local/zed/resources/* to get all the models while building zed docker image. Can you elaborate more if this seems the correct path?

  2. Under the Volumes section. Can you elaborate a how to go about mounting Object Detection AI module if I plan on downloading and optimizing only one AI model manually?

It’s correct concerning rm -rf /usr/local/zed/resources/*, not completely correct concerning skip_tools.
I do not recommend downloading and optimizing the AI model inside a Docker Image.
The tools will make the docker image size bigger.
You can use the ZED_Diagnostic commands to download and optimize the AI model on the host and put the relative files in the shared folder that will be visible from inside the running docker image.

You must simply put the files of the downloaded model and the optimized model in the shared folder on the host.
For example, on my machine, for Object Detection Medium, the files are

Please note that if you optimize a model on an host, you cannot move it to another host if it does not have the same identical hardware and software configuration (OS, ZED SDK, CUDA, GPU model)

While using the ZED_Diagnostic -ais

I ran into

ZED_Diagnostic: error while loading shared libraries: /usr/lib/aarch64-linux-gnu/tegra/libnvbuf_utils.so.1.0.0: file too short

Can you point out what is causing this and solution to it?

Please reinstall the ZED SDK being sure that the AI modules are correctly installed.

Hi @Myzhar

I am switching gears and trying to take a different approach.

My Question:
Since there is no zed-ros docker file for R35 Version 4.1 jetpack. I found your docker file

and using zed-ros2-wrapper_l4t35_1_humble_16-add-point-cloud-transport-support tag.

However, when running ros2 launch zed_wrapper zed_camera.launch.py camera_model:=zed2 I am facing:

[zed_wrapper-2] /root/ros2_ws/install/zed_wrapper/lib/zed_wrapper/zed_wrapper: error while loading shared libraries: libnvbuf_utils.so.1.0.0: cannot open shared object file: No such file or directory
[ERROR] [zed_wrapper-2]: process has died [pid 112253, exit code 127, cmd '/root/ros2_ws/install/zed_wrapper/lib/zed_wrapper/zed_wrapper --ros-args -r __node:=zed_node -r __ns:=/zed --params-file /root/ros2_ws/install/zed_wrapper/share/zed_wrapper/config/common.yaml --params-file /root/ros2_ws/install/zed_wrapper/share/zed_wrapper/config/zed2.yaml --params-file /tmp/launch_params_art38blr'].

Do you know what is causing this?

  1. In order to record .svo file I am using zed docker image for R35 V 4.1 and installed the ZED_API for python using python3 get_python_api.py. Everything installed properly and when running
    python3 zed_svo_recording.py --output_svo_file zed.svo
    Based on: https://github.com/stereolabs/zed-sdk/blob/master/recording/recording/mono/python/svo_recording.py
    I am getting
No EGL Display 
nvbufsurftransform: Could not get EGL display connection
[ERROR] (/builds/sl/ZEDKit/lib/src/sl_input/mmapi/R354/NvV4l2Element.cpp:63) <enc_16375> Could not open device '/dev/nvhost-msenc': Cannot allocate memory
[ZED][Encoding] Critical Error : No Video Enc [enc_16375]
Segmentation fault (core dumped)
  1. In my previous approach using my own docker file and running ros2 launch zed_wrapper zed_camera.launch.py camera_model:=zed2. Despite changing common.yaml parameter object_detection.od_enabled=true and linking AI module to zed_resources. I don’t see /zed/zed_node/obj_det/ published.

Do you what can be the reason?

This is a Docker image used for testing and we cannot guarantee that it works as expected.
I do not recommend using it.

Are you in headless mode?
did you launch the Docker container with all the required options?

Please post the log of the node, I suspect that it’s stuck while downloading and optimizing the AI model. See this.