I am working on a robotics project and plan to use Isaac ROS and I am looking to purchase cameras that are Argus cameras to be able to use the most effective path for my machine vision on the Jetson Orin Nano, I dont see this on the list of specified cameras but I am asking to not assume. If the Zed x is not hardware configured for Argus to be able to bypass the CPU and go straight to GPU, is it being planned?
The ZED SDK uses Argus to retrieve ZED X data and it allows to directly retrieve the information in GPU memory, without passing from GPU memory.
This is allowed by calling the retrieveImage and retrieveMeasure functions with sl::MEM::GPU as type parameter.
Hello @Myzhar im trying to get the Zedx mini to work with the isaac ros argus package on the jetson orin nx, but it fails to load. you mentioned This is allowed by calling the retrieveImage and retrieveMeasure functions with sl::MEM::GPU as type parameter. Is this an automatic process or do i have to change anything myself for this to work? also should i start the zed wrapper before i start the argus package so the information is available? here is an output from the zed wrapper there is an error about retreiving sensor data could this be an issue?
when i run the argus ros package i get errors about:
Failure to get the CaptureSession interface for the Argus camera component.
Error setting up output streams for the Argus camera component.
GXF (Graph eXecution Framework) errors indicating issues with entity execution, graph interruption, and graph deactivation.
here is the complete output i know this is nvidias docker container but if you could shine some light on how i could go about making it work that would be greatly appreciated:
admin@airbud-desktop:/workspaces/isaac_ros-dev$ ros2 launch isaac_ros_argus_camera isaac_ros_argus_camera_stereo.launch.py
[INFO] [launch]: All log files can be found below /home/admin/.ros/log/2024-03-05-12-07-42-338266-airbud-desktop-34971
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [component_container_mt-1]: process started with pid [34982]
[component_container_mt-1] [INFO] [1709658462.912735992] [argus_stereo_container]: Load Library: /workspaces/isaac_ros-dev/install/isaac_ros_argus_camera/lib/libstereo_node.so
[component_container_mt-1] [INFO] [1709658462.981518580] [argus_stereo_container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::argus::ArgusStereoNode>
[component_container_mt-1] [INFO] [1709658462.981647161] [argus_stereo_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::argus::ArgusStereoNode>
[component_container_mt-1] [INFO] [1709658463.014639202] [NitrosContext]: [NitrosContext] Creating a new shared context
[component_container_mt-1] [INFO] [1709658463.015842319] [argus_stereo]: [NitrosNode] Initializing NitrosNode
[component_container_mt-1] [INFO] [1709658463.017546286] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/std/libgxf_std.so
[component_container_mt-1] [INFO] [1709658463.025411090] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_gxf_helpers.so
[component_container_mt-1] [INFO] [1709658463.029295075] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_sight.so
[component_container_mt-1] [INFO] [1709658463.049072225] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_atlas.so
[component_container_mt-1] [INFO] [1709658463.055526001] [NitrosContext]: [NitrosContext] Loading application: '/opt/ros/humble/share/isaac_ros_nitros/config/type_adapter_nitros_context_graph.yaml'
[component_container_mt-1] [INFO] [1709658463.056851266] [NitrosContext]: [NitrosContext] Initializing application...
[component_container_mt-1] [INFO] [1709658463.069443382] [NitrosContext]: [NitrosContext] Running application...
[component_container_mt-1] 2024-03-05 12:07:43.069 WARN gxf/std/program.cpp@514: No system specified. Nothing to do
[component_container_mt-1] [INFO] [1709658463.084623817] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_argus.so
[component_container_mt-1] [INFO] [1709658463.356994349] [NitrosContext]: [NitrosContext] Loading application: '/workspaces/isaac_ros-dev/install/isaac_ros_argus_camera/share/isaac_ros_argus_camera/config/argus_nitros_context_graph.yaml'
[component_container_mt-1] [INFO] [1709658463.358634090] [NitrosContext]: [NitrosContext] Initializing application...
[component_container_mt-1] [INFO] [1709658463.360496431] [NitrosContext]: [NitrosContext] Running application...
[component_container_mt-1] 2024-03-05 12:07:43.362 WARN gxf/std/program.cpp@514: No system specified. Nothing to do
[component_container_mt-1] [INFO] [1709658463.367384111] [argus_stereo]: [NitrosNode] Starting NitrosNode
[component_container_mt-1] [INFO] [1709658463.371064856] [argus_stereo]: [NitrosNode] Loading built-in preset extension specs
[component_container_mt-1] [INFO] [1709658463.374603131] [argus_stereo]: [NitrosNode] Loading built-in extension specs
[component_container_mt-1] [INFO] [1709658463.374688575] [argus_stereo]: [NitrosNode] Loading preset extension specs
[component_container_mt-1] [INFO] [1709658463.376799501] [argus_stereo]: [NitrosNode] Loading extension specs
[component_container_mt-1] [INFO] [1709658463.380557753] [argus_stereo]: [NitrosNode] Loading generator rules
[component_container_mt-1] [INFO] [1709658463.382813996] [argus_stereo]: [NitrosNode] Loading extensions
[component_container_mt-1] [INFO] [1709658463.384615375] [argus_stereo]: [NitrosContext] Loading extension: gxf/lib/libgxf_message_compositor.so
[component_container_mt-1] [INFO] [1709658463.389912788] [argus_stereo]: [NitrosContext] Loading extension: gxf/lib/cuda/libgxf_cuda.so
[component_container_mt-1] [INFO] [1709658463.393066537] [argus_stereo]: [NitrosContext] Loading extension: gxf/lib/serialization/libgxf_serialization.so
[component_container_mt-1] [INFO] [1709658463.401102292] [argus_stereo]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_messages.so
[component_container_mt-1] [INFO] [1709658463.410429006] [argus_stereo]: [NitrosContext] Loading extension: gxf/lib/multimedia/libgxf_multimedia.so
[component_container_mt-1] [INFO] [1709658463.413364731] [argus_stereo]: [NitrosContext] Loading extension: gxf/lib/image_proc/libgxf_tensorops.so
[component_container_mt-1] [INFO] [1709658463.425612002] [argus_stereo]: [NitrosContext] Loading extension: gxf/lib/image_proc/libgxf_rectify_params_generator.so
[component_container_mt-1] [INFO] [1709658463.433827923] [argus_stereo]: [NitrosContext] Loading extension: gxf/lib/libgxf_timestamp_correlator.so
[component_container_mt-1] [INFO] [1709658463.441178820] [argus_stereo]: [NitrosContext] Loading extension: gxf/lib/utils/libgxf_utils.so
[component_container_mt-1] [INFO] [1709658463.443255185] [argus_stereo]: [NitrosNode] Loading graph to the optimizer
[component_container_mt-1] [INFO] [1709658463.450425531] [argus_stereo]: [NitrosNode] Running optimization
[component_container_mt-1] [INFO] [1709658463.621259204] [argus_stereo]: [NitrosNode] Obtaining graph IO group info from the optimizer
[component_container_mt-1] [INFO] [1709658463.625320123] [argus_stereo]: [NitrosNode] Creating negotiated publishers/subscribers
[component_container_mt-1] [INFO] [1709658463.650198103] [argus_stereo]: [NitrosNode] Starting negotiation...
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/argus_stereo' in container '/argus_stereo_container'
[component_container_mt-1] [INFO] [1709658464.656352774] [argus_stereo]: [NitrosNode] Starting post negotiation setup
[component_container_mt-1] [INFO] [1709658464.656549901] [argus_stereo]: [NitrosNode] Getting data format negotiation results
[component_container_mt-1] [INFO] [1709658464.656586670] [argus_stereo]: [NitrosPublisher] Negotiation ended with no results
[component_container_mt-1] [INFO] [1709658464.656603919] [argus_stereo]: [NitrosPublisher] Use only the compatible publisher: topic_name="/left/image_raw", data_format="nitros_image_rgb8"
[component_container_mt-1] [INFO] [1709658464.656618959] [argus_stereo]: [NitrosPublisher] Negotiation ended with no results
[component_container_mt-1] [INFO] [1709658464.656632848] [argus_stereo]: [NitrosPublisher] Use only the compatible publisher: topic_name="/right/image_raw", data_format="nitros_image_rgb8"
[component_container_mt-1] [INFO] [1709658464.656642256] [argus_stereo]: [NitrosPublisher] Negotiation ended with no results
[component_container_mt-1] [INFO] [1709658464.656649809] [argus_stereo]: [NitrosPublisher] Use only the compatible publisher: topic_name="/left/camerainfo", data_format="nitros_camera_info"
[component_container_mt-1] [INFO] [1709658464.656658641] [argus_stereo]: [NitrosPublisher] Negotiation ended with no results
[component_container_mt-1] [INFO] [1709658464.656666737] [argus_stereo]: [NitrosPublisher] Use only the compatible publisher: topic_name="/right/camerainfo", data_format="nitros_camera_info"
[component_container_mt-1] [INFO] [1709658464.656974045] [argus_stereo]: [NitrosNode] Exporting the final graph based on the negotiation results
[component_container_mt-1] [INFO] [1709658464.670669977] [argus_stereo]: [NitrosNode] Wrote the final top level YAML graph to "/tmp/isaac_ros_nitros/graphs/VKSQBZVGFG/VKSQBZVGFG.yaml"
[component_container_mt-1] [INFO] [1709658464.670769277] [argus_stereo]: [NitrosNode] Calling user's pre-load-graph callback
[component_container_mt-1] [INFO] [1709658464.670785277] [argus_stereo]: [NitrosNode] Loading application
[component_container_mt-1] [INFO] [1709658464.670804254] [argus_stereo]: [NitrosContext] Loading application: '/tmp/isaac_ros_nitros/graphs/VKSQBZVGFG/VKSQBZVGFG.yaml'
[component_container_mt-1] 2024-03-05 12:07:44.678 WARN gxf/std/yaml_file_loader.cpp@1058: Using unregistered parameter 'left_camera_frame' in component ''.
[component_container_mt-1] 2024-03-05 12:07:44.678 WARN gxf/std/yaml_file_loader.cpp@1058: Using unregistered parameter 'right_camera_frame' in component ''.
[component_container_mt-1] 2024-03-05 12:07:44.679 WARN gxf/std/yaml_file_loader.cpp@1058: Using unregistered parameter 'allocator' in component ''.
[component_container_mt-1] [INFO] [1709658464.680416899] [argus_stereo]: [NitrosNode] Linking Nitros pub/sub to the loaded application
[component_container_mt-1] [INFO] [1709658464.680697485] [argus_stereo]: [NitrosNode] Calling user's post-load-graph callback
[component_container_mt-1] [INFO] [1709658464.680726094] [argus_stereo]: [ArgusCameraNode] postLoadGraphCallback().
[component_container_mt-1] [INFO] [1709658464.680795185] [argus_stereo]: [NitrosContext] Initializing application...
[component_container_mt-1] [INFO] [1709658465.087028569] [argus_stereo]: [NitrosContext] Running application...
[component_container_mt-1] [INFO] [1709658465.087519851] [argus_stereo]: [NitrosNode] Starting a heartbeat timer (eid=93)
[component_container_mt-1] 2024-03-05 12:07:45.089 ERROR extensions/hawk/components/argus_camera.cpp@354: Failed to get CaptureSession interface
[component_container_mt-1] 2024-03-05 12:07:45.089 ERROR extensions/hawk/components/argus_camera.cpp@705: Error setting up output streams
[component_container_mt-1] 2024-03-05 12:07:45.089 WARN gxf/std/multi_thread_scheduler.cpp@325: Error while executing entity E32 named 'VKSQBZVGFG_argus_camera': GXF_FAILURE
[component_container_mt-1] 2024-03-05 12:07:45.090 ERROR gxf/std/entity_executor.cpp@563: Entity [VKSQBZVGFG_argus_camera] must be in Started, Tick Pending, Ticking or Idle stage before stopping. Current state is StartPending
[component_container_mt-1] 2024-03-05 12:07:46.087 ERROR gxf/std/entity_executor.cpp@210: Entity with eid 93 not found!
[component_container_mt-1] [WARN] [1709658466.087949615] [argus_stereo]: [NitrosNode] The heartbeat entity (eid=93) was stopped. The graph may have been terminated.
[component_container_mt-1] [INFO] [1709658466.094404062] [argus_stereo]: [NitrosNode] Terminating the running application
[component_container_mt-1] [INFO] [1709658466.094464000] [argus_stereo]: [NitrosContext] Interrupting GXF...
[component_container_mt-1] 2024-03-05 12:07:46.094 ERROR gxf/core/runtime.cpp@1400: Graph interrupt failed with error: GXF_FAILURE
[component_container_mt-1] [ERROR] [1709658466.094595557] [argus_stereo]: [NitrosContext] GxfGraphInterrupt Error: GXF_FAILURE
[component_container_mt-1] [INFO] [1709658466.094602981] [argus_stereo]: [NitrosContext] Waiting on GXF...
[component_container_mt-1] 2024-03-05 12:07:46.094 ERROR gxf/std/program.cpp@556: wait failed. Deactivating...
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Job Statistics Report (regular) |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Name | Count | Time (Median - 90% - Max) [ms] | Load (%) | Exec(ms) | Variation (Median - 90% - Max) [ns] |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Entity Statistics Report (regular) |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Entity Name | Entity State | Count | Time (Median - 90% - Max) [ms] |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] 2024-03-05 12:07:46.126 WARN gxf/std/entity_warden.cpp@433: Component of type nvidia::gxf::MultiThreadScheduler, cid 124 failed to deinitialize with code GXF_FAILURE
[component_container_mt-1] 2024-03-05 12:07:46.126 ERROR gxf/core/runtime.cpp@731: Could not deinitialize entity 'VKSQBZVGFG_UEBLGUEHXG' (E122): GXF_FAILURE
[component_container_mt-1] 2024-03-05 12:07:46.127 ERROR gxf/std/program.cpp@558: Deactivation failed.
[component_container_mt-1] 2024-03-05 12:07:46.127 ERROR gxf/core/runtime.cpp@1408: Graph wait failed with error: GXF_FAILURE
[component_container_mt-1] [ERROR] [1709658466.127022040] [argus_stereo]: [NitrosContext] GxfGraphWait Error: GXF_FAILURE
[component_container_mt-1] [INFO] [1709658466.127039353] [argus_stereo]: [NitrosNode] Application termination done
[INFO] [component_container_mt-1]: process has finished cleanly [pid 34982]
admin@airbud-desktop:/workspaces/isaac_ros-dev$