ZED X grab stuck in FAILURE after Argus / ZED-X daemon failure

matialmar · July 28, 2026, 5:49pm

Hi there!

We are currently developing a perception solution using the ZED X Camera and Orin NX. And currently facing the problems using the zed-ros2-wrapper with sporadic reboots and failures.

The core problem

While a ZED X camera is streaming, if nvargus-daemon and/or zed_x_daemon are restarted (or otherwise disrupted), sl::Camera::grab() enters CAMERA_REBOOTING, then reports a critical Argus reboot failure and transitions to FAILURE. After that point, grab remains in FAILURE indefinitely; the SDK does not recover the session even with async_grab_camera_recovery = true.

The only reliable recovery we found is:

Kill the application / ROS node that holds the camera session
Restart nvargus-daemon + zed_x_daemon
Re-open the camera (sl::Camera::open)

This mechanism is used for both hotplug and unhandled reboots by the SDK.

Environment

Item	Value
Platform	NVIDIA Jetson Orin NX Engineering Reference Developer Kit
Host OS / L4T	Linux 5.15.185-tegra, L4T R36.5.0
ZED Link package	`stereolabs-zedlink-duo` 1.4.2-LI-MAX96712-L4T36.5.0
Camera	ZED X, S/N 46817456
Capture mode	HD1200@30, GMSL
Depth mode	NEURAL LIGHT
ZED SDK	5.4.0 — Build `115927_a75bfaef_3175036`
CUDA (SDK)	12.6
Runtime	Docker image based on L4T 36.4.0 + ZED SDK 5.4.0 (`zed_ros2_l4t_36.4.0_sdk_5.4.0`) with NVIDIA runtime, privileged, `pid: host`
Application	ZED ROS 2 wrapper (Humble), `component_container_isolated`
Host daemons	`nvargus-daemon.service`, `zed_x_daemon.service` (`/usr/sbin/ZEDX_Daemon`)

How to reproduce

Open the ZED X camera and stream continuously (grab() loop / ROS wrapper publishing depth + RGB).
Confirm healthy streaming.
On the host, restart the GMSL/Argus stack while the session is still open:


sudo systemctl restart zed_x_daemon.service

sudo systemctl restart nvargus-daemon.service

Observe grab() status codes and ZED/Argus logs.

Also reproducible in our setup by any hard disruption of Argus while the camera is open (not only an intentional systemctl restart).

4. Observed behavior (timeline from 2026-07-28 test)

Approximate UTC times from container logs:

Time (UTC)	Event
18:54:50	Host: `systemctl restart zed_x_daemon` + `nvargus-daemon`
18:54:54 → 18:55:02	`grab()` returns `CAMERA_REBOOTING` repeatedly
same window	Flood of Argus client errors (see below)
18:55:03	`[ZED-Argus][Critical] Reboot has failed for CAM0 with current state=4`
18:55:03	`grab()` returns `FAILURE`
after	`FAILURE` persists on every subsequent `grab()`; no return to `SUCCESS` without process kill + reopen

4.1 Critical ZED SDK / Argus messages


[ZED][INFO] CAMERA REBOOTING in sl::ERROR_CODE sl::Camera::grab(sl::RuntimeParameters)

(Argus) Error InvalidState: Receive thread is not running cannot send.

(in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)

(Argus) Error InvalidState:

(propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 92)

[ZED][ERROR] [ZED-Argus][Critical] Reboot has failed for CAM0 with current state=4

[ZED][ERROR] FAILURE in sl::ERROR_CODE sl::Camera::grab(sl::RuntimeParameters)

4.2 Host `nvargus-daemon` journal (around re-open after recovery)

After killing the process and restarting daemons, Argus still logged transient open errors before the new session succeeded:


NvPclStateControllerOpen: Failed ImagerGUID 13. (error 0xA000E)

NvPclOpen: PCL Open Failed. Error: 0xf

SCF: Error BadParameter: Sensor could not be opened.

(in src/services/capture/CaptureServiceDeviceSensor.cpp, function getSourceFromGuid(), line 725)

(Argus) Error AlreadyAllocated: Device 0 (of 1) is in use

(in src/api/CameraProviderImpl.cpp, function createCaptureSessionInternal(), line 286)

The camera eventually opened again after a full process restart + daemon restart.

Questions

Is Reboot has failed for CAM0 with current state=4 a known issue on ZED X + L4T 36.5 / SDK 5.4.0? What does state=4 mean?
After this critical reboot failure, is grab() → FAILURE forever until close() (and possibly daemon restart) expected?
What is the recommended production recovery sequence when Argus reports InvalidState: Receive thread is not running mid-stream?
Should applications avoid restarting nvargus-daemon / zed_x_daemon while a session is open, or is the SDK supposed to handle reboots, failures and hotplug?
Is running SDK 5.4.0 inside an L4T 36.4-based container on a host with L4T 36.5 + ZED Link 36.5.0 a supported configuration?
Are there known issues with AlreadyAllocated: Device 0 is in use after a failed reboot / unclean session teardown?
Is there a known solution for this kind of issues?

Myzhar · July 28, 2026, 7:52pm

Hi @matialmar
please upgrade the ZED SDK to v5.4.1 and the ZED X Driver to v1.4.3 because they both introduces improvements to this behavior.

matialmar · July 30, 2026, 4:52pm

Thank you @Myzhar. This version fixes almost all of the problems.

Just to mention, trying to break the docker I stopped the zed_x_daemon and restarted it and it caused an invalid state that could not be restored from restarting the zed_x_daemon along with nvargus daemon

Not even removing the docker, manually unplugging and plugging the ZED X, restarting the daemons and afterwards restarting the docker would restore the previous working state. Just a full reboot did the job.

Although I know this is not intended usage, it may be helpful to improve ZED X robustness.

A quick log in case it is usefull.

[component_container_isolated-2] [ERROR] [1785343391.723567985] [zed.zed_node]: Connection issue detected: CAMERA REBOOTING
[component_container_isolated-2] (Argus) Error FileOperationFailed: Failed socket read: Connection reset by peer (in src/rpc/socket/common/SocketUtils.cpp, function readSocket(), line 79)
[component_container_isolated-2] (Argus) Error FileOperationFailed: Unexpected error in reading socket (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 277)
[component_container_isolated-2] (Argus) Error FileOperationFailed: Receive worker failure, notifying 1 waiting threads (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 350)
[component_container_isolated-2] (Argus) Error InvalidState: Argus client is exiting with 1 outstanding client threads (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 366)
[component_container_isolated-2] (Argus) Error FileOperationFailed: Receiving thread terminated with error (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadWrapper(), line 379)
[component_container_isolated-2] (Argus) Error FileOperationFailed: Client thread received an error from socket (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 145)
[component_container_isolated-2] (Argus) Error FileOperationFailed:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 92)
[component_container_isolated-2] (Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
[component_container_isolated-2] (Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 92)
[component_container_isolated-2] (Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
[component_container_isolated-2] (Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 92)

Myzhar · August 4, 2026, 2:51pm

This is expected because by stopping the zed_x_daemon and restarting it, you’ve “unloaded” and “reloaded” the ZED X Driver, causing unexpected behaviors for the software that was using it.

This is not a recommended operation with GMSL2 cameras.
GMSL2 is not plug-and-play. If you want to unplug and replug the cameras we recommend to turn off the host device and remove the power to the capture card before.