ZED X camera publishes frozen (stale) image frames indefinitely — requires node restart to recover

Software
1

Product: ZED X (GMSL2)

Setup:

  • 2x ZED X cameras connected via GMSL2

  • NVIDIA Jetson AGX Orin (Syslogic carrier board)

  • ZED SDK 5.1.0

  • ZED ROS 2 Wrapper v5.1.0 (zed-ros2-wrapper, multi-camera launch)

  • ROS 2 Humble

  • Running inside Docker container

  • Grab resolution: HD1080 at 5 FPS

  • Depth mode: NEURAL

  • depth_stabilization: 30

  • enable_image_validity_check: 0

Problem description:

After running for a variable period of time, one of the ZED X cameras begins publishing the same image frame repeatedly on the image topic. The image content is frozen (identical pixel data), but the following continue to function normally:

  1. The publish rate on the image topic remains at the expected frequency — messages keep arriving on schedule.

  2. The ROS message header timestamps continue to advance with each published message (they are not stuck at the old frame’s timestamp).

  3. The camera_info topic continues publishing normally.

  4. Object detection (obj_det/objects) continues publishing with updating timestamps. but publishes empty list even though there are objects (humans) in front of the camera

  5. No errors or anomalies appear in the ROS diagnostics.

The frozen state persists indefinitely — it does not self-recover. The only way to restore live image data is to restart the ZED wrapper node, after which the camera works normally again until the issue recurs. The issue can affect either camera while the other continues to operate normally.

Key observations:

  • The issue is intermittent and has occurred multiple times.

  • The cameras are mounted on heavy construction equipment (BOMAG roller), which is subject to significant vibration during operation.

Questions:

  1. Is there a known condition in the ZED SDK where grab() and retrieveImage() return SUCCESS with advancing timestamps, but the actual image data is stale/repeated?

  2. Does enable_image_validity_check detect this condition, or does it only cover pixel-level corruption?

  3. Is there a recommended way to detect this state from the application side (e.g., a Mat-level timestamp or frame counter that would remain unchanged when the image data hasn’t actually been refreshed)?

  4. Is there a known interaction between GMSL2 link degradation (e.g., from vibration or connector issues) and this type of silent frame staleness?

  5. Are there any SDK-level recovery mechanisms (short of full camera re-open) that could restore the live image stream when this occurs?

2

Hi,

Any thoughts on this ?

@Myzhar

3

Hi @mohamedfawzy96

What you described is not a common behavior. I’ve read about it for the first time, I will check with the team if other users ever reported something similar.

No, it doesn’t because the described condition is not common and we have not added checks for something similar.

Normally, if a camera gets stuck, the timestamp does not change, and the anomaly is detected thanks to this. In your case, the timestamp is updated, hence only an image processing analysis can detect it.

Unfortunately, the ZED ROS2 Wrapper does not support calling the reboot function at runtime. Hence, stopping the node to close and re-open the camera is the only available solution.

In any case, the latest ZED SDK v5.2.3 with the latest ZED X Driver v1.4.1 provides improved connection stability; I recommend you upgrade your system to verify if the problem persists with them.

4

Thanks @Myzhar

Unfortunately, upgrading is not an option for us at the moment — we are using a Syslogic ECU, and the latest ZED X driver available for it appears to be v1.3.

One important detail I forgot to mention in my original post: I was primarily observing the compressed image topic (/zed/zed_node/left/image_rect_color/compressed), as that is the topic we record for offline analysis. You mentioned that normally when a camera gets stuck, the timestamp stops advancing, which allows the anomaly to be detected. Could you confirm whether the same applies to the compressed topic i.e., if the underlying image is invalid or stale, would the compressed topic’s timestamp also remain unchanged?