NEURAL depth mode FPS drop

jlrk · August 28, 2025, 12:50am

I am running the ZED SDK v5.0.2 on a ZED Box Orin NX 16GB 1TB (running @MAXN, jetson_clocks enabled) and I am noticing a significant drop in fps when using the NEURAL depth mode vs the PERFORMANCE depth mode. I would stick with PERFORMANCE but there is a warning that PERFORMANCE mode is deprecated.

I am streaming two ZED X Ones as a virtual stereo camera at 960x600 at 60fps.

For just a loop where the images are grabbed, changing depth mode to NEURAL from PERFORMANCE drops fps from 41 to 23.

For object detection using a custom YOLOLIKE detector, changing depth mode to NEURAL from PERFORMANCE drops fps from 32 to 20.

Is this type of fps drop expected? And is there some documentation I can read to determine the pros and cons of the depth modes?

Myzhar · August 28, 2025, 10:21am

Hi @jlrk

I recommend you use the latest ZED SDK v5.0.5 for improved performance.

The expected FPS for a single stereo camera on a ZED Box Orin NX 16GB is 30 when using NEURAL and NEURAL_LIGHT depth modes.

Please be sure that the YOLO detector that you are using is optimized for the GPU of the Jetson Orin NX module.

Are you using the detector as a standalone inference engine, or are you using our ONNX model integration?

Please read here for more information.

jlrk · August 28, 2025, 3:56pm

Thank you for providing me with the expected FPS of 30 for a ZED One X stereo camera. That is helpful. At 23 FPS with no object detection, I am definitely missing something. Is that FPS with zed.grab() and zed.read()? In my test, I am still grabbing the image, just not running object detection.

For object detection, I am using the ONNX integration, so I expect the SDK is taking care of optimization. Much of my detection code is following the python/yolov5-v6-v8_onnx_internal example with GUI disabled. Amazingly, adding this object detection only drops FPS from 23 to 20.

As for updating the SDK: I don’t see anything on the changelog for SDK v5.0.5, 5.0.4, or 5.0.3 that suggests there were changes to processing speeds. What performance improvements can I expect with an upgrade?

Myzhar · August 28, 2025, 4:43pm

Can you please check with ZED Depth Viewer to confirm that you can obtain depth only at 23 FPS?

The information is available in the bottom left of the status bar.

jlrk · September 4, 2025, 6:35pm

In NEUTRAL depth mode, ZED_Depth_Viewer reports DEPTH: 25HZ (note, this is with a SVGA@60FPS stream on an Orin NX 16GB running at MAXN with jetson_clocks).

If depth mode is switched to NEUTRAL LIGHT the value changes to DEPTH: 35HZ

This is the stdout when ZED_Depth_Viewer is run:

~$ ZED_Depth_Viewer 
[2025-09-04 18:33:07 UTC][ZED][INFO] Logging level INFO
[ZED][Streaming] Switching to SHM-boost backend input
[Streaming] Warning : receiving port 34000 is not available (already used)... switching to port 34002. Retrying...
[Streaming] No backward compatibility required.
Opening in BLOCKING MODE 
NvMMLiteOpen : Block : BlockType = 279 
NvMMLiteBlockCreate : Block : BlockType = 279 
[2025-09-04 18:33:07 UTC][ZED][INFO] [Init]  Depth mode: NEURAL
[2025-09-04 18:33:07 UTC][ZED][INFO] [Init]  Serial Number: S/N 106640498
[2025-09-04 18:33:12 UTC][ZED][WARNING] CAMERA NOT INITIALIZED in sl::ERROR_CODE sl::Camera::grab(sl::RuntimeParameters)
[2025-09-04 18:33:16 UTC][ZED][WARNING] CAMERA NOT INITIALIZED in sl::ERROR_CODE sl::Camera::grab(sl::RuntimeParameters)
[2025-09-04 18:33:20 UTC][ZED][WARNING] CAMERA NOT INITIALIZED in sl::ERROR_CODE sl::Camera::grab(sl::RuntimeParameters)
[2025-09-04 18:33:24 UTC][ZED][WARNING] CAMERA NOT INITIALIZED in sl::ERROR_CODE sl::Camera::grab(sl::RuntimeParameters)
[2025-09-04 18:33:29 UTC][ZED][WARNING] CAMERA NOT INITIALIZED in sl::ERROR_CODE sl::Camera::grab(sl::RuntimeParameters)
[2025-09-04 18:33:33 UTC][ZED][WARNING] CAMERA NOT INITIALIZED in sl::ERROR_CODE sl::Camera::grab(sl::RuntimeParameters)
[2025-09-04 18:33:37 UTC][ZED][WARNING] CAMERA NOT INITIALIZED in sl::ERROR_CODE sl::Camera::grab(sl::RuntimeParameters)
[2025-09-04 18:33:41 UTC][ZED][WARNING] CAMERA NOT INITIALIZED in sl::ERROR_CODE sl::Camera::grab(sl::RuntimeParameters)
[2025-09-04 18:33:45 UTC][ZED][WARNING] CAMERA NOT INITIALIZED in sl::ERROR_CODE sl::Camera::grab(sl::RuntimeParameters)
[2025-09-04 18:33:49 UTC][ZED][WARNING] CAMERA NOT INITIALIZED in sl::ERROR_CODE sl::Camera::grab(sl::RuntimeParameters)
[2025-09-04 18:33:53 UTC][ZED][WARNING] CAMERA NOT INITIALIZED in sl::ERROR_CODE sl::Camera::grab(sl::RuntimeParameters)
(Argus) Error EndOfFile: Unexpected error in reading socket (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 277)
(Argus) Error EndOfFile: Receiving thread terminated with error (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadWrapper(), line 379)
[2025-09-04 18:33:57 UTC][ZED][WARNING] CAMERA NOT INITIALIZED in sl::ERROR_CODE sl::Camera::grab(sl::RuntimeParameters)

Myzhar · September 5, 2025, 1:22pm

This is not expected. Please use jtop to confirm that you are working in MAXN power mode with jetson_clocks.sh running.

jlrk · September 8, 2025, 6:28pm

Yep, jtop shows Jetson Clocks: running and NV Power[0]: MAXN

Myzhar · September 9, 2025, 9:55am

Do you have other applications running on the Jetson?

jlrk · September 9, 2025, 10:20pm

The only program running is Zed Media Server serving a virtual stereo camera made up of two Zed X One cameras.

Myzhar · September 10, 2025, 10:13am

I recommend you upgrade the ZED SDK to the latest v5.0.5

jlrk · September 11, 2025, 6:09pm

Ok I’m now on latest SDK (v5.0.6) and camera driver (v1.3.1). Same number for NEURAL depth mode - 25 Hz. NEURAL LIGHT is 36 Hz. This means there is no change in performance. I used jtop to confirm jetson_clocks is running and the power mode is MAXN.

stdout:

$ ZED_Depth_Viewer 
[2025-09-11 18:02:10 UTC][ZED][INFO] Logging level INFO
[ZED][Streaming] Switching to SHM-boost backend input
[Streaming] Warning : receiving port 34000 is not available (already used)... switching to port 34002. Retrying...
[Streaming] No backward compatibility required.
Opening in BLOCKING MODE 
NvMMLiteOpen : Block : BlockType = 279 
NvMMLiteBlockCreate : Block : BlockType = 279 
[2025-09-11 18:02:10 UTC][ZED][INFO] [Init]  Depth mode: NEURAL
[2025-09-11 18:02:10 UTC][ZED][INFO] [Init]  Serial Number: S/N 106640498
[2025-09-11 18:02:15 UTC][ZED][WARNING] CAMERA NOT INITIALIZED in sl::ERROR_CODE sl::Camera::grab(sl::RuntimeParameters)
[2025-09-11 18:02:19 UTC][ZED][WARNING] CAMERA NOT INITIALIZED in sl::ERROR_CODE sl::Camera::grab(sl::RuntimeParameters)
[2025-09-11 18:02:23 UTC][ZED][WARNING] CAMERA NOT INITIALIZED in sl::ERROR_CODE sl::Camera::grab(sl::RuntimeParameters)
[2025-09-11 18:02:27 UTC][ZED][WARNING] CAMERA NOT INITIALIZED in sl::ERROR_CODE sl::Camera::grab(sl::RuntimeParameters)
[2025-09-11 18:02:31 UTC][ZED][WARNING] CAMERA NOT INITIALIZED in sl::ERROR_CODE sl::Camera::grab(sl::RuntimeParameters)
[2025-09-11 18:02:35 UTC][ZED][WARNING] CAMERA NOT INITIALIZED in sl::ERROR_CODE sl::Camera::grab(sl::RuntimeParameters)
[2025-09-11 18:02:39 UTC][ZED][WARNING] CAMERA NOT INITIALIZED in sl::ERROR_CODE sl::Camera::grab(sl::RuntimeParameters)
[2025-09-11 18:02:44 UTC][ZED][WARNING] CAMERA NOT INITIALIZED in sl::ERROR_CODE sl::Camera::grab(sl::RuntimeParameters)
[2025-09-11 18:02:48 UTC][ZED][WARNING] CAMERA NOT INITIALIZED in sl::ERROR_CODE sl::Camera::grab(sl::RuntimeParameters)
[2025-09-11 18:02:52 UTC][ZED][WARNING] CAMERA NOT INITIALIZED in sl::ERROR_CODE sl::Camera::grab(sl::RuntimeParameters)
[2025-09-11 18:02:56 UTC][ZED][WARNING] CAMERA NOT INITIALIZED in sl::ERROR_CODE sl::Camera::grab(sl::RuntimeParameters)
(Argus) Error EndOfFile: Unexpected error in reading socket (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 277)
(Argus) Error EndOfFile: Receiving thread terminated with error (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadWrapper(), line 379)
[2025-09-11 18:03:00 UTC][ZED][WARNING] CAMERA NOT INITIALIZED in sl::ERROR_CODE sl::Camera::grab(sl::RuntimeParameters)
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)

Myzhar · September 12, 2025, 7:25am

What IP address are you using when opening the virtual camera stream?

jlrk · September 12, 2025, 8:26pm

IP address: 127.0.0.1

Myzhar · September 16, 2025, 9:26am

Hi @jlrk
The problem has been reported to the team, and we suspect that ZED Media Server is the cause of the FPS drop.
We are working on improving this pipeline.