Neural Depth model optimization issue

I am attempting to use the neural depth mode on a ZED2 camera on a Windows 11 laptop (hardware details at bottom). I am able to use ZED Diagnostic.exe --nrlo to run the model optimization, but I believe that there is an issue with this process on this computer. The optimization process indicates an estimate of roughly 33 minutes to complete. As it starts, the progress indicator increments slowly, as expected. However, around 11% complete, the optimization process suddenly runs quickly, and finishes the whole process in 3 minutes, 46 seconds. When using neural depth mode, it is clear that something went wrong with optimization. Here is an example from the ZED Depth Viewer:

Processing: depthviewer-poor_neural_model_optimization.png…

There is a strip of good depth resolution near the middle of the images, but outside of that there is wild variance frame to frame. This seems to suggest that the optimization works starting in the center and the rectangle of stable depth reflects that portion of the model that optimizes slowly, then the rest of the depth values are in the area where optimization ran quickly.

I have been using neural mode with this camera on a workstation running Windows 10 and CUDA 11.7 with an RTX 2070 Super without issue - the neural model optimizes normally there.

Is there a way to prevent optimization from running through so quickly over the bulk of the model? Any suggestions for ways to ensure that the model is completely optimized?

Hardware:
Processor: 11th Gen Intel Core i7-11800H @ 2.3GHz (16 CPUs)
Memory: 16 GB
GPU: NVIDIA GeForce RTX 3060 Laptop GPU (6 GB)
Camera: Stereolabs ZED 2

Software:
OS: Windows 11
CUDA: 11.7
Platform: Python 3.10
ZED SDK: 3.8.1

EDIT: In case the image does not finish processing, here is a link to a hosted copy.

Hello,

Although 3min is pretty fast, it not necessary an issue. The timers are based on how long a usual setup would compute, but it actually depends on your GPU. When the process is finished, the progress bar accelerates.
However your depth map seems quite broken. I suggest you clean your models (in ProgramData) ad try to optimize them again. In theory, you should not run anything during the optimization, especially no other gpu app.
Also can you send us a diagnostic report ?

Antoine

Thanks for your response. I have cleaned the models and re-optimized several times, but get the same results every time, with the broken depth map. I have no other GPU apps running; in fact, the only program running (shown on the taskbar) is Windows Explorer. I’m attaching the diagnostic report. Thanks!

ZED_Diagnostic_Results.json (14.0 KB)

More data: on a whim, I copied the optimized model from my workstation and ran the Depth Viewer. Obviously the Depth Viewer did not consider the copied model to be applicable, and it prompted me to optimize the neural model. This time it worked! I am now getting stable depth resolution as expected. I repeated this process twice and it worked both times. As a final test, I removed all optimized models and ran the Depth Viewer, and this optimization worked as well.

My immediate need has been answered, but I’m still curious why the command line method of neural optimization failed in such way.

1 Like

I am having the same issue, works fine on one computer but received the new computer with an Nvidia T1000 GPU and I get stripes and missing depth on the Neural. I delete the files and retry and get different results each time.

My diagnostic report attached. ZED_Diagnostic_Results (13.3 KB)

I can confirm the behavior, as well as the fix described by nomaded. Go to ZED Depth Viewer, click top right on settings, processing, depth/initialisation/mode/neural, apply.

Need to delete the failed file in “C:\ProgramData\Stereolabs\resources”. Very happy with what I am seeing :slight_smile:

2 Likes