I would like to use the neural depth mode on my Jetson Xavier NX, but it seems to be too compute intensive. I am running ROS1 and attempting to grab at the minimum frame rate of 15FPS, but it can only manage about 7 FPS. I get over-current/throttling warnings and jtop shows my GPU pinned at 99+%. Of everything I have tried, the only thing that seems to make a difference in GPU usage is switching from NEURAL mode to ULTRA, but the depth map quality is not as nice as I would like. I am wondering if it is possible to somehow get a more pared-down/lightweight neural depth model so that it could run faster on my system and hopefully still yield depth maps better than ULTRA mode.
Any suggestions for how to reduce GPU load while still getting decent depth maps would be appreciated. A very high frame rate is not essential (I could live with 7 FPS), but I would like to not be running right at the limit and getting overcurrent and throttling warnings.
We are constantly working on improving the performance of the NEURAL depth engine and ~7 FPS is the maximum framerate available with this device.
We will release improvements with the next releases of the ZED SDK, but you can expect a few more FPS, not something very big.
You can jump into the next generation of NVIDIA Jetson devices, the Jetson Orin Nano can run the NEURAL depth engine at ~15 FPS.
Thanks for that confirmation and clarification. I have indeed been considering a hardware upgrade to one of the Orin models.
That said, just to clarify my previous question, I guess I was wondering if a neural depth model analogous to, say, tiny-Yolo for object detection existed. I suspect the answer is no, but I am curious why not.
Also, using the Zed ROS wrapper, I seem to be limited to a minimum grab frame rate of 15FPS. Is there any way around this? It would be nice to eliminate the âElaboration takes longer than requested by the FPS rateâ warnings, at the very least. It could also be desirable to go to a frame rate lower than 7FPS in order to not be operating at the Xavier NXâs limit. Is there any way to do this?
Edit: one more thing â the Orin Nano can do ~15FPS, but do you know what FPS I could expect with the Orin NX?
With respect to the first point, a web search doesnât yield anything, as the neural depth model selection/optimization appears as a âblack boxâ to users, as far as I can tell. That is, when I run ZED_Diagnostic -nrlo, I donât know what model I am getting and how it has been optimized. All I know is that it is âoptimizedâ for my system, but I really donât know what that means at a functional level. Is it optimized for maximum accuracy on a given GPU? Or speed? Or some combination? It would be nice if it could be optimized for some middle ground, where accuracy is less, but speed is more. I realize maybe this isnât possible (i.e. perhaps less accuracy would mean depth maps worse than in a non-NEURAL mode) â but Iâm just asking the question.
In any case, thanks for all the info you have provided thus far â it is very helpful.
Yes, I thought you wanted to use your own AI network, not a smaller model provided by us.
We currently provide only one AI model for depth map inference, but I cannot exclude that in the future we can provide more models like we do for Object Detection and Body Tracking.