# Zedsrc vs nvarguscamerasrc latency comparison

**URL:** <https://community.stereolabs.com/t/zedsrc-vs-nvarguscamerasrc-latency-comparison/6585>\
**Category:** ZED X\
**Tags:** jetson\
**Created:** [July 18, 2024, 2:17pm UTC](https://community.stereolabs.com/t/zedsrc-vs-nvarguscamerasrc-latency-comparison/6585 "2024-07-18T14:17:05Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![wisteca](https://community.stereolabs.com/letter_avatar_proxy/v4/letter/w/8e8cbc/32.png) [@wisteca](https://community.stereolabs.com/u/wisteca)\
**Post date:** [July 18, 2024, 2:17pm UTC](https://community.stereolabs.com/t/zedsrc-vs-nvarguscamerasrc-latency-comparison/6585/1 "2024-07-18T14:17:05Z")

</div>

Hi all,

We need to achieve the lowest possible latency for our application. The first goal is to send the ZED X streams through a local network and display them on a monitor.

The setup I’m currently using :

- ZED X camera
- ZED Box Orin™ NX 8GB
- Asus ZenScreen (used as ZED Box monitor)
- POE+ Switch
- Personal computer, GTX 1660Ti

For the first two tests I tried to have the simplest possible pipeline by displaying the left stream directly on the screen connected to the ZED Box.

**Test 1**  
nvargus nv3dsink  
pipeline : `gst-launch-1.0 nvarguscamerasrc sensor-id=0 sensor-mode=0 ! "video/x-raw(memory:NVMM),width=1920,height=1200,framerate=60/1,format=NV12" ! nv3dsink`  
latency 5 : 84ms  
latency 4 : 84ms  
latency 3 : 84ms  
latency 2 : 83ms  
latency 1 : 84ms  
latency 0 : 84ms  
mean : 84ms

**Test 2**  
zedsrc nv3dsink  
pipeline : `gst-launch-1.0 zedsrc camera-fps=60 stream-type=0 camera-resolution=2 ! nv3dsink`  
latency 5 : 100ms  
latency 4 : 100ms  
latency 3 : 116ms  
latency 2 : 116ms  
latency 1 : 100ms  
latency 0 : 100ms  
mean : 105.3ms

For the two following tests I’ve sent the stream in H264 on the local network.

**Test 3**  
nvargus udp autovideosink  
pipeline jetson : `gst-launch-1.0 nvarguscamerasrc sensor-id=0 sensor-mode=0 ! "video/x-raw(memory:NVMM),width=1920,height=1200,framerate=60/1,format=NV12" ! nvv4l2h264enc maxperf-enable=true insert-sps-pps=true bitrate=8000000 ! rtph264pay ! udpsink host=10.1.99.255 port=20000 sync=false async=false max-lateness=1000`  
pipeline pc : `gst-launch-1.0 udpsrc port=20000 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! avdec_h264 ! autovideosink`  
latency 4 : 105ms  
latency 3 : 107ms  
latency 2 : 110ms  
latency 1 : 107ms  
latency 0 : 106ms  
mean : 107ms

**Test 4**  
zedsrc udp autovideosink  
pipeline jetson : `gst-launch-1.0 zedsrc camera-fps=60 stream-type=0 camera-resolution=2 ! videoconvert ! "video/x-raw,format=NV12" ! nvvidconv ! nvv4l2h264enc maxperf-enable=true insert-sps-pps=true bitrate=8000000 ! rtph264pay ! udpsink host=10.1.99.255 port=20000 sync=false async=false max-lateness=1000`  
pipeline pc : `gst-launch-1.0 udpsrc port=20000 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! avdec_h264 ! autovideosink`  
latency 6 : 156ms  
latency 5 : 173ms  
latency 4 : 173ms  
latency 3 : 158ms  
latency 2 : 154ms  
latency 1 : 143ms  
latency 0 : 170ms  
mean : 161ms

The last test is by directly streaming using the ZED SDK (camera streaming sample apps). Therefore It is probably less relevant as both streams are sent, unlike the previous tests.

**Test 5**  
zed stream  
jetson : camera sender sample app  
pc : camera receiver sample app  
latency 4 : 233ms  
latency 3 : 236ms  
latency 2 : 234ms  
latency 1 : 233ms  
latency 0 : 202ms  
mean : 227.6ms

To measure the latency, I simply filmed the different screens (all 60Hz) this way :

 ![nvargus udp 0](https://community.stereolabs.com/uploads/default/original/2X/5/5c3778d441487ed9d7f3858c91770b4aa2a1d552.jpeg)

For our application, a latency above 150ms is getting really annoying, the goal is to drive a machine in a FPV like way.

I found a related topic here : [zedsrc low framerate with left+depth & object detection GST pipeline on Jetson Nano · Issue #16 · stereolabs/zed-gstreamer · GitHub](https://github.com/stereolabs/zed-gstreamer/issues/16)  
From my understanding, where the ZED SDK have to convert the video in BGRA format and have it in RAM memory in order to “perform all the elaborations”, the nvarguscamerasrc is directly able to work in the NVMM memory in NV12 format, which accelerates a lot the pipeline execution.

At a first glance, we could use the nvargus plugin. However, in the future we would like to use all the features brought by this wonderful SDK, in order to help the operator during his driving sessions.

Is there a way of getting the best of both worlds ? For example a way of retrieving the raw streams before the ZED SDK performs all the elaborations ?

I hope the following basic diagram could summarize what I mean :

 ![image](https://community.stereolabs.com/uploads/default/original/2X/8/8e1703760db694da21bb3c0be3e913d7de58e653.png)

That wouldn’t be an issue to get depth and other information with 200-300ms latency, the problem is only about the real-time perception of the operator.

Thanks a lot for your time and your work.

---

<div class="post-metadata">

**Author:** ![Myzhar](https://community.stereolabs.com/user_avatar/community.stereolabs.com/myzhar/32/11_2.png) [@Myzhar](https://community.stereolabs.com/u/Myzhar)\
**Post date:** [July 19, 2024, 1:01pm UTC](https://community.stereolabs.com/t/zedsrc-vs-nvarguscamerasrc-latency-comparison/6585/2 "2024-07-19T13:01:46Z")

</div>

Hi @wisteca  
Welcome to the [Stereolabs](https://www.stereolabs.com/) Community.

The ZED engineering team is already working on the ZED SDK and ZED X Driver to reduce the latency of our GMSL2 cameras.

---

<div class="post-metadata">

**Author:** ![wisteca](https://community.stereolabs.com/letter_avatar_proxy/v4/letter/w/8e8cbc/32.png) [@wisteca](https://community.stereolabs.com/u/wisteca)\
**Post date:** [July 19, 2024, 1:52pm UTC](https://community.stereolabs.com/t/zedsrc-vs-nvarguscamerasrc-latency-comparison/6585/3 "2024-07-19T13:52:57Z")

</div>

Hi @Myzhar  
Thanks for your reply. The performance of the ZED SDK is already very impressive with respect to all of the computations that it handles and maybe I expressed myself badly.

I would like to know if there is a way of retrieving the raw stream (in YUY2 format) before it is feed into the ZED SDK, in order to directly send it on the network. I know that this is possible using the open capture driver for the ZED2 and ZED Mini, however the ZED X aren’t supported by this driver.

I know that only one process can read the camera stream at a time, so I did some tests with v4l2loopback to create virtual devices that could be read once by the ZED SDK and once by a gstreamer pipeline for example, but I didn’t succeed for now.

I also struggle to get a working gstreamer pipeline using v4l2src as the input source. Is this plugin only usable with the USB cameras ? I always get an error saying that the sink pad isn’t available on v4l2src0.

Isn’t there a way of faking the /dev/video\* in order to add this extra step allowing me to tee the stream before it is handled by the SDK, or some other way I haven’t thought of ?

I’ve also seen that’s possible to get the streams directly with OpenCV by performing an UVC Capture. Is it also possible to retrieve the depth map this way ? And would it be suitable to achieve low latency ?

Thanks 🙂

---

<div class="post-metadata">

**Author:** ![Myzhar](https://community.stereolabs.com/user_avatar/community.stereolabs.com/myzhar/32/11_2.png) [@Myzhar](https://community.stereolabs.com/u/Myzhar)\
**Post date:** [July 19, 2024, 2:04pm UTC](https://community.stereolabs.com/t/zedsrc-vs-nvarguscamerasrc-latency-comparison/6585/4 "2024-07-19T14:04:43Z")

</div>

> [@wisteca](#):
>
> I would like to know if there is a way of retrieving the raw stream (in YUY2 format) before it is feed into the ZED SDK, in order to directly send it on the network. I know that this is possible using the open capture driver for the ZED2 and ZED Mini, however the ZED X aren’t supported by this driver.

The right way is by using the Argus library. `nvarguscamerasrc` is an easy workaround that you already tried.  
We use the Argus library internally, so using it directly can help you reducing the latency.

> [@wisteca](#):
>
> I also struggle to get a working gstreamer pipeline using v4l2src as the input source. Is this plugin only usable with the USB cameras ?

Yes, `v4l2src` only works with USB3 cameras.

> [@wisteca](#):
>
> I’ve also seen that’s possible to get the streams directly with OpenCV by performing an UVC Capture. Is it also possible to retrieve the depth map this way ? And would it be suitable to achieve low latency ?

This is not possible for two reasons:

- the depth map is generated by the SDK
- UVC is for USB cameras

---

<div class="post-metadata">

**Author:** ![wisteca](https://community.stereolabs.com/letter_avatar_proxy/v4/letter/w/8e8cbc/32.png) [@wisteca](https://community.stereolabs.com/u/wisteca)\
**Post date:** [July 19, 2024, 3:12pm UTC](https://community.stereolabs.com/t/zedsrc-vs-nvarguscamerasrc-latency-comparison/6585/5 "2024-07-19T15:12:11Z")

</div>

Is it possible to switch to the 16GB ZED Box, which can handle two stereo cameras, to use one ZED X for FPV view (nvargus) and another one for depth perception and AI (zedsrc) ? Or is there a technical impediment?

We have a system design meeting next week and I need to be sure about the capabilities of [Stereolabs](https://www.stereolabs.com/) solutions.  
Thanks a lot

---

<div class="post-metadata">

**Author:** ![Myzhar](https://community.stereolabs.com/user_avatar/community.stereolabs.com/myzhar/32/11_2.png) [@Myzhar](https://community.stereolabs.com/u/Myzhar)\
**Post date:** [July 19, 2024, 3:27pm UTC](https://community.stereolabs.com/t/zedsrc-vs-nvarguscamerasrc-latency-comparison/6585/6 "2024-07-19T15:27:09Z")

</div>

> [@wisteca](#):
>
> Is it possible to switch to the 16GB ZED Box, which can handle two stereo cameras, to use one ZED X for FPV view (nvargus) and another one for depth perception and AI (zedsrc) ? Or is there a technical impediment?

Yes, I can’t see anything blocking this type of configuration
