OpenCV Videowriter not able to write Zed stream to RTSP server

I am using a cv2.Videowriter to try and write the Left Camera stream to an RTSP server using python. But It dose not seem to be able to work.

Now I am able to view the left stream from the camera with

cv2.imshow(‘Image’, image_ocv)

But when I use


to write to the RTSP server I get this error

WARN:0] global /home/joev/JEP/script/workspace/opencv-4.5.0/modules/videoio/src/cap_gstreamer.cpp (1631) writeFrame OpenCV | GStreamer warning: cvWriteFrame() needs images with depth = IPL_DEPTH_8U and nChannels = 3.

Here is my out send string

out_send = cv2.VideoWriter(‘appsrc ! videoconvert ! video/x-raw,format=YUY2 ! nvvidconv ! video/x-raw(memory:NVMM),format=(string)NV12,width=1280,height=720 !
nvv4l2h264enc bitrate=12000000 insert-sps-pps=1 idrinterval=1 insert-vui=1 ! video/x-h264
! rtph264pay pt=96 !
udpsink host= port=5400 async=false sync=0’,
cv2.CAP_GSTREAMER, 0, 30, (1280,720), True)

and here is my RTSP factory set launch string

factory.set_launch("(udpsrc name=pay0 port=%d buffer-size=524288 caps=“application/x-rtp, media=video, clock-rate=90000, encoding-name=(string)%s, payload=96 " )” % (updsink_port_num, codec))

I have tried numerous combinations for the
out_send = cv2.VideoWriter string
but nothing seems to get past the error


It seems that your image_ocv has the wrong format. Your missing channels, or you have too many channels. Try to convert your matrix to the right format first.


I agree with you conclusion. Now the trick is to find out what type of matrix the “out.write”
wants to see.
Im curious how cv2.imshow is able to parse the same matrix and show the stream from the zed camera.

Does anybody out there know how the “get data” converts to a NumPy array
to be used with opencv?

The get_data actually does not copy anything, it’s a shared pointer. You can check what it does on github, our wrapper is open source.


I searched in the github repo for “get_data:” from the wrapper to see what it does but was not able find it.
I found code examples using “get_data:” .

Would you be able to point me in a direction in the github repo to find how “get_data:” works
within the wrapper.

To recover data from sl.Mat to use it with opencv, use the get_data() method
# It returns a numpy array that can be used as a matrix with opencv
image_ocv = image_zed.get_data()

Does image_zed.get data() grab a specidic location from sl.Mat?

Hello, the function is here :

You should find everything you need.


Just a bit of info on what I am trying to accomplish.

I have modified the python github ZED-Opencv example so it can retrieve the Zed camera stream. Then get the depth data from any pixel that is clicked on.
Then I am attempting to send the stream back out RTSP to a Nvidia Deepstream YOLO pipeline for object detection.

You can use our streaming sample, it uses RTP. you can then do your object detection on your sl.Mat or CV.Mat. You won’t need to convert the sl.Mat to opencv much later.


So the issue was


wants to see a RGB stream and

image_ocv = image_zed.getdata()

returns a RGBA stream

So the solution is

image_RGB = cv2.cvtColor(image_ocx, cv2.COLOR_RGBA2RGB)

Again thanks for the help.
Once I get it all cleaned up I will post a video with explanation and code.

Glad you could find a solution :slight_smile: