Zed X Mini Does NOT work with OpenCV

Hi, The existing ZED python driver has high GPU utilization even with the depth inference turned off, so we want to switch to OpenCV and use just the left eye as a UVC camera. However, it does not work. The following code gives a green image:

To run:

pip install vuer==0.0.15rc3
# import the opencv library
import asyncio

import cv2
from vuer import Vuer, VuerSession
from vuer.schemas import Scene, ImageBackground
from vuer.serdes import jpg

# define a video capture object
vid = cv2.VideoCapture(-1)
# vid.set(cv2.CAP_PROP_FPS, 120)

app = Vuer(port=8112, uri="ws://localhost:8112")


@app.spawn(start=True)
async def main(session: VuerSession):
    print("session started")
    session.set @ Scene()

    while True:
        try:
            # Capture the video frame
            # by frame
            ret, frame = vid.read()
            if not ret:
                print("failed to read frame")

            frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)

            session.upsert(
                ImageBackground(
                    src=jpg(frame, 95),
                    key="image",
                ),
                to="bgChildren",
            )
            await asyncio.sleep(0.01)

        except KeyboardInterrupt:
            break

    # After the loop release the cap object
    vid.release()

Hi @geyang
the VideoCapture module of OpenCV does not support direct GMSL2 input, so you cannot use it.
You should use NVIDIA’s LibArgus to retrieve the native camera stream:
https://docs.nvidia.com/jetson/l4t-multimedia/group__LibargusAPI.html

Great, I will take a quick look!