Half SBS WebRTC Streaming

Software C++
1

Hi,
I am still a beginner, but I was able to make C++ tutorials and camera controls using Jetson orin NX.

My goal is to make Half SBS (Half size side by side is explained here: ) video with ZED mini, I want to make a Half SBS (Half size side by side) video with ZED mini and distribute it via P2P using WebRTC.

I need Half SBS to view 3D video on my Leia tablet(3D view moniter).

Could you give me some advice on making HalfSBS as a first step?

2

Hi,

You can first follow our tutorials to see how to retrieve the left and right images from the ZED SDK.
You can directly retrieve the images at the half size by setting the correct resolution when you call retrieveImage (https://www.stereolabs.com/docs/api/classsl_1_1Camera.html#a01dce4f0af6f8959a9c974ffaca656b5).

Then, you can create a side-by-side image from both the left and right images.

Jean-Loup MACARIT
R&D Engineer
Stereolabs Support

3

Thank you for your reply.

I was able to create the following code and create a Half SBS.

#include <sl/Camera.hpp>
#include <opencv2/opencv.hpp>

cv::Mat slMat2cvMat(sl::Mat& input) {
int cvType;
switch (input.getDataType()) {
case sl::MAT_TYPE::F32_C1:
cvType = CV_32FC1;
break;
case sl::MAT_TYPE::F32_C2:
cvType = CV_32FC2;
break;
case sl::MAT_TYPE::F32_C3:
cvType = CV_32FC3;
break;
case sl::MAT_TYPE::F32_C4:
cvType = CV_32FC4;
break;
case sl::MAT_TYPE::U8_C1:
cvType = CV_8UC1;
break;
case sl::MAT_TYPE::U8_C2:
cvType = CV_8UC2;
break;
case sl::MAT_TYPE::U8_C3:
cvType = CV_8UC3;
break;
case sl::MAT_TYPE::U8_C4:
cvType = CV_8UC4;
break;
default:
cvType = -1;
break;
}

if (cvType != -1) {
    return cv::Mat(input.getHeight(), input.getWidth(), cvType, input.getPtr<sl::uchar1>(sl::MEM::CPU));
} else {
    return cv::Mat();
}

}

int main(int argc, char **argv) {
sl::Camera zed;

// Configure camera parameters
sl::InitParameters init_params;
init_params.camera_resolution = sl::RESOLUTION::HD1080;
init_params.camera_fps = 30;
init_params.camera_image_flip = false;

// Open the camera
sl::ERROR_CODE zed_open_error = zed.open(init_params);
if (zed_open_error != sl::ERROR_CODE::SUCCESS) {
    std::cout << "Unable to open the camera. Exiting..." << std::endl;
    return EXIT_FAILURE;
}

// Create OpenCV window to display Half SBS image
cv::namedWindow("Half SBS Image", cv::WINDOW_NORMAL);

// Main loop
while (true) {
    if (zed.grab() == sl::ERROR_CODE::SUCCESS) {
        sl::Mat leftImage, rightImage;
        zed.retrieveImage(leftImage, sl::VIEW::LEFT);  // Retrieve left image
        zed.retrieveImage(rightImage, sl::VIEW::RIGHT); // Retrieve right image

        // Convert sl::Mat to cv::Mat
        cv::Mat leftCvMat = slMat2cvMat(leftImage);
        cv::Mat rightCvMat = slMat2cvMat(rightImage);

        // Resize the images to half width
        cv::resize(leftCvMat, leftCvMat, cv::Size(leftCvMat.cols / 2, leftCvMat.rows));
        cv::resize(rightCvMat, rightCvMat, cv::Size(rightCvMat.cols / 2, rightCvMat.rows));

        // Create a Half SBS image by horizontally stacking left and right images
        cv::Mat halfSbsImage(leftCvMat.rows, leftCvMat.cols + rightCvMat.cols, leftCvMat.type());
        cv::Mat leftHalf = halfSbsImage.colRange(0, leftCvMat.cols);
        leftCvMat.copyTo(leftHalf);
        cv::Mat rightHalf = halfSbsImage.colRange(leftCvMat.cols, halfSbsImage.cols);
        rightCvMat.copyTo(rightHalf);

        // Display Half SBS image using OpenCV
        cv::imshow("Half SBS Image", halfSbsImage);

        // Press 'q' to exit the loop
        if (cv::waitKey(10) == 'q') {
            break;
        }
    }
}

// Close the camera
zed.close();

// Close OpenCV window
cv::destroyAllWindows();

return EXIT_SUCCESS;

}

1 Like