Zed pointcloud with open3d gpu support

Software C++
1

Hi,

I need help with zed and open3d integration for gpu. So far, i manage to convert from sl::MaType::F32_C4 to open3d::geometry::PointCloud format.
However, i would like to know if it possible to map the sl::Mat gpu data directly.
Something along the lines of
cv::cuda::GpuMat depth_image_ocv_gpu = slMat2cvMatGPU(depth_image_zed_gpu); // create an opencv GPU reference of the sl::Mat

If you need a reference of what i had, its in the following link.

2

Hello @skl1g14,

Thanks for the reference!

I think you could call retrieveMeasure with the GPU memory type, and then access your sl::Mat data in which you stored the measure by also passing the GPU memory parameter when accessing the Data.

3

Hi @JPlou,

I did try to initialize an open3d tensor with data pointer obtained from the retreieveMeasure
auto state_pc = ptr_zed->retrieveMeasure(sl_pc, sl::MEASURE::XYZ, sl::MEM::GPU, low_res);

but open3d expect a contiguous data? I am not sure of the underlying memory layout of the sl:Mat in the GPU and I am not familiar with how to profile it with the CUDA debugger.

4

The data is organized like OpenCV and most computer vision libraries, like RGB, the pixel n is red, the next one is green then blue, repeating. It’s similar for XYZ.

The other standard format is planar like all the Reds pixels (image size plane) then all the Green etc. This is not the one used by the ZED SDK.

The CPU and GPU layouts are identical.