How to convert zed camera pixel coordinates (u, v) to world coordinates (X, Y, Z)?

When I installed the zed api, there was a file called SN845234.conf in /usr/local/zed/setting on my linux system, Can I use the parameters inside to achieve pixel coordinates (u, v) to world coordinates (X, Y, Z)?

There is a problem, in the/usr/local/zed/settingSN845234 conf file, what do you mean the following parameters are said: [STEREO]
CV_2K = 0.00623976
CV_FHD = 0.00623976
CV_HD = 0.00623976
CV_VGA = 0.00623976
RX_2K = 0.000916611
RX_FHD = 0.000916611
RX_HD = 0.000916611
RX_VGA = 0.000916611
RZ_2K = 0.000770041
RZ_FHD = 0.000770041
RZ_HD = 0.000770041
RZ_VGA = 0.000770041
How do we get the rotation matrix R with these parameters? Please help me show my thanks!

Hi @zhangYQHBAU
please refer to this support page for an easy solution by using ZED SDK API.
Do not use the information of the conf file.

I am very confused that why “Do not use the information of the conf file” . The parameters of fx,fy in the .conf whether it is can be used to accomplish the pixel coordinates (u, v) to world coordinates (X, Y, Z)?

The parameters in the conf file are good for the raw unrectified images, while the ZED SDK already provides rectified images and the parameters are “tuned” when the camera is opened to fix eventual little changes introduced by temperature or little shocks/vibrations.

The information that you are searching for is correctly provided by the ZED SDK at runtime, see left_cam, right_cam, and stereo_transform

1 Like

You means that the conf file will be used in the zed sdk to rectify the raw unrectified images, while the output images for the zed camera are all rectified images that don’t need the conf file?

Yes, that’s correct as far as you use the retrieveImage API function with the sl::VIEW::LEFT or sl::VIEW::RIGHT parameters.
You can also retrieve unrectified images with sl::VIEW::LEFT_UNRECTIFIED or sl::VIEW::RIGHT_UNRECTIFIED , in this case, you should not use the information in the conf file, but retrieve the camera parameters at runtime

In stereolab document with these content:

What are the formulas to get the 3D coordinates of an image pixel?

Given the 2D coordinates (u,v) of a pixel we can calculate the 3D coordinates (X,Y,Z) in the image frame (Z forward, X right, Y down) by using the following formulas:

Z = depth value of the pixel (u,v) from the depth map
X = (( u - c_x) * Z) / (f_x)
Y = (( v - c_y) * Z) / (f_y)

but i used the function
zed.retrieve_measure(point_cloud, l.MEASURE.XYZRGBA, sl.MEM.CPU)
s, point_cloud_value = point_cloud.get_value(x_coord, y_coord) to calculated the coordinate x y z. I wonder if the x, y and z coordinates from the (u, v) coordinates are the same in these two ways?

Hi @zhangYQHBAU
yes, they are the same

In the ros system I use,

import pyzed.sl

when editing the program in the ros environment,it shows:
Traceback (most recent call last):
File “/home/young/my_yolo/src/yolov7-ros/src/detect_ros.py”, line 24, in
import pyzed.sl as sl
ModuleNotFoundError: No module named ‘pyzed’

Please tell me how to configurein ros environment so that we can use the import pyzed.sl as sl module . thank you.

import pyzed.sl
it can be used successfully in my python environment,but can not used in ros environment,I wonder if we should configure the environment in cmakelist.txt

What is the accuracy of using deep point clouds in ROS? What is the error? Where can I find this data? Especially with Zed_ ROS_ Please help me answer these questions when using the wrapper package for deep point cloud computing. Thank you

This is not a ROS specification but pertains to your specific camera model. The depth accuracy information for each ZED camera can be found in its respective datasheet.