Hello, I want to use zed2i camera to do a reverse perspective projection transformation, get the top view, and generate the depth map in the top view. Can I use the library that comes with zed to do this? What do I need to do?
The installation position of the camera is not overlooking the ground, but has a included Angle. I need to use the program to make the camera’s Angle look down at the ground, and output the top-down image and depth map.
I want to transform to get the top view and get the height of things on the ground (assuming there is an obstacle on the ground).
What is the most convenient and effective way, thank you very much！
We currently do not have methods to perform what you wish to do out of the box.
The main steps that you have to perform are mainly coordinate system transforms (matrix transforms):
First, determine the pitch of the camera using the positional tracking module from the ZED SDK. This will be located in the WORLD transform of the camera: zed.getPosition(pose, sl::REFERENCE_FRAME::WORLD)
Create a “virtual camera” that is placed top-down above your scene and calculate its transform by rotating the actual camera with the pitch angle you’ve determined
Transform all of the points of the point cloud with this virtual camera transform, so that they are in the “virtual camera space”
You now have to project the points in camera space to a 2D plane defined by the virtual camera intrinsic parameters (we can use the same parameters as the real camera for e.g.). Some points will be projected to the same (x, y) values in the image, so you can set the value at the maximum world height as you wish to have the elevation values from the ground.
With this method, you can create the 2D reprojections of both the RGB image and the depth values.
I can recommend this documentation about Model View Projection transforms here: Model View Projection
In your case you already have the world coordinates so the View and Projection matrices are the ones I’ve mentioned previously
