I am trying to locate a person through face recognition. I then extract the point cloud data and pack it into a list. With the u, v from the center of the face recognition bounding box I am able to get an index value in order to search in the point cloud. However, the obtained coordinates is flipped along the x/y axis. I had a look through the ros coordinate system and I am unsure what I am doing wrong.
I further extracted the whole point cloud data in order to back trace values and compare to what the camera is actually facing. All the coordinates obtained are behind the camera somehow.
ROS uses the X forward, Y left, Z up coordinate system (see REP105), while computer vision normally use X right, Y down, Z forward. So you must perform a coordinate transformation.
Thank you for your prompt reply. Since I am only get the pixels i,j from the 2d image and finding the 3d pointcloud of that same pixel i,j, would the coordinate system matter as I saw multiple posts where you have mentioned that the data should match on each side.
What I am more confused about is why would the data in the pointcloud represent 3d points behind the camera’s field of view? I am visualizing the pointcloud on rviz and it is aligned with my 2d map as well as objects in front of it. However when I print out the point cloud matrices in general, and pick random points, all of them are behind the field of view of the camera, an exact reflection to what is being seen.
Please find attached the script I am using. face_rec.py (2.3 KB)
@Myzhar After remapping my area and tweaking certain parameters, the behavior is different and points are instead only reflected. Please find a screenshot attached. The blue circle is the person being detected, while the green area are points extracted from the pointcloud for that particular u,v.
@nagem07 you are surely using a wrong reference system somewhere in your URDF or your code.
Please remember that a normal camera reference frame is Z Forward X Right Y Down, while ROS World is X Forward Y Left Z Up.
When you map from an image to a 3D world you must always take this into consideration.
