I have a problem finding the position of a 3D object. I’ve tried taking an image from topic ~/left/image_rect_color, then detecting the object using yolov8 and looking for the center point of the bbox. then once I know the pixel coordinates, I take the depth in topic ~/depth/depth_registered, based on those coordinates and get the position of the 3d object. however I don’t get accurate 3d coordinates. I tried to display the pointcloud in rviz from the zed ros2 wrapper and display the 3d object detection results but the object position does not match the pointcloud.
Do I need to calibrate or something? My current code just takes a picture of the topic, then does detection, gets pixel coordinates, and looks for the 3D position using topic depth.
did I miss something? do I need to process the right and left images first and then find the pixel coordinates? or something else?