I’m using a ZED2i on a UR15 with hand–eye calibration (OpenCV reprojection error < 0.1 px, 2.5 cm checkerboard).
I’m detecting AprilTags (Isaac ROS) and trying to position the robot at the center of the marker (~70 cm distance). The distance estimation is mostly fine, but I consistently see an offset of ~2 cm in the y-axis (sometimes also x-axis).
Is this expected accuracy at that distance, or does it indicate an issue?
I also notice some pose instability (marker “flashing”), so I’m averaging multiple detections even with 8cm marker size of April tag.
Any tips to improve lateral accuracy? What precision can I realistically expect from the ZED2i in this setup?
What is the reference system? Can you add a diagram?
If the offset is stable, then it’s possible that you are using the wrong reference points with your calibration setup.
I attached everything in this document. My base frame is ur15_base and my goal was to calibrate the wpg_tcp to the left optical frame. Therefore I placed it in the tree like this directly connected with the TCP
Do I also have to calibrate the Camera itself? It has been done at your factory I read and therefore I did not change anything.
How accurate is the Camera normaly with Marker and the pointcloud assuming a distance of 80cm?
This is not required. The factory calibration is always more precise than any manual calibration.
Manual calibration is required only if the optics setup changes.
Regarding the offset… is it possible that you are performing the hand-eye calibration with respect to the optical frame, while considering the camera base link as the origin?
I looked it up and changed it to optical frame. But that frame is 90 degree tilted or?
joint name=“zed2i_calibration_joint” type=“fixed”
parent link=“wpg_tcp”/
child link=“zed2i_left_camera_frame_optical”/
origin xyz=“${cam_x} ${cam_y} ${cam_z}” rpy=“${cam_roll} ${cam_pitch} ${cam_yaw}”/
/joint
Which camera Frame should I use for calibration? But now i saw when I put there optical frame the camera tree is broken.
Is it intended to be calibrated to the camera link? I often faced there a angle error and therefore went to the optical frame
Yes, this is expected. I meant the left camera frame, not specifically the optical frame which is normally used to perform computation in “OpenCV-like” reference frame.
This depends on your setup and how you perform the processing.
Yes I normally now have used the optical frame for extracting Informations.But when the camera frame is calibrated automatically the optical frame also has to be right calibrated too when I understand the internal coordinate systems right. So I can keep the rest as it is and only calibrate the camera frame I think. The otherer internal transformations are known.
Ok for openCV I thought it was also optical but due to the mountage I will calibrate it with reference to the camera frame.
I calibrated it on that frame and wanted to drive to that position of the marker but still there is a distance shift of over 10mm in the y axis. It also depends witht the execution how good the depth estimation is. Do you have a hint there? The april tag is the one I did from the april tag tutorial
What’s the size of the april tag?
If you use a biggee tag you can get better calibration results?
Have precisely configured the calibration software with the measure of the tag?
My calibration grid is a board with 3cm x 3cm sizes. The tag itself is 8cm x 8cm great. How much accurancy can I expect from the zed2i? What would you suggest? I took from moveit the calibration method and the better one from OpenCV with SolvePnP
I also already tested a checkerboard with 5cmx5cm board size. But not really better. The point cloud is not better. What do you suggest for configuration. I take neural mode and HD720 resolution. IMU is nesseccarry or what to concigure the best in ros2? Range will be 20cm till 1,50m I guess
Yes I measured it it is totally 8cm. Ok do you have hints how it can be totally flat? I reprint it and put it on that bar there. I choosed the H720 mode for jetson. I am testing on the isaac ros on my ubuntu 22.04 and there the image came directly out of the camera. The mode I enabled was NEURAL and I also tested NEURAL_PLUS. Do you have a hint what to impove of settings further? I reprint now the Tag with 1200dpi and try pushing it flat on the shield
Normally, the distance returned by the tag detectors does not depend on the selected depth mode… unless I have not understood how you are retrieving the distance from the target.
I use the isaac ros AprilTag tutorial of you. There you get a pose estimation of the Tag. That I average with 90 images because the point cloud of the camera is not 100% flat.
In this case, I confirm that you are not using the Depth information to retrieve the distance from the AprilTag, but pure visual information because the AprilTage Isaac ROS node only use the left camera images to detect the tag.
I recommend you read the related documentation and ask on the NVIDIA developer forum to understand how to improve the performance of the detection.
I mean what do you mean by the depth the tag distance is already determined by the april tag and the libaries where nvidia refers to are mostly only for jazzy not humble made and isaac ros 4.4. As work around I can only take the zed depth information on the pixel of the intersection of x, y, z frame of the tag?