I belong to an ecology research group and am interested in 3d object tracking insects for behavioral experiments.
What can be adjusted to solve the problem of depth sensing of the bounding box?
(upper left) left scene of the experiment taken with zed2i.
(upper right) depth of the point cloud
(lower) the result of depth sensing of the bounding box
The experiment is conducted in a dark room and there is lighting in the back.
The relative position and angle of the camera, acrylic chamber, and lighting are fixed for each experiment.
The acrylic chamber has a horizontal width almost parallel to the left camera. The standard is a rectangular parallelepiped of 20Ă—30Ă—40 cm.
Our insects are 10 in an acrylic chamber for each experiment and recorded for two hours.
In this way, 3d object tracking is performed in the image taken to use sequential positional information for analysis.
We use darknet_zed.py in zed_python_samples of zed-yolo to obtain x,y, and z information on the insect’s boudning box.
It was confirmed that the bounding box could be well created.
However, a problem occurred in the process of sensing z (depth) using the bounding box and depth map.
If you look at the point cloud image, the depth of the right side and the left side is very different. (There is also a problem that the depth difference of the same position is large for each image.)
Therefore, the actual depth of the binding box is also smaller on the left than on the right, and the location and depth of the insect do not match on the right.
Solving this problem is very important for our research.
What filming environment & analysis method changes or programming can solve this problem?
We are thinking of methods such as using lighting without flicking, using acrylic cage without reflection, and adjusting the position of the light source.
The main options are as follows.
-
Filming Options:
init.camera_resolution = sl.RESOLUTION.HD2K
init.depth_mode = sl.DEPTH_MODE.NEURAL
recording_param = sl.RecordingParameters(path_output, sl.SVO_COMPRESSION_MODE.H264) -
Analysis Options
init.coordinate_units = sl.UNIT.METER
runtime.sensing_mode = sl.SENSING_MODE.FILL
cam.retrieve_image(mat, sl.VIEW.LEFT)
cam.retrieve_measure(point_cloud_mat, sl.MEASURE.XYZRGBA)
Thank you very much for reading the long article.
