SLAM and regions exclusions

Software
1

Hi

Is there a way to exclude some pixels or rectangle areas frome the stereo frames prior to SLAM processing?
E.g. exclude moving objects related areas in order to make the point cloud of the static world more robust?

2

Hi Tetsujinfr,
this is not an available feature in the ZED SDK.

However, dynamic objects should already be removed by the process itself, the only thing you can adjust is the depth confidence.

3

The dynamic objects removal of the SDK is decent from experience, but there are 2 scenarios where I observe (understandably) limitations:

  1. When a person is quasi static and then moves, the SLAM is building a static mesh where the person was (e.g. when you talk to a person, it would get acquired as static), so I would like to tell the SLAM algo “please ignore this area since it is not a static environment”
  2. When looking at an moving object , say a car, through a linear speed rotation of the camera, the car appears static relative to the overall world and the SLAM does capture the car as static and start translating the overall world at the car speed

Do not get me wrong, the SDK algo is quite impresssive, but those 2 scenarios limitations are significant robustness issues for my project.

Any chance that a future SDK update can accept those exclusion areas rectangles coord which would be provided by the user? Or something like that?

If you are planning for a deep learning augmented algo I would take that of course, but please note that I use a ZED Mini, which as I understand does not benefit from the work you are doing on the ZED2 for object detection etc.

4

Thank you for your suggestions. We will consider them for future releases, if compatible with our algorithms.

5

Those guys from SLAMcore have the dynamic object removal (they say), leveraging neuralnet segmenetation plus rgbd. I guess it is pretty slow and it is not clear which piece is executed realtime and which is not, plus the removal is not perfect (human shoes not removed), but it illustrate well my points above.

At 2’:22"

(The heightmap function is pretty cool too btw)

6

A paper on dynamic object removal for robust SLAM algo:

Any plan on your side to propose to feed the SDK with a set of object box coordinates or segmentation mask, so that it can take those info, fuse them with the camera tracking to predict obj positions until new detection is provided, and remove those areas from the scene for SLAM execution?
Thanks