Handling the Depth of Partially Occluded Objects

Software Python
1

Hi there,

I’m wondering if there is a recommended way to handle depths of partially-occluded objects.

I am having issues with scenarios that resemble the following:

Suppose there is a tall cardboard box ~3m away from a Zed X camera.
When a person ~5m away from the camera walks across the frame, they are detected at ~5m, but as they pass behind the box, for a brief moment their depth reads ~3m.

I have numerous down-pipeline attempts at recognizing when this situation is happening, but I am wondering if there is a recommended way (or ways) that people have overcome these false depth readings.

Thanks.

2

Hi @stripedapus
Is it possible to record a video or take screenshots to show the problem while appearing?

3

Hi @Myzhar

Thanks for your response. Unfortunately I am not able to share the videos publicly.

We could either start a private conversation or keep it more hypothetical so hopefully other people can benefit as well.

This is a fairly well-known issue with any depth problem, so I’m wondering if ZED itself takes actions to try to mitigate this and if there is any well-known indicators that a person-detection (sl.ObjectData) could be occluded.

Some thoughts off the top of my head:

  • Occlusion could somehow be inferred from a detection’s dimensions or 3D bounding_box size (perhaps they are very “long” depth-wise)
  • Perhaps the position_covariance could somehow be used
  • We could use the depth confidence map to disregard poor readings downstream.

Could any of these or other values be used as hints that occlusion could be happening?

Thanks.

4

You can send videos that you cannot share publicly to support@stereolabs.com with a link to this discussion.

We will continue on that channel.