I’m developing an application where I need to detect when an object hits a wall, and for that my plan was getting the depth texture from the Zed2i and compare it with previous depth textures to know when that happens.
But I’m getting a lot of noise between different frames, and its making this more difficult than I expected. I didn’t expect this behaviour as I thought that in an static environment, frames would be exactly the same, but as you can see in the video, is not the case.
This is the configuration I had when I recorded the video
This kind of depth quality in “ULTRA” depth mode is not unimaginable, it depends on the scene itself. We will gladly take a deeper look if you send an SVO recording either here or to support@stereolabs.com.
It mostly depends on the lighting, and if the surfaces are very uniform, which will give depth some difficulty.
Can you see if you get better results with NEURAL? Maybe in a lower resolution?
I recorded another video, with I think a bit more of natural light, so you can check both.
As soon as I publish this post, I’ll send an email with both files, with the subject Depth Noise on Zed2i [Hardware] [ZED2] [JPlou] [R2R0]
As you’ll see in the video, the surface is pretty uniform, just a white wall with some forniture around.
Neural depth mode retrieves results that are so much better than Ultra, but I’m a bit worried about how much that can impact the performance in the game, as I would like to run it in the cheapest PC I can find.
Changing resolutions doesn’t change it too much luckily, as I need HD1080 because it has exactly the FOV I need for my project.
The main issue is the very uniform surface, this is precisely what stereo algorithms struggle with.
NEURAL can deal with it to an extent, but as you say, it comes with a performance cost.
I can think of 3 things from your SVOs:
Don’t have a good part of the FoV occupied by a close, unmoving object/surface (your table or desk, I suppose). It impedes the depth quite a lot.
Add features to the wall to break the uniformity, I can think of colored painter’s tape, for example. This should help ULTRA a fair bit.
Keep the room lit, as the depth is vision-based, the more well-lit the environment is, the better it will work.
That makes sense, I thought a uniform surface would help.As soon as I can I’ll try using a tripod to avoid having the desk in the camera view and I’ll try the colored painter’s tape trick too, as I think that would be something we can have in the final implementation.
I can try improving the lightning too, but I don’t know if I have room for that. Do you know if led bulbs flickering can mess with the algorithm?
So, I couldn’t wait more and I tried what you suggested and now the results are way way better in Neural depth mode.
Sadly, in Depth Mode Ultra there is still a lot of noise. This was recorded under led bulbs lightning at night, so maybe the results are better during daylight.
Update: I tried iluminating the wall using a projector and the results were the same
Unfortunately, this looks like the limits of ULTRA.
I recommend you test the other depth modes too, but if you need stability, you will most probably need NEURAL, or to find a workaround specific to your app. (Off the top of my head, maybe our Plane Detection sample can inspire you?)
Well, after more testing it seems it may need Neural as you said, I’ll try it as soon as I can in a real environment and also do the proper performance testing to check the kind of hardware we would need.
My idea in this test was detecting a ball hitting some targets set in the wall, so I don’t know if the Plane Detection sample could help, my plan b was using the Custom Detector , but it seems it requieres a bit more of time and learning a technology I’m not familiarised to, so I’ll save tha card on my sleeve at the moment.
There are more depth testing I wanted to do, so maybe I’ll write another post soon with some questions I have if I’m not able to get what I’m looking for
