3D Ball tracking in real space

Software Unity
1

I have tried to track a ball and show her a 3D presentation in Unity but tracking seems really slow, it can’t track the ball if it moves fast.
Was using 2D and 3D tracking examples, is there any better way to track ball position in real space?
Thanks

2

Hi @TomiHR, welcome to the forums

I have tried to track a ball and show her a 3D presentation in Unity but tracking seems really slow, it can’t track the ball if it moves fast.

  • When you mean slow, do you mean you lose tracking when it moves fast, or that the 2D/3D box is slow to follow?
  • What model/classes are you using to track the ball?

Was using 2D and 3D tracking examples, is there any better way to track ball position in real space?

Unfortunately, I think the Object Detection samples are the way to go right now for it. I noticed an important latency while testing this sample on my side, we’ll work on reducing it.

Best,
Jean-Loup

3

I am using an Object Detection sample, and yes, there is latency when tracking the ball; the virtual ball is catching up with the actual ball, which is really obvious.
Can you give me some tips on how to maybe reduce it a little bit?

4

@TomiHR
I found out that, unfortunately, the smoothing parameter is not (yet) exposed in the API. I added this to our backlog, and I’ll try to push it, however I have no date to give for when it will be available.

In the meantime, using more fps (so, lowering the resolution + specifying max fps for the mode) should decrease the latency a little. You can also try using MULTI_CLASS_BOX_MEDIUM instead of ACCURATE, but I don’t think you there is anything else for now.

Sorry for the inconvenience, it’s definitely something we will improve.

Best,
Jean-Loup

5

We were testing today, and tracking has problems when the ball is too fast, do you have any tips on what we can do to make it more reliable?
We have a case where the ball is hitting the wall and because of the delay id does not detect contact with the wall in 3D space, 3D ball is returning back much sooner because of smoothing.

6

For now, except for increasing the camera’s FPS, there is not really anything you can do.

Please send us a video and an SVO of what’s happening (either here or to support@stereolabs.com), we should be able to try things faster and hopefully provide some help before the smoothing setting is available.

Sorry for the inconvenience,

Best,
Jean-Loup

7

I have set camera FPS to 60, can it go higher?

8

If you use the VGA resolution, you can go to 100 fps.

9

Here is the video of use, this is just a demo, there will be a led screen in the place of the black cloth.

Smoothing is really aggressive, if you maybe can provide SDK with smoothing turned off?
We have a project that needs to be done by 14. of this month.

10

Hello, is there any solution to this?

11

@TomiHR Hi, sorry I missed this message.
Unfortunately, there is no definite solution for now, as in an exposed smoothing parameter.
Depending on your exact setting (your video is a bit cropped), if you know you will only have one ball, you could disable the object tracking and make a custom script using the object data to position the ball in Unity.
In the same idea, you could implement a sort of tracking by checking each time you receive object detection data which object it most likely corresponds to.

12

How do I do that

In the same idea, you could implement a sort of tracking by checking each time you receive object detection data which object it most likely corresponds to.

13

@TomiHR

Disabling the tracking (and thus, the smoothing)

First step is see what you get when you “just” disable the tracking.
In the ZED_Rig_Mono, go to the “Object Detection” section and:

  • uncheck “Enable Tracking”.
  • if you’re using MULTI_CLASS_BOX model, make sure to only enable the “sport” category, it should detect only the ball.

Then, in the 3D Object Visualizer gameobject, check the “Show OFF Tracked” to display not tracked detected objects.
Then, you can play the scene. The main drawback is that it will only work with 1 object detected, since all objects are tagged with the same ID “-1” and the scripts use that ID.

Making a custom tracking

To do the tracking, the whole concept is to give the right id to a detected object. You would have to create a function called just after this line, where all detected objects are retrieved. You’d have to store a map pairing an ID and a “previous position”, maybe also a velocity to have a threshold for your search, and find the position in the detected objects that matches this the most (or find out that the object is probably hidden or lost, depending on your thresholds), and update this list each frame.