How to make body tracking results more stable

Software
#1

Hello,

Recently, we were using the body tracking example, and found that the position and key points results obtained kept shaking, when a person remained static. The camera is fixed somewhere.

We would like to know if there is a parameter setting for the zed camera to make the result more stable, or if we want to use Kalman filter to smooth the result, how should it be adapted, can you give us some advice, thank you!

#2

Hi @jiankors
what version of the ZED SDK are you using?
Please provide more information about your system and the configuration used for the body tracking.

#3

Hi, @Myzhar , thanks for your reply, and our system and configuration are discribed as fllow

OS：Win11
GPU: NVIDIA RTX 5000

the camera was fixed, and the PositionalTrackingParameters.set_as_static was set to true.

init_parameters.camera_resolution = RESOLUTION::HD720;
init_parameters.camera_fps = 15;
init_parameters.depth_mode = DEPTH_MODE::ULTRA;

ObjectDetectionParameters.image_sync = true;
ObjectDetectionParameters.body_format = sl::BODY_FORMAT::POSE_18;
ObjectDetectionParameters.detection_model = DETECTION_MODEL::HUMAN_BODY_ACCURATE;

when we draw the 2d bounding box of the head, the box jitter obviously, just like bounding box detected with Yolo. Shown as https://www.youtube.com/watch?v=vGiHciI-NC0

Thanks

#4

Hi @jiankors
thank you for the information, still missing the ZED SDK version.
Have you tried to set the depth mode to NEURAL?
The RTX 5000 that you are using should have enough power and memory to handle both Object Detection and NEURAL depth simultaneously.

Please also be sure that enable_tracking is TRUE.

#5

Hi @Myzhar

Sorry about the missing ZED SDK version, we used 3.7.5 at the moment. And when we tried to set the depth mode to NEURAL, we got error:-2146697211 twice, then the skeleton in opened cv::imshow window is missing, only video image.

Thanks.

#6

The SDK v3.7.5 has regression issues that we fixed in v3.7.6 released a few days after:

#7

Hi @Myzhar

We have updated the ZED SDK to the latest 3.7.6, and the NEURAL depth mode can work now. The detected skeletons are stable than before, but still have obvious jitter, and the latency increased a little. The camera FPS is 15.

Can we use filter like kalman ourself to stable the position result?
Thanks!

#8

Yes, you can surely another Kalmanf filter level to your code to smooth this behavior.