Body tracking: odd lower-limb pose estimation when processing SVO offline

Software
, ,
1

Hi all,

When processing my .svo files offline, I notice that the skeleton’s lower limbs lag behind and tend to move around even when the person’s legs are standing still on the floor. This happens in a simple sit-to-stand task where I would expect fairly stable leg tracking. In contrast, the upper limbs look much better aligned.

See in the link:

  • the .svo file

  • an .avi video showing the skeleton overlay for visualisation.

The key parameters I’m using are:

  • InitParameters:

    • svo_real_time_mode = False

    • coordinate_system = RIGHT_HANDED_Y_UP

    • coordinate_units = METER

    • enable_right_side_measure = True

    • enable_image_enhancement = True

    • depth_mode = NEURAL_PLUS

    • depth_stabilization = 80

  • BodyTrackingParameters:

    • enable_body_fitting = True

    • enable_tracking = True

    • detection_model = HUMAN_BODY_ACCURATE

    • body_format = BODY_34

  • BodyTrackingRuntimeParameters:

    • detection_confidence_threshold = 70

    • skeleton_smoothing = 1

  • PositionalTrackingParameters:

    • set_as_static = True

    • enable_pose_smoothing = True

    • set_floor_as_origin = True

    • set_gravity_as_origin = False

I recorded at 60 Hz, 720p resolution (SDK 4.2.5)

I’ve tried playing with some of the settings (e.g., skeleton smoothing, confidence threshold, depth stabilisation), but I don’t notice any visible difference in the resulting video.

Am I missing any parameter or runtime adjustment that might improve lower-limb fitting for this type of movement? Or is there something in my current configuration that you would recommend changing? Happy to share the code if needed.

Any guidance would be greatly appreciated.

Thanks,
PPM

2

Hi,

thanks for the videos, it’s very helpful. You can remove them from your message if you don’t want them to stay publicly available.

Did you try using the sl::BODY_FORMAT::BODY_38 instead? It should be more precise for this kind of movements.

3

Hi Benjamin,

Thanks for the advice. I tried it today but didn’t get great results (e.g., it detected the chair as a leg). I’ve added the .avi file to the folder for you to check.

Is there anything else I could try to improve the pose estimation with either model? Also, does GPU usage affect this? (e.g., if I get a better GPU, will the estimation improve?)

Cheers,
PPM

4

No, changing your GPU will only increase the frame rate, not the quality.

I re tested it and the issue seems to come from the body fitting. With body model 38 and fitting disabled, the results is significantly better from what I can see (but more jittery).
I’d reduce the confidence a bit as half of the keypoints are not visible in your video, their conf is probably relatively low.

5

This helped a lot, thanks.

Is there any other parameter you reckon is important for Body Tracking? (beside the ones mentioned in the tutorial). For instance, something within Depth Settings (e.g., texture_confidence_threshold)? Also, is there anything critical in SDK 5.0 in this regard?

Cheers,

PPM

6

Mostly, the BodyTrackingParameters and BodyTrackingRuntimeParameters.

The depth quality is high enough that changing the parameter value will have no significant impact, as long as you are using one of the Neural modes.

1 Like