This is a weird one, but I’ve noticed it consistently. Working in SDK 4.2, Unity 2022.3 with Body Tracking HUMAN_BODY_ACCURATE / BODY_38 / FULL, I can be in a straight-on H-pose at acquisition and the tracked skeleton’s arm will be angled straight from shoulder to wrist with a locked elbow. I turn my arm(s) so my elbows are still 90 degrees but my hands face the camera, and suddenly the skeleton begins tracking my elbows correctly, and going back to the straight-on H-pose is fine (modulo the usual imprecision with joint placement). I would think that a stereoscopic inference system would be able to get adequate info from a clear 2D right-angle, so it’s odd (buggy?) that it can’t parse accurately until it sees the bone straight-on…
(Ignore all the fancy masking, alignment, and avatar fitting stuff I’ve added to the SkeletonHandler; the core skeleton generation is still SDK-vanilla, it’s just easier to show it 1:1 with an isolated human subject)
Hi,
I think the issue is not coming from the detection itself but to the postprocessing we apply to it called body_fitting.
Unfortunately, the current implementation of the body fitting can create this kind of errors where the arms are “stuck” in certain positions.
The general documentation seems to imply that “fitting” is a 3rd tier of performance. Is there a way to toggle it within the Unity SDK as a test, or is it integral to the Body Tracking Module as provided?
After twiddling several settings without success, it does (provisionally) seem like dropping the tracking module to HUMAN_BODY_MEDIUM precludes this joint lock. Is that consistent with what the tech is doing under the hood?
The fitting is necessary to have access to the joint rotations, otherwise only the positions are available. That’s why you can’t disable it in Unity.
I don’t think so, it’s probably a coincidence and this will not be consistent.
A few days in, it continues to appear that MEDIUM has <25% the prevalence of arm-lock from ACCURATE. Not ideal, but hopefully good enough to go to production.
