ZED Unity LiveLink avatar not moving like the SDK skeleton

Software Unity
1

Regardless how I tweak the smoothing/IK settings the avatar is not moving like the SDK skeleton. Sometimes are worse than others, but it is never perfect. Check the following video: ZED_Skeleton.mp4 - Google Drive

Additionally, how can I apply “Root motion” to the avatar? The gameobject holding the avatar does not change its position and rotation, only the joint objects changes position and rotation. How can I get the parent gameobject to change position along with the character so the joints matches up? The red spheres should correspond to key joints (knees, elbows etc) on the avatar. I can’t simply set the position to be that of one of the joints as I need the correct rotation as well.

image
image675×788 64.5 KB

2

Hi @haakonflaar ,

Does it do the same with the default avatar available in the plugin?
What versions of the plugin/SDK are you using?
Are you testing with live data or from SVOs files? Can you try with SVOs and send us SVOs and their configuration file to support@stereolabs.com?
If it is not possible and the behavior is different with the default avatars, can you send us this particular avatar?

Your settings seem normal to me in the video, I suspect a rigging issue since the stickman seems ok. :thinking: However, it should work as long as there is an animator with IK, on a Humanoid rig. And it should not work at all in other cases, so…

For the knees gizmos, the behavior is indeed different than with the Unity plugin, where the knee joints move. That’s weird, as the code is nearly the same. I don’t have a solution for this, I’ll log this for investigation.

To simulate root motion, you could get the animator.bodyPosition at the end of the animation, but our testing to directly move the root had shown heavy jitters in some cases iirc, so we decided to not have it our-of-the-box.

3

Hi @JPlou,

It does indeed look like it is due to the avatar we use although it is a humanoid avatar (Mixamo). Are there some requirements for the humanoid avatars? For example, does it need to have only a single mesh, or can it contain multiple meshses of different body parts each with its own skinned mesh renderer?

As for the root motion, I tried moving it to the position of the hip bone which seemed decent, but not nearly perfect.

4

@haakonflaar
When I say “Humanoid”, I speak in terms of rig type in Unity (see this step of the doc). But it must be ok, since your avatar is moving at all.

Apart from being set to “Humanoid”, the IK pass has to be enabled in the animator, but again if it’s not the case you should not see any movement. Having several mesh renderers should not be an issue.

You can also check that the avatar is correctly set up in the Avatar Mapping tab. If imported from Mixamo, there should be no issue, but you have some issues with your animation, so maybe… (you can access it by clicking the “Avatar” value from the animator on your prefab, then “Configure Avatar”.

Also, check that there are not several animators.

5

@JPlou The error is actually happening because I am instantiating the avatar before playing the scene. In ZedBodyTrackingManager I am fetching that gameobject and using that instead of spawning a clone of a prefab. I need to do so as all avatars must be present before the scene is played. I don’t understand why I get the error, however as I am essentially doing the same thing.

ZedBodyTrackingManager

I replace:

handler.Create(avatars[Random.Range(0, avatars.Length)], bodies.body_format);

with

SkeletonHandler handler = ScriptableObject.CreateInstance<SkeletonHandler>();
handler.nurseId = availableNurseIds[0];

  // Find correct game object avatar
  string fullName = "Nurse_" + handler.nurseId;
  Debug.Log("Looking for " + fullName);
  GameObject avatarGameObject = GameObject.Find(fullName);

  if (avatarGameObject == null)
  {
      Debug.LogError("Avatar " + fullName + " not found");
      return;
  }

Vector3 spawnPosition = bodyData.position;
handler.Create(avatarGameObject, bodies.body_format);

SkeletonHandler:

I replace:

  public void Create(GameObject h, sl.BODY_FORMAT body_format)
  {
      humanoid = (GameObject)Instantiate(h, Vector3.zero, Quaternion.identity); // (GameObject)h;

With:

  public void Create(GameObject h, sl.BODY_FORMAT body_format)
  {
      humanoid = (GameObject)h;

For whatever reason, when I am fetching an existing game objects instead of creating a copy of the prefab the joint position and rotation are different from the SDK skeleton (as shown in the video). Any ideas why, and a fix for it? I need to instantiate all game objects before I play the scene.