Hi there,
I’m using ZED2 for body tacking, and need to use the data to animate 3D characters in the browser.
I wonder if there are any library/standards that I could follow? Or any suggestion on how that could be done?
Beside, is BODY_38 an open standard, or is it something specific to the ZED SDK?
Thanks a lot!
Hi,
There is no universal standard for skeleton rigging to my knowledge but our skeleton is very close to what can be done by Mixamo (https://www.mixamo.com/).
Stereolabs Support
Hi, I’m trying to use the local_orientation_per_joint array to pose Mixamo 3D models in three.js.
At first I think it would be straight-forward, by comparing BODY_38 and Mixamo bone structure, and mapping IDs to bone names accordingly. This approach works for one model (but need to flip the rotation quaternion to [-x, y, -z, w]), but failed in different ways for other models. And all those models share the same bone hierarchy.
We tried to build our own rigs, but are very confused what works and what’s not. Some of our exports have twisted joints; and sometime the meshes are twisted terribly.
Btw we have tried all the Coordinate Systems and “RIGHT_HANDED_Y_UP” seems to be the least broken one for us.
Perhaps it is more of an 3D application issue than a ZED issue, but would someone here be able to provide some insight? We thought it might be related to bone rolling or difference in 3D world coordinate, but there’s so many variables we don’t know where to look into.
Thanks in advance!
Yes, I think you should use the RIGHT_HANDED_Y_UP coordinate system in the SDK as it is the one used by three.js.
I don’t know how you are applying the rotations to the joints but those are local rotations which means this is the rotation of a joint relative to its parent. So, make sure your implementation expects this type of data.
For debugging purposes, you can apply “fake” rotations to your model, in a very specific, known position and see what is the final result.
For example, if you set all the rotations to zero, is the model is T pose?
Stereolabs Support
Thanks @BenjaminV
By ‘setting all rotations to zero’, do you mean setting the rotation quaternion of all joints to (0,0,0,1)? I just tried that and they ended up like this.
(And while the first model looks okay after zeroing the rotations, The model is in fact mirrored on both X and Z axis)
So I guess I have to modify the models so that they are in T-pose when joint rotations are set to (0,0,0,1) - are there any sample character models that we could compare ours against?
All those rigs works fine in Blender and C4D, and we are lost what we could do to make them compatible when ZED output.
I think you should take a look at our Unity plugin (https://github.com/stereolabs/zed-unity). We are animating all these models correctly.
Especially this method : https://github.com/stereolabs/zed-unity/blob/master/ZEDCamera/Assets/ZED/SDK/Helpers/Scripts/BodyTracking/SkeletonHandler.cs#L843.
Stereolabs Support
Thanks @BenjaminV . Let me look into it.
Btw I’m very new to 3D, and wanna confirm if the followings are right:
I’m just not sure how I should overcome this challenge.
Should I rig a “correct” model (does such model exist)?
or is the script you provided aim to solve this issue?
Thanks!
What you are doing should be working only for models that have rotations at (0,0,0,1) when in T-pose position.
For the other models, you have to take into account the default rotations of each joint. That is what we do in the MoveAvatar() method in the Unity plugin (https://github.com/stereolabs/zed-unity/blob/master/ZEDCamera/Assets/ZED/SDK/Helpers/Scripts/BodyTracking/SkeletonHandler.cs#L843).
Stereolabs Support
