how to get the keypoint 2D and 3D of persons, to spawn an object on that keypoint?
Hi,
The data is stored in a sl.BodyData structure. You can look at how we access it in our body tracking sample, here : https://github.com/stereolabs/zed-unity/blob/master/ZEDCamera/Assets/ZED/SDK/Helpers/Scripts/BodyTracking/ZEDBodyTrackingManager.cs#L341
When you subscribe to the OnBodyTracking event (https://github.com/stereolabs/zed-unity/blob/master/ZEDCamera/Assets/ZED/SDK/Helpers/Scripts/BodyTracking/ZEDBodyTrackingManager.cs#L181), you can have access to all the data from that frame as a BodyTrackingFrame which contains a list of BodyData.
thanks - reading through all this now
Hi there,
I think it’s almost at the location, where it’s suppose to be spawning but not quite.
The necklace needs to be facing the opposite direction and it needs to be sitting on node 4.
The necklace is only suppose to be spawning once per human body detected.
Based on the image and explanation, would you be able to guide me…
public void Start(){
if (spawned == false)
{
var zedManager = ZEDManager.GetInstance(ZED_CAMERA_ID.CAMERA_ID_01);
zedManager.OnBodyTracking += OnBodyTracking;
}
}
private void OnBodyTracking(BodyTrackingFrame bodyFrame)
{
bodyFrame.detectedBodies.ForEach(body =>
{
spawned = true;
var neckPosition = body.rawBodyData.localPositionPerJoint[(int)BODY_38_PARTS.NECK];
var neckRotation = body.rawBodyData.localOrientationPerJoint[(int)BODY_38_PARTS.NECK];
Instantiate(necklace, neckPosition, neckRotation);}
});
}
}
Can you try :
var neckPosition = zedManager.GetZedRootTansform().TransformPoint(body.rawBodyData.keypoint[(int)BODY_38_PARTS.NECK]); instead ?
You are getting the local position, not the world position of the neck.
Stereolabs Support
Ok, i’ll try that out and test it - thank you so much.
I’ll let you know how it goes
Hi Benjaminv,
So, i tried what you suggested and applied other changes to my prefab.
Things are working.
I have one last question, though.
This image renders through the camera.
As you can see, the necklace isnt so shiny.
But the prefab itself, is quite shiny.
How do i get the shiny necklace to remain shiny within the image rendered through the ZEDCamera, as i see it trips the shininess away from the rendered image…
Thanks
I also have it that the object around the neck, being the necklace is spawning multiple times infinitely.
Do you know any solution that you can recommend, where the necklaces only instantiate once per detected body?
my work around is… see below:
public class DetectKeypointsOnHuman : MonoBehaviour
{
[SerializeField] public GameObject necklace;
private GameObject necklaceSpawning;
private bool spawned = false;
private List<GameObject> _spawnedObjects = new List<GameObject>();
// Update is called once per frame
public void Start()
{ if (spawned == false)
{
var zedManager = ZEDManager.GetInstance(ZED_CAMERA_ID.CAMERA_ID_01);
zedManager.OnBodyTracking += OnBodyTracking;
}
}
private void OnBodyTracking(BodyTrackingFrame bodyFrame)
{
_spawnedObjects.ForEach(x =>
{
Destroy(x);
});
_spawnedObjects.Clear();
bodyFrame.detectedBodies.ForEach(body =>
{
spawned = true;
var zedManager = ZEDManager.GetInstance(ZED_CAMERA_ID.CAMERA_ID_01);
var neckPosition = zedManager.GetZedRootTansform().TransformPoint(body.rawBodyData.keypoint[(int)BODY_38_PARTS.NECK]);
var neckRotation = body.rawBodyData.localOrientationPerJoint[(int)BODY_38_PARTS.NECK];
_spawnedObjects.Add(Instantiate(necklace, neckPosition, neckRotation));
});
}
}
For the first question, can you try to disable the “AR Post Processing” option, available on the ZED Manager script, in the AR PassThrough Settings section.
You are indeed spawning the necklace each frame. You need to instantiate it once for each detection. For that, you can use the detection id.
We are doing something similar to spawn only one avatar per detection. You can take a look at the UpdateSkeletonData function.
Stereolabs Support
1 Like
Thanks Benjamin, for the reply.
I’m going to go through all your feedback soon & i’ll let you know how it goes.
First part - done - feedback: Looks better but not as shiny, in it’s original shiny state but it’s okay for now.
Working on the next part - will let you know how that goes.
And thanks for the help.
I’m learning so much!
Hi
I got my game object to spawn/instantiate only once instead of multiple times, around the neck area.
public void Start()
{
zedManager = ZEDManager.GetInstance(ZED_CAMERA_ID.CAMERA_ID_01);
zedManager.OnBodyTracking += BodyTracking;
}
private void BodyTracking(BodyTrackingFrame bodyFrame)
{
var missingBodies = currentlyTrackedBodies.Where(existingBodies => !bodyFrame.detectedBodies.Select(detectedBodies => detectedBodies.id).ToList().Contains(existingBodies.Key)).ToList();
missingBodies.ForEach(existingBody =>
{
Destroy(existingBody.Value);
currentlyTrackedBodies.Remove(existingBody);
});
bodyFrame.detectedBodies.ForEach(body =>
{
int person_id = body.rawBodyData.id;
var position = zedManager.GetZedRootTansform().TransformPoint(body.rawBodyData.keypoint[(int)BODY_38_PARTS.NECK]);
var rotation = body.rawBodyData.localOrientationPerJoint[(int)BODY_38_PARTS.NECK];
//////// THIS ROTATION MIGHT NEED TO BE CHANGED
rotation = new Quaternion(rotation.x,
rotation.y,
rotation.z * -1.0f,
rotation.w * -1.0f);
if (!currentlyTrackedBodies.ContainsKey(person_id))
{
var necklace = Instantiate(necklacePrefab, position, rotation);
currentlyTrackedBodies.Add(person_id, necklace);
}
else
{
var necklace = currentlyTrackedBodies.SingleOrDefault(x => x.Key == person_id).Value;
necklace.transform.position = Vector3.Lerp(necklace.transform.position, position, Time.deltaTime * 100);
necklace.transform.rotation = Quaternion.Lerp(necklace.transform.rotation, rotation, Time.deltaTime * 100);
}
});
}
I have another question.
How do i get the global rotation on the neck keypoint from the human body.
You have given me the global position and that worked lovely, would you be able to provide me with the code to get the global rotation, as i’ve tried a few things but none seems to work.
current code snippet:
var position = zedManager.GetZedRootTansform().TransformPoint(body.rawBodyData.keypoint[(int)BODY_38_PARTS.NECK]);
var rotation = body.rawBodyData.localOrientationPerJoint[(int)BODY_38_PARTS.NECK];
My rotation is happening in reverse to how i move my neck.
See how it’s not following my head… it’s going in reverse and not mirrored to the rotation of my neck.
Also, i’ve disabled the AR post processing but the necklace is not as shiny, as the mesh that sits within our game object. Is there something else, I can do to enhance the shininess, please?
Hi,
To get the global rotation of the neck, you need to add up all the rotations of the neck and all of its parents, up to the root.
There is actually another parameter you can disable in the ZED Manager, it’s called “Grey out Skybox at Start”. On top of that, you can play with the lights in the scene to make it more shiny.
Stereolabs Support
Ok, trying this out - thanks!
Hi @BenjaminV
Could you give me an example of something to reference, for the below statement:
“To get the global rotation of the neck, you need to add up all the rotations of the neck and all of its parents, up to the root.”
Struggling to get my head wrapped around this statement.
A local rotation is the rotation of a joint relative to its parent. So, to have the global rotation, you need to not only apply the local rotation of the neck and also the local rotation of its parent, and the local rotation of its parent, until applying the rotation of the root (which is the pelvis keypoint for the sdk).
Actually, maybe there is another solution. If you set the necklace game object as the child of the neck game object in the hierarchy (on the game object of the avatar), it should automatically follow the neck’s rotation.
Stereolabs Support
1 Like
Ok, the issue was not code related or how i set up my game object but how very lit the room was & all the reflective surfaces from behind me (i.e. the snake tanks).
Issue sorted, on having the game object rotate with the users neck