Unity using Multi Camera ZED 2 in bodytracking

Software Unity
1

I am trying to set up a scene using multi camera to do the bodytracking of avatar.
I have set 2 Zed_Rig_Mono and to BodyTrackingModule in the scene.
But the avatars are sticked together.
I tried to make an offset in the Zed Body Tracking Manager.
But the offset affects all the avatars at the same time.
How can I fix it?
As I will add 2 more cameras, while the whole scene will be very wide.
How can I offset the avatar according to the specific Camera_ID?

Thanks.

2

Hi @dragonxii2002, welcome to the forums!

Each ZED Body Tracking Manager should manage its own ZED Manager (found on ZED_Rig_Mono), referenced in ZED Manager variable. The ZEDManager should have different Camera ID set.

Then if you expose a setting for your offset in Body Tracking Manager, you should be able to apply it to the bodies it references and only it.

Do not hesitate if further clarification is needed or if I understood your problem wrong.

Side note:

To use the body tracking Fusion module with Unity, allowing several camera to work together on and in the same space, you would have to use our Live Link for Unity plugin. The process involves getting familiar with the Fusion and the configuration procedure for the cameras: ZED 360 usage.

3

Thanks for your reply.
I have recorded the video for your reference.
https://youtu.be/nnt6qop98LQ
It did not behave to the bodies it references and only it…
I am not sure why.

Thanks for the side note. But my case is complicated. I need to use the depth view of the ZED Cameras as the canvas view. And I need the bodytracking for human detection to make collider event which related to the canvas screen.

It would be good if I can solve the simple offset.

4

@dragonxii2002
I found the issue!

It’s not much to change, but basically the sample was not designed for multicamera, so some assumptions are made that don’t work in your case.
If you navigate through the variables you use, from “Manual Offset”, you will find that it’s applied in the HeightOffsetter.cs which grabs the BodyTrackingManager with a FindObjectOfType call. It’s not ideal, but perfectly works when only one is in the scene.

The easiest way to bypass that in your case would be to add an variable in BodyTrackingManager and apply it in this loop in UpdateAvatarControl(). You’d just add your offset to worldJointsPos[i].

However, this is limiting. The extra step would be to give a reference to the BodyTrackingManager when creating the HeightOffsetter, so that FindObjectOfType is not used, to be able to use the automatic offset/manual offset normally. It will be fixed in the next release.

5

@dragonxii2002
See this commit: Comparing master...dev-4.0.7 · stereolabs/zed-unity · GitHub
To see where to put the code

6

Hi there,
Thanks for the solution. It works well.

By the way, I am also studying how to use Unity Live Link. Is there any way to get the depth image in the Unity?

7

Hi @dragonxii2002,
Unfortunately, we only transmit the body tracking data in Unity Live Link. You could modify the sender code to extract this data in retrieveImage() with the depth view parameter, but we don’t have a sample for this.
See the API reference about retrieveImage() for info on the function.