Issues in Camera Orientation, Character Movement, Height and Getting Height and Width in Body Tracking in Unity

Software C#
1

Hello,

I’m encountering a few issues with body tracking in Unity that I hope you can help me with:

  1. Camera Orientation: I need assistance in setting the ZED camera from horizontal to vertical rendering. How can I achieve this?
  2. Character Movement: I’m using a 3D character to visualize my movements, but I notice that the body continuously moves in the Y direction. I’m looking for a solution to ensure that the movement data reflects my actual position accurately.
  3. Height Issues: When small people (like children) come in front of the camera, they appear on the ground rather than being tracked correctly. How can I improve the tracking accuracy in these scenarios?
  4. Getting Height and Width: I’m having trouble obtaining the correct height and width using zedmat.GetHeight() and zedmat.GetWidth(). They seem to return unexpected values. Any tips on how to get these dimensions accurately?

Thank you!

2

Hi,

  1. Could you be more specific about what you wish to achieve here? I’m not sure to fully understand. Thanks!

  2. Can you make sure the ground floor is visible by the camera?

  3. What do you mean by “on the ground”? Do you have a screenshot to share? You can try to enable this option on the ZED Body Tracking manager script. This forces the 3D characters to be on the ground.
    image

  4. Are you sure the Mat is correctly initialized? Can you share your code snippet so I can take a look at it ?

Thanks.

4

Below, I have detailed each issue and included relevant references, such as a demo video and a photo of my current setup.

YouTube link

IMG20241001105929
IMG202410011059293468×4624 351 KB

List of issues :

  1. Camera Setup Orientation (Horizontal to Vertical): I need to set up the camera vertically since my application is in portrait mode. I have attached a reference image of my current setup. Could you please confirm if this is the best way to set up the camera? If not, kindly provide suggestions for an optimal horizontal-to-vertical setup.

  2. Character Hand Overlapping Issue: In the attached demo video, I noticed that the 3D character’s hand overlaps with the body when I touch my chest. Could you provide guidance on resolving this issue?

  3. Character Non-Stop Movement: In the same video, the character appears to move in every direction non-stop, without any input. What could be causing this behavior, and what steps can I take to correct it?

  4. Body Part Movement in Idle Position: The character’s body parts, including fingers, move in an idle position based on the camera input. Is there a way for me to manually set the position of any body part as I want? If yes, please guide me on how to do this.

  5. Random Spawn Position: Despite the camera remaining in a fixed position, the character spawns in a random position each time. I would like to spawn the character in a fixed direction consistently. Could you advise on how to achieve this?

  6. Camera Overheating: I am using the ZED camera for about 9 to 10 hours, and I notice that it becomes extremely hot after some time. Is this normal behavior, or are there any recommendations to prevent overheating during extended use?

5

Thanks for the detailed information, It’s really helpful !

  1. You can place your camera vertically like you did. However, you should know the body tracking module overall quality might be slightly degraded using this camera setup. This is a known limitation of our current model but that is something we will certainly looking to improve in the future.

  2. We do not provide a ready-to-use solution for that in our Unity plugin. A solution would be to work with the physics and colliders to prevent the hands to to pass through the rest of the body.

  3. You can try to reduce the jitter by increase the smoothing value available in the ZED Manager script (see screenshot). This will also increase the latency.

    image
    image743×323 10.3 KB

  4. This is done by this function : zed-unity/ZEDCamera/Assets/SDK/Helpers/Scripts/BodyTracking/SkeletonHandler.cs at master · stereolabs/zed-unity · GitHub
    You will need to modify the code to apply a specific rotation or position to a joint.

  5. Same as 4.

  6. No, the camera is not known to overheat in general. Do you have an idea of the temp of the camera? You can open our tool called “ZED Sensor Viewer” to get the temp of each sensor.

6

Thank you so much for your support.
will try all of these