I need help to set up multiple ZED 2i cameras.
I have floor space of 32ft x 17 ft and I want to track human movement on that floor which will trigger some animation.
we are using 4 x ZED 2i and to develop the application we are using Unity 3D software with ZED SDK 4.0 version.
There are multiple things that Zed Sdk provides
Depth sensing
object detection
Skeleton detection
and much more
We have tried using Zed 360 Fusion API with 4 cameras, 3 cameras, and 2 cameras, it is working perfectly fine with calibration.
But when we live link it to the unity application it freezes when the 8th person enters the scene.
and we want to use it for at least 20 people simultaneously.
I am not using any skeleton data I am attaching one object to the skeleton pelvis which will interact with the Animation and other objects.
Can anyone tell us which module I should use to set up multiple cameras?
I am hanging cameras in the topdown position in 4 parts to cover the floor area. in which I am facing overlap in all 4 four cameras.
Check the setup image here
What am I doing wrong?
can someone please tell me how should i execute this best way?
I have already wasted my time on Zed 360 Fusion API.
How can I Use multiple cameras without using ZED 360 Fusion API?
and seamless interaction between all 4 cameras to map the whole floor area.
Thanks for the reply. but as I’ve mentioned, we are developing our app in Unity3D game engine.
The only problem is the FPS/performance.
It’s working fine till 7 people in the scene but freezes when 8th person enters.
Can you suggest some guidelines to achieve good and stable performance with atleast 15-20 people in the scene for fusion with Unity3D platform ?
I asked that question to understand if the issue is coming from the live link part, the way we send the data to Unity, or directly from the ZED SDK.
I should have given more context in my previous message.
If I understand correctly, all the avatars stop moving when an 8th person enters the room.
I’ll investigate on my side and I’ll come back to you once I have more information.
Great news.
just confirming you found issue of limiting person to 7 in unity live link ?
When it will be updated ? in 2 or 3 days or it will take time ?
Thank you very much. I’ll make sure to test it as soon as possible and provide you with an update on our progress, including how many skeletons we’re able to track.
I’ve conducted testing on the updated Unity live link project and have some findings to share. There’s both good news and bad news to report.
First, the good news is that the project now supports 8+ skeletons, which is a positive development.
However, the bad news is that there’s a significant performance issue causing every character to stutter, giving the impression of running at around 1 frame per second (FPS). I’ve included a video for you to review, which demonstrates this problem.
I’ve carried out these tests on three different systems, all equipped with 2 ZED2i cameras connected by 10-meter-long cables. It’s worth noting that previous tests were conducted on the same system without encountering this issue.
We’ve taken measures to diagnose the problem by implementing an FPS counter in our Unity 3D application. Surprisingly, the FPS counter consistently shows 60 FPS, even when there is minimal content in the scene, suggesting that the performance itself doesn’t seem to be the root cause of the stuttering.
The stuttering issue seems to manifest when two characters are present simultaneously in the scene. In the video, you can observe that when a second character enters, the stuttering becomes noticeable (please refer to the third part of the video where there are two characters on screen).
Additionally, when a character exits the camera’s view, their skeleton freezes in place. When that same character re-enters the camera’s view, a new skeleton is created while the previous one remains frozen.
It’s important to note that this stuttering problem was not present in the previous version of the Unity live link and seems to have emerged after the recent update.
Thank you for your assistance and support in resolving this issue.
I’ve identified the issue on the fix
Didn’t test enough.
Basically, it only updates one skeleton each tick, I’m working on a fix for the fix to aggregate the body tracking data before it gets to the animation part.
To clarify, what’s happening is in ZEDStreamingClient.cs, I think the call to GetLastBodiesData in Update() drops all changes except the last json got.
Sorry for the inconvenience, it will be fixed asap, that’s my mistake
Apologies for the delay in responding. We’ve made attempts to address the issue from our end, but unfortunately, we haven’t been successful in resolving it.
Have you managed to find a solution or make any progress on your end?
I’m sorry to say it’s still WIP as we couldn’t take the time to finish this yet.
This is an important issue and I’ll get on it as soon as I can.
I wanted to express my gratitude for your efforts in updating the project. I’ve conducted tests using two sensors and involving 11 people, and the results have been quite promising. Today, I plan to run additional tests and will share the outcomes here. It’s evident that the performance has improved significantly.
Once again, thank you for your valuable contributions.
After extensive testing of various configurations and settings for my project, I’ve identified the most suitable solution. I’m using a multiple-camera setup with the ZED 360 fusion API.
Initially, I had four cameras running in Ultra mode at a resolution of 720p@30fps. This configuration worked well for tracking up to 5 or 6 people in a 32ft x 17ft area, delivering a minimum of 15 FPS in the ZED view application. However, when more people were added to the scene, the frame rate dropped, causing lagging, and we reached only 4 FPS in the ZED view application. This performance was not acceptable.
After multiple tests, I settled on the following settings:
Number of Cameras: 3
Mode: Performance
Resolution: 720p@30fps
With this configuration in the ZED 360 fusion API, I achieved good skeleton detection performance, consistently providing at least 11 FPS even with 11 to 13 people in the scene (as observed in the ZED View).
It’s important to note that skeleton detection can be influenced by lighting conditions. I conducted my tests on a terrace during the evening when it gradually became darker. In low light conditions, skeleton tracking performance began to degrade, causing the skeletons to jump around.
I also noticed that when I opened the ZED Depth Viewer Application, all the cameras appeared darker. Enabling auto exposure for each camera individually helped restore proper exposure levels and improved skeleton detection. However, the issue is that the auto exposure setting reverts to its default state every time I restart my system. I’m looking for a way to make the auto exposure setting permanent because I cannot set the exposure manually, given that lighting conditions change over time, and I want the exposure to adjust accordingly.
I hope this feedback proves valuable in enhancing your SDK. Thank you for your continued efforts.
So in conclusions, if I want to track 15 skeletons I can’t use 4 cameras or even 3 camera because the performance will be too slow, I can only use 2 cameras to get a decent framerate. Not really good news… In fact it could kill my project before it starts.
