Hi there
Attempting to setup two cameras with 360 fusion.
Two ZED2i with jetsons xavier JETSON_L4T=35.4.1 over the network with SDK 5.1.2 running 360 senders.
Receiver running ZED360 on win 11 SDK 5.1.2 CUDA 13.
Sometimes zed360 just crashes when trying to setup the room. Sometimes it does not and adds both cameras in one spot. See attached image. When in calibration it does not show any bodies detected or moving and calibration saves no data.
these are snippets of the code with parameters on the senders:
InitParameters init_parameters;
init_parameters.camera_resolution = sl::RESOLUTION::HD720;
init_parameters.camera_fps = 30;
init_parameters.depth_mode = sl::DEPTH_MODE::NEURAL_LIGHT;
init_parameters.coordinate_system = sl::COORDINATE_SYSTEM::RIGHT_HANDED_Y_UP;
init_parameters.sdk_verbose = 1;
PositionalTrackingParameters positional_tracking_parameters;
positional_tracking_parameters.set_as_static = true;
BodyTrackingParameters body_traking_parameters;
body_traking_parameters.enable_tracking = false;
body_traking_parameters.enable_body_fitting = false;
body_traking_parameters.body_format = sl::BODY_FORMAT::BODY_18;
body_traking_parameters.detection_model = BODY_TRACKING_MODEL::HUMAN_BODY_FAST;
BodyTrackingRuntimeParameters body_tracking_rt;
body_tracking_rt.detection_confidence_threshold = 40;
body_tracking_rt.skeleton_smoothing = 0.7;
CommunicationParameters configuration;
returned_state = zed.startPublishing(configuration);
grab is in the main loop
Both senders are adding receiver:
[RTPSession\] Adding Receiver with IP: 10.30.30.141 ID 187110364
Am I missing any crucial parameters on senders? Also is there a log or a way to see zed360 app log to see what is happening?
Last question does zed360 handle time sync of both jetsons or that has to be done as per your documentation Setting Up Multiple 3D Cameras - Stereolabs is this the missing bit in this chain?
Hi,
You should at least see the raw skeletons in 360 even if the calibration fails…
Can you first double check the senders are actually detecting someone? You can simply add a print in the main loop displaying the number of detections.
Your main loop should look like that :
Bodies bodies;
while (true) {
// Grab images
auto err = zed.grab();
if (err == ERROR_CODE::SUCCESS) {
// Retrieve Detected Human Bodies
zed.retrieveBodies(bodies, body_tracker_parameters_rt);
std::cout << "Nb of detections : " << bodies.body_list.size() << std::endl;
}
}
Hi there,
I can confirm that senders are detecting and getting body count.
I have made two identical senders C++ and Python. From many tests, python senders do not crash ZED360, both adding the receiver. But all I can see is two cameras added to the ZED360 scene, no skeletons are showing, cameras are slightly moving around the scene.
C++ version senders are running, but when adding them in ZED360 sometimes only one sender gets added or second sender is added a second later and this is when ZED360 just crashes. C++ route is almost impossible to add senders. Python feel much more solid.
Also, what is important is to use configuration.set_for_local_network(port, ip) on each sender if this is not added ZED360 will never add senders and will always crash. Your sample code does not have this added.
Not sure what more can be debugged on my end. Can you please confirm that you can successfully setup this workflow from scratch on your end (2 jetsons via network with a PC receiver running ZED360) with latest SDK.
To add Jetsons running same SDK as PC 5.1.2 but Jetson’s CUDA is 11.4.315 as this is latest can run on Xaviers with Jetpack 5.1.2.
Thanks
Quick update.
Upgraded to SDK 5.2, on jetsons and PC running ZED360
ZED360 terminal outputs “WRONG BODY FORMAT”
I have tried all combinations on senders .18 .34 .38, also different combinations of BODY_TRACKING_MODEL
On ZED 360 all possible body model combinations.
Both Senders output Adding Receiver with IP: 10.30.20.141 ID 1551339472
This is ZED360 output:
Camera “{“input”:{“fusion”:{“configuration”:{“ip”:“10.30.20.95”,“port”:31950},“type”:“LOCAL_NETWORK”},“zed”:{“configuration”:“1”}},“world”:{“rotation”:[0.0,0.0,0.0],“translation”:[0.0,0.0,0.0]}}”
Camera “{“input”:{“fusion”:{“configuration”:{“ip”:“10.30.20.98”,“port”:31980},“type”:“LOCAL_NETWORK”},“zed”:{“configuration”:“2”}},“world”:{“rotation”:[0.0,0.0,0.0],“translation”:[0.0,0.0,0.0]}}”
Running the fusion for 2 devices.
INIT FUSION
START RUN FUSION
enableSK
ERROR : WRONG BODY FORMAT
STOP RUN FUSION
So still failing to get skeletons on ZED360, any clue to why this workflow not working for us?
This error appears when both senders are using different formats.
I doubt that’s what’s happening here, but can you double check this is not the case?
If not, I’ll try to reproduce your issue at the office.
Best,
Hi,
We have recently been experiencing the same issue. Based on the information shared in the thread Framerate issue when using the Fusion Network Workflow , we performed some tests on our side.
With the same setup, ZED360 reporting WRONG BODY FORMAT error when running with SDK 5.1 & 5.2, while SDK 5.0 works correctly.
Our project requires multi-camera Body Tracking and Object Detection. According to SDK 5.1 release notes, the fllowing feature was introduced:
Added support for object detection streaming from sl::Cameratosl::Fusion in the network configuration.
Additionally, based on the FPS issues discussed in the thread mentioned above, it seems we will likely need SDK5.1+ for our project.
So we would like to ask if there has been any update, fix or recommended workaround for this issue?
Here is some additional information about our setup:
We have two Zed Mini Box [JetPack: 6.1 (nvidia-jetpack 6.1+b123) L4T: 36.4.4 Ubuntu: 22.04] each having two Zed X connected.
Thanks
