Object detection on edge devices can NOT be published to fusion on local network (C++)

Hi,

I’m working on a project that uses ZED 2i cameras with Jetson edge devices. I’m running both body tracking and object detection simultaneously on the Jetsons, and both functions work correctly — I’m able to retrieve both body and object data locally.

However, when I try to publish the detected data using the startPublishing() function with setForLocalNetwork, only the body data is successfully sent to the Fusion server. The object data doesn’t appear, even when I attempt to publish only the objects. I followed the approach shown in the multi-camera object detection sample, but instead of using shared memory, I’m publishing over the local network.

Is there a known issue with object data publishing over the local network in the SDK, or am I missing something in the setup?

I’m using SDK version 4.2.5 on both the Jetsons and the PC running the Fusion script.

Thanks in advance for your help!

Hi Zoltan,

There is indeed a known issue in the ZED SDK preventing the Object detection fusion module from working in network mode.

We are currently working on a fix, which is planned for the 5.1.0 release of the SDK.

Best,

Stereolabs Support

do this apply also to systems in which all cams are connected to a zed box and fusion processed on another pc?
This is a part of the configuration file for cameras

“42844251”: {
“FusionConfiguration”: {
“communication_parameters”: {
“CommunicationParameters”: {
“communication_type”: “INTRA PROCESS”,
“ip_add”: “”,
“ip_port”: 0
}
},
“input_type”: {
“InputType”: {
“input”: “addr:port”,
“input_type_conf”: “42844251”,
“input_type_input”: “STREAM”
}
},
“override_gravity”: false,
“pose”: “1.000000 0.000000 0.000000 0.000000 0.000000 1.000000 0.000000 -2.051296 0.000000 0.000000 1.000000 0.000000 0.000000 0.000000 0.000000 1.000000”,
“serial_number”: 42844251
}
},

The communication type is intra process, while the stream is only used as an input, but i can’t see any detection displayed

Since you want to perform the fusion on a different PC, you’ll need to publish the data over the local network. In that case, the issue with object detection may still persist. However, if you move the fusion process to the ZED box itself, you can use shared memory to publish the data, and that should work. (The shared memory approach works for me.)

Ok, but my goal was to use a different pc to perform fusion and depth retrievement for resources reasons. The zed box has already 4 cams connected and i can mantain a good framerate only by disabling depth mode.
I’m not getting why it is not working, the zed data are streamed to the external pc that computes depth and perform both detection and fusion (in ClientPublisher) over the cams. All the data are shared in that pc if i understood correctly

Hi,

I confirm the OD data is not correctly working while using the network workflow.
This is a known issue and a fix is planned for v5.1.

Sorry for the inconvenience.