Can't control Cameras through ZED Hub

I can no longer control any of my cameras through the ZED hub. They do not respond to attempts of starting apps or rebooting. I restarted a zed box, but still have the same problem. I can connect to the cameras locally by ssh user@IP from my computer. I tried adding a camera as a new device to the hub - the installation process was successful, however, I still can’t control the newly added camera through the hub.

  1. How do I fix it?
  2. Is it possible to start applications manually by accessing the cameras locally through the network?

Hi,

You can restart your application manually through docker.
You can see what app is running with the command docker ps. You can restart them with docker restart <id of the container>. The app will have the same id that you can find in /usr/local/sl_hub`

However what you observe is quite weird. Does that occurs for all your cameras of your workspace ?

@alassagne Yes, it’s the same for all cameras - you are welcome to test it for yourself in our hub if you like Antoine :slight_smile:

I manage to connect to some of the cameras through an application running from my computer, but can’t make anything work in the hub.

Maybe your devices were rate-limited. This can (rarely) happen, and we are working on something to prevent it.
To solve it, on a box, please run
edge_cli stop
and then wait more than one full minute
and then

edge_cli restart
journalctl -f -u edge_hub_client

And send me the output.

@alassagne Still can’t control anything through the hub. Here’s the log:

user@GTW-XNX-D1EACJBW:~$ sudo journalctl -f -u edge_hub_client
– Logs begin at Mon 2023-10-02 07:13:05 UTC. –
Oct 03 06:17:49 GTW-XNX-D1EACJBW edge_hub_client[2682768]: [sl_mqtt] Using secured communication with CertAuth : /etc/ssl/certs/ca-certificates.crt
Oct 03 06:17:49 GTW-XNX-D1EACJBW edge_hub_client[2682768]: MQTT persistence is enabled.
Oct 03 06:17:49 GTW-XNX-D1EACJBW edge_hub_client[2682768]: Connecting to the MQTT server… PAHO mqtt client connected.
Oct 03 06:17:50 GTW-XNX-D1EACJBW edge_hub_client[2682768]: Subscriptions…
Oct 03 06:17:50 GTW-XNX-D1EACJBW edge_hub_client[2682768]: Subscribing to MQTT topic /v1/devices/8280915/twin/delta
Oct 03 06:17:50 GTW-XNX-D1EACJBW edge_hub_client[2682768]: Subscribing to MQTT topic /v1/devices/8280915/apps/+/twin/delta
Oct 03 06:17:50 GTW-XNX-D1EACJBW edge_hub_client[2682768]: Subscribing to MQTT topic /v1/devices/8280915/functions/in
Oct 03 06:17:50 GTW-XNX-D1EACJBW edge_hub_client[2682768]: Subscribing to MQTT topic /v1/devices/8280915/terminals/in/+
Oct 03 06:17:50 GTW-XNX-D1EACJBW edge_hub_client[2682768]: Subscribing to MQTT topic /v1/workspace_only/14130056/mqtt/+
Oct 03 06:17:50 GTW-XNX-D1EACJBW edge_hub_client[2682768]: MQTT client successfully initialized.

Found the issue. Does it work now ?

Yes, it works - thanks :smiley: What was the problem and the solution?

I have also had the regular problem that I need to manually restart the zed boxes from time to time as they stop responding (both from connections from the hub, and locally from my computer). Any idea why that may be? I see that I need to do it for a couple of the cameras now.

I simply noticed that one messaging service was down. We’ll need to add more monitoring of these.

I don’t know about that your boxes going offline. Are they not even recoverable with SSH ?

Don’t know if you noticed but we released 4.0.7 and you should notice improvements in your body tracking FPS. Don’t forget to disable tracking and fitting, since you are using Fusion you don’t need them.

I am getting 8-9 FPS now. Here is what I am running (SDK 4.0.7, Edge body tracking 0.8.1, newest edge_cli):

HUB:

  • Edge Body Tracking App
  • Accurate detection model
  • Body 18 (No out-of-the-box option for body 34)
  • Body tracking and body fitting DISABLED
  • HD1080, Ultra depth, 30 fps

Application:

  • Accurate detection model
  • Multi-camera application from your ZED SDK samples (C++)
  • Body 34
  • Body tracking and body fitting ENABLED (both in sender and receiver - I need to get positions of all joints at all time steps, and I need ID of detected people).
  • allow_reduced_precision_inference set to TRUE

How can I increase FPS while still making sure all joint positions are estimated at all timesteps? I find it strange that I have to have the Edge body tracking app enabled to run my sample - why is that? Can I optimize the app to not send its stream to the hub if that helps?

You should definitely disable the fitting. The fusion will not take it in account anyway, and will re-fit the persons afterwards.

@alassagne Disable it in the ClientPublisher (BodyTrackingParameters), or in BodyTrackingFusionParameters or both?

In the client publisher. They key is to increase the framerate of the sender. The receiver is less important because it has a lot less work to do. Also, in your case if you use Body 34 you need the fitting, but on receiver side.

I am getting 8-9 fps running with body tracking and fitting disabled in client, enabled in receiver (body 34). How can I get it higher?

Are the recordings enabled ? That can save some framerate. Also what about the MEDIUM model ?

All recordings are off. I get 10-12 FPS running the medium model, and the detection is a bit smoother (less strange glitches where body parts are stretched a lot).

I am testing using your Multi-Camera sample ( zed-sdk/body tracking/multi-camera/cpp at master · stereolabs/zed-sdk (github.com)) and I am running the default Edge Body Tracking app on the cameras. For that I have some questions:

  1. Are the client publisher settings SOLELY set through the Edge Body Tracking HUB application? So the ClientPublisher.cpp in the Multi-Camera repo is of no use for me as I am running with config to the cloud (network)? Because if that is the case I need to include the option to set allow_reduced_precision_inference in the camera hub application. Do you have a sample for that? Or could you priovide hub application code to include it as an option?
  2. Am I harming my detection by setting BODY 18 in the Edge Body Tracking application or will it automatically use the appropriate detection model/settings when I enable body fitting? There is no native option to set BODY 34 in the hub application - why is that? Could you include Body 34 in future releases so I don’t have to manually create an altered version of the app after every new update. Also, is it still possible to include Body 34 in this application?

Hi, I’ll send you the edge body tracking app in PM, it’s probably the simplest way. You can use the ClientPublisher.cpp, ZED Hub is not mandatory to use the networked fusion.

I am unable to run the Multi-Camera sample (or any other multi-cam sample) without having the Edge Body Tracking Hub app running on the cameras. I get Connection Timed Out. That leads me to believe that The ClientPublisher.cpp in the Multi-camera sample only applies if the cameras are connected via USB? And the hub app controls settings when running over the network. Is that correct? (I am pretty sure of this as changing detection model in ClientPublisher.cpp does nothing to the FPS, however, changing it in the hub app improves/decreases it).

With the following settings in the hub I now get 12-13 FPS:

  • Accurate detection model
  • allow_reduced_precision_inference = true
  • Body 34

With the same settings but medium detection model I get 14-15 FPS.

Q: With my given FPS results, would you recommend accurate or medium model for most precise and non-glitchy skeleton detection? Also, what do you recommend during calibration?

Q2: Given that the hub application does control the cameras, can it be optimized to run faster? Any suggestions?

Note: For future releases of the hub application “Edge Body Tracking” please include the possibility of selecting Body 34 and the option to enable/disable allow_reduced_precision_inference :slight_smile:

Hi,

The client publisher is only here for USB cameras. The multicamera sample can use network zed (with or without hub) but you’ll have to fill the values in the configuration file, and run an app on the sender side.