Share ZED camera frames across multiple applications / programming languages

#1

Hi,

I would like to share the grabbed frame of the ZED to several programs, with as much performance as possible, e.g. I open a connection to the camera in a cpp application, grab grab the frame, but then need to share it to other cpp or python applications (more than 1 other app).

One simple way is to write the frame to the file system, i.e. the main application which owns the connection to the camera write the frame on the file system and consuming applications read it (read only) whenever they need it and at their own frequency. But 1) it is slow, 2) I am using a SSD and I do not want to ruin its life cycle by doing crazy frequent writes on it.

I have explored named pipes but they block the push application and once the data is read by one application on the right end of the pipe the data is gone and not available anymore for a second consuming app for instance.

I would like to avoid networking type of libraries which are usually slow and tedious to use, but happy to be proven wrong.

Ideally I would need a method to write frame files in memory, but in a persistent and non-blocking way, i.e. not named pipes but quite close to it.

Or I would need a magic SDK function which enables me to get the ZED frames across several apps?

My 2 programming languages here are Cpp and Python really.

thanks for your help/ideas

#2

Hi,

That’s a good use case and to be honest, there is no perfect solutions.
However, I think there is basically 2 ways of handling that :

  • use IPC technics
  • use the ZED SDK streaming module.

A) Inter-Process Communication.
To my knowledge, there is 3 main ways of doing it:

  • Shared storage (file or memory)
  • Pipes / Message Queue
  • IPC sockets (I don’t know much about that)

As you said, Pipes are unidirectional (one sender/one receiver), therefore it does not fit your needs if you have multiple consumer.
IPC sockets can have multiple consumer (I think) but it needs to be handle in the code.
Shared storage is either file or memory. Most of the time, shared memory will be used since it is more efficient (even if it use a backing file behind) and most API for shared memory will provide efficient lock with semaphores mechanism.
Therefore, I would recommend to use shared memory.
I have used QtSharedMemory (https://doc.qt.io/qt-5/qtcore-ipc-sharedmemory-example.html). It is easy to use (like most Qt modules) but it is a bit of a blackbox (like most Qt modules). At the time, the performance requirement on my application was not big, so I cannot tell if it is efficient compared to low level API (POSIX for ex).
Boost also have IPC mechanism implemented (https://www.boost.org/doc/libs/1_38_0/doc/html/interprocess.html)
–> To sum up, I would use QtSharedMemory or Boost shared memory.

B) Use ZED SDK streaming module
It is currently possible to have 1 camera sender and multiple camera receivers on the same PC.

Advantages :

  • It’s flexible : if somehow you decide to move one of the client or server App to another PC, it can still be used.
  • Even if compression is used to transmit data, it uses hardware encoder/decoder, therefore the impact on performances/cpu usage/gpu usage should not be big.
  • It’s simple. (no need to integrate other libs, …)

Downside:

  • Images are compressed… therefore all the receivers will not have the RAW image as the sender can have.

–> to sum up, If compression is not a blocker on the app, I would use that.

Again, that’s based on what I know, and I might be wrong and influenced by previous good or bad experiences…

Some interesting links for IPC :



#3

Thanks a lot for your prompt and comprehensive reply.

I actually found a good tradeoff for now, acheiving performance and being quite simple to implement: ramdisk. It is fast, does not eat my SSD life expectency, and is easy to implement in different languages (writing files on the system).

#4

hi, coming back on this topic, I ended u using redis to cross share frames across multiple programming languages. It kind of works but it is a bit slow (I cap at 30fps) and quite CPU intensive which is my main issue.

Do you or could you plan for an option to share the camera resources across the supported programming interfaces through the SDK, i.e. C++, c# and Python?

Personally I care only about C++ and Python but doing it across all languages supported by the SDK would make sense I think.

Looking to reading your reply on this (even a short 'No we cannot not and will never do." so that I know what to expect)
thanks

#5

Did you try by using the local streaming module on localhost with one sender and multiple receiver?
Maybe that’s the less consuming for CPU/GPU