Video Stabilization Using IMU

Hardware ZED2
1

Hey, just considering buying the ZED2, but was unsure about two things:

  1. I want to install it on a boat, but I heard, that for IMU calibration, the camera needs to be fixed for a period of time. This assumption is hurt when the boat is slightly moving in dock. Do you think this is a problem?
  2. In rough sea conditions I would like to stabilize the video feed via IMU. Is there a logic for that available somwhere?

Thanks a lot!

2

Hi @C-der-Baum
Welcome to the Stereolabs community

This is not true. The IMU is factory calibrated, so you can use it on dynamic systems.
You can experience a slight drift in the warmup phase because the factory calibration is performed at a stable temperature reached after a few seconds after the camera boot up.
What the SDK does is “tune” the calibration at the operating temperature by estimating eventual biases when the accelerometer reveals that the camera is static, but this is an improvement of the parameters, not the full calibration.

We do not provide this feature with the ZED SDK, but you can try to apply camera stabilization algorithms. There are many possible ways all described in scientific papers available online.

3

All right, that makes sense, thank you! I ordered it.

One last followup question if I may:

The Unity interface SDK is currently not available for arm architectures if I see that correctly. Is there some alternative to interfacing the ZED2 and Unity?

4

Hi,

What platform do you want to use exactly ? The ZED SDK can only be ran on Windows/Linux computers and Nvidia Jetson plarforms (L4T).

5

Hi,
thanks for your reply!
I want to use it either on a Xavier or Orin.

On Getting Started with Unity and ZED | Stereolabs it says

‘x86_64 builds only’,

which led me to believe that it’s not possible on Jetson platforms. Is that incorrect?

6

Yes,

I think Unity itself is not supported on this platform.

7

All right, thanks!
Do you have experience with interfacing Unity and the camera on this platform?
For Python, I’d just try to do it natively via, e.g., ZeroMQ: