Body tracking with a camera in portrait mode: status after SDK 5.5 and GEN_2

Hi Stereolabs team and community,

We would like to pick up a topic that has come up in several threads over the past years: body tracking with a ZED camera mounted in portrait orientation (rolled by 90°). We tested the new GEN_2 model in SDK 5.5 (without the impression that it’s solved after comparing both, landscape and portrait orientation tracking) and would like to know where this topic stands and how you recommend we deal with it.

We have several questions down the same avenue:

  1. Do we miss something that enables body tracking with a camera in portrait mode? Does GEN_2 address camera roll, or only body orientation relative to an upright camera?
  2. Does the body tracking module use the IMU or gravity to compensate for any camera orientation?
  3. Without the IMU, is there a supported way to give the SDK the fixed camera orientation (for example initial_world_transform in PositionalTrackingParameters) which also has an effect on the body tracking itself?
  4. If none of the above points holds true, what do you recommend? For example, can we rotate the images before they reach the body tracking module?
  5. And most importantly is camera roll support for body tracking on your roadmap, and if so, for which release?

We can also share short SVO2 recordings of both orientations via pm.

/PointCloudNine

Hi @PointCloudNine,
Welcome to the StereoLabs community.

I’ve forwarded all the details that you reported here to the ZED SDK team to try to replicate the behavior in our labs and understand if it’s expected.

This would be greate. Please share them via email to support@stereolabs.com