Jetson Orin Nano - ZED X - Unreal Engine

Software Unreal Engine
1

Hey guys,

A bit of a pickle here unless I’m missing something obvious.

We have a ZED X camera as well as a Jetson Orin Nano with the capture card. We have the SDK installed and everything is working fine as a standalone app.

Our goal is to get livelink body tracking data into Unreal running on a Windows machine. We’ve tried a few different approaches but non seem to work.

  • Stream ZED data to the Windows machine and do all the calculations there. Apparently the Jetson Orin Nano does not have a hardware accelerator and it crashes the unit every time you try to stream (h.264/h.265) (had posted another message about this issue a week ago, apparently not possible to stream with the current SDK on the jetson nano)

  • Tried Running the example executables for linux/windows on the nano through QEMU/Wine (as original files are compiled for x64 and not arm64)

  • Tried to compile/recompile our own app/plugin to do the processing for us. Another roadblock here, apparently you need to have Unreal installed in order to compile a livelink compatible app. No Unreal for arm64, unless we compile it ourselves which might be another rabbit hole.

Another solution that I can think of is to do the “Raw” data capture/some kind of forwarding (not streaming, think device forwarding) on the box and just transmit it ourselves over UDP or TCP and have an encoder on the other side receiving the data and processing it and then relaying it to livelink, but is that really the only option?

Isn’t there an established workflow on how to get the Zed X camera data sent to Windows?

Are there even capture cards that are compatible for regular PCs/Windows based machines to avoid all of this?

The goal is to use the Zed X camera with Unreal as it has better hardware. We have a cluster of Zed 2i cameras and we’re hoping that the Zed X will provide better fidelity but we’re having to jump through a bunch of hoops in order to make this work. Hoping to find a more streamlined solution even if it means getting a windows compatible capture card.

Any helps is appreciated. Thank you!

2

Hi @GevC,

Sorry for the inconvenience, but for now, the Orin Nano specifically will not work with the Live Link plugin for all the reasons you said.

We are working on a solution for allowing streaming for an Orin Nano that will be released some time this year, enabling software encoding.

For now, you’d have to build something like our Live Link for Unity, like you said: Body Tracking sample that sends just the body tracking data in the format you want (json for example) to an app packaging this data in Live Link format. You can base yourself on the Live Link for Unity sender sample and our ZED UE5 Live Link plugin. Going through the main() should give you the expected workflow.

I hope this helps a bit. :sweat:

3

Hi @JPlou,
Does the Live Link plugin work with NVIDIA Jetson Orin developer kit?