Hi, we’re working on streaming video from zed x through unreal engine 5. Our VR headset is XR elite and there’s problem with getting real time stream. How we can solve this problem?
Hi @NeginHeiran,
Thank you for reaching out.
I will need more information regarding your setup in order for me to answer.
On what system are you using the ZED X camera? Are you able to view the camera’s video streams on the Jetson device?
Are you capable of streaming the ZED X data successfully to another tool such as ZED_Explorer on another PC?
Hi Matt,
Thank you for answering. we use ZED X on NVIDIA Jetson AGX Orin.
Yes we did all above tasks that you mentioned.
The problems and crash comes up when we want to change the project to VR in unreal engine. we checked both setting the Zed X firstly on unreal engine and then change it to VR and other making a new VR project in unreal engine and using available plugins of ZED X. In both case the unreal engine start crashing. Therefore, we can’t even use the ZED X camera as a webcam.
Please kindly let me know as soon as possible. we are in deadline of our project.
Kind regards,
Negin
Hi @NeginHeiran,
I would like to see if this issue is coming from the streaming from the AGX Orin, or the subscription in Unreal.
Can you run ZED_Explorer on the Jetson and validate that you can see the video feed of the camera?
yes we can see it, the problem reveal when we make it VR.
Hi @NeginHeiran,
Thank you for the information, could you share the crash logs that Unreal outputs? This will help me understand may be going on.
Hi Matt,
I attached the log note from unreal engine when we’re trying to use Zed x mini for our VR project.
ZEDSamples.log (147.3 KB)
Hi @mattrouss, @Myzhar , @JPlou ,
The entire project has been stopped just because we expected Zed X mini to support Unreal Engine VR platform.
Can your support team provide an updated SDK to support this camera in Unreal Engine VR platform?
We were very excited to use your product, but now it is an absolute useless product for VR and robotics. Please kindly push your team forward to provide such SDK? Without it Zed X mini has no place in robotics and VR market. With due all respect, a team of 30 people are waiting for a proper response from your company.
Regards,
Negin
Hi Negin,
I’m sorry for the late reply.
Unfortunately, the crash log does not seem very helpful.
I’ll try to reproduce your crash on my side, could you describe me your configuration so I can replicate it here?
Thanks
Hi @BenjaminV,
Yes sure, we’re using NVIDIA Jetson AGX Orin 64 GB developer kit with Zed X mini. We launched a VR project with Unreal Engine 5, using the latest plugins that you released. We have the video stream on unreal engine from Zed X mini, but whenever we try to change to project to VR project, it crash.
Thank you, please kindly let me know about this issue as soon as possible.
Negin
When you say " we try to change to project to VR project, it crash", what are you doing exactly?
And you are using this project (GitHub - stereolabs/zed-UE5: ZED plugin and examples for Unreal Engine 5 (Standard Engine)), right?
Thanks!
Yes, we used this version and all the other available versions, even the latest version that you released last month.( 5.4 version).
Does your example is compatible with VR?
Our Unreal integration is not VR-compatible. For that, we recommend using our Unity plugin instead (GitHub - stereolabs/zed-unity: ZED SDK Unity plugin).
We have to use unreal for our system because all the other devices like Nova haptic gloves launched on Unreal Engine. Is there any way that you can develop the VR compatible version?
Hi,
It’s not planned for the moment, we prefer to focus our resources on one platform.
Hi @BenjaminV ,
Our team working with Unreal engine, so please let me know about these subjects so that I can decide about what our team can do. because they developed all systems on Unreal Engine and it takes time to migrate from Unreal to Unity.
- How we can open Zed X camera video stream like webcam on browser?
- Is there any compatible version of Unreal that is compatible with VR and Zed X?
- What is your suggestion for our team on this issue?
Best regards,
Negin
Hi,
-
The ZED X camera cannot be opened using webcam applications based on the UVC standard that is only available to USB devices. You might need to use Gstreamer : GitHub - stereolabs/zed-gstreamer: Package of GStreamer elements to interface with ZED Cameras.
-
If by “compatibility with VR and ZED X” you mean it can display the images of the ZED X inside a VR headset, the answer is no.
-
We have a very old Unreal plugin/project (using a custom version of UE4.21) available here :
GitHub - stereolabs/zed-unreal-examples: Stereolabs ZED - UE4 Examples and GitHub - stereolabs/zed-unreal-plugin: Stereolabs ZED - UE4 Plugin. This plugin was VR compatible but required a custom version of the Engine.
This plugin is obsolete and no longer supported. You can take a look at it to see how we did the VR compatibility.
I want to make it clear, this is no longer supported by Stereolabs.
Hi @BenjaminV,
-
From the first and second items that you mentioned I’m a little misunderstood. Can I use GStreamer to interface with Zed camera in an browser to have the video stream inside the headset?
-
From the second item you mean just with Unreal? or Unity too? Because I thought if we change the VR engine from Unreal to Unity we can have the image of the Zed X inside a VR headset.
-
As you mentioned, there’s an old version of Plugin which is compatible with VR. Can you guide us or send us documents so that we can develop the compatible version with Unreal Engine 5 and share it here too? If you can send us drivers and needed files from Zed X hardware layer we are prepared to work on it. I’m sure that there are or there would be other developers that need such plugin like us.
Best Regards,
Negin