Chroma key flight sim and real world cockpit with zed mini

Software
1

Hello all. I have seen a demo of the varjo vr headset with a stereo camera running a flight simulator with a real world cockpit.

The cockpit is shown using the the camera. There is a green screen around the cockpit and the flight simulator is rendered in the headset using chroma key. Does any one know if this can be done with the zed mini? And not too much programming. I am not a programmer but might be able to figure it out if it doesnt require any coding. How to insert flight sim video into the chroma key space?

Robertw

2

Hi,

You should take a look at our Unity Green screen capture documentation (Unity - Green Screen VR Capture | Stereolabs).
That’s look very similar to what you are trying to do.

Best,
Benjamin Vallon

3

I just got a Varjo XR3 2 weeks ago, come PM if you have some question and if I can help. ChromaKey is native in Varjo base, just one click and everything green around you will be used.

1723Ă—1000 178 KB

Fast explanation :

Create :

  • new Unity project
  • New scene
  • Click on camera → Transform to XRRig
  • drop Zed_RIG_Stereo prefab into your main camera (XRRIG now)
  • set your ZED setting (USB, svo or stream)
  • click run and your done.

Its work with all app, you play game? Run your game normally and all green around you will be your game.

Just be careful, i have ZED2 and the resolution is very low compared to XR-3 HMD, the result is not really good.

Here a fast video i send to my boss for showing how chromakey work :

PS: Its not a VR HMD on your video, but an XR HMD.

4

No need to use “complex” green scene with Varjo system. Chromakey is all one inside Varjo Base.

5

Thanks for the information. I have read through the unity with green screen documentation. What I am trying to do might work but like I said I am not a programmer. The last programming I did was when I was getting my Electronics Technician training and did some machine language programming on a 8080 CPU and some Microsoft BASIC programming back in the mid to late 80’s.
The Varjo looks nice but the price is way too high for the average user. I don’t know if the Zed mini would have good enough resolution to see the cockpit gauges. I would be running this whit the Pimax 8K X headset 3840x2160 resolution per eye. The camera might be the limiting factor.

6

Well the XR-3 is only for professional user, they do not sell for personal use, its weird I know…

7

Hello RobertW62, any news with your project?

Possibly I am on the same track and striving to sort it out in a similar way…

8

I have built a cockpit but I am using 3 55" tv’s for the displays. I am waiting for reality mixer on steam to have chroma key enabled with stereo cameras like the ZEDmini.

1 Like
9

Hi everyone, thanks for all the posts so far, I have also decided to take the similar route - to KEEP USING my Reverb G2 for the time being, and yes I want to go in the Augmented/Mixed Reality direction for DCS flying, in combination with WinWIng MIP components. ZED mini is on the way as we speak! :slight_smile:

Has any of you came through with the ZED Mini AR cockpit solution? I plan to use either the Steam Reality Mixer, or if possible, create an Unity project that will deliver the solution I need.

I’d be really grateful if you would share your experiences, since I will embark on the same journey with ZED mini, and I must say I haven’t found a lot of discussions, threads or videos on this particular topic, utilizing ZED Mini for a “window” into the physical world of our cockpit.

Please share! Thanks!

10

Someone? Please? Is it possible to use Unity and inject the Flight Sim in the background? Some of you were so brave to try this by keeping your G2s and embark on using ZED MINI. I did the same… I hope it wasn’t t a mistake? :frowning: