Chroma key flight sim and real world cockpit with zed mini

Hello all. I have seen a demo of the varjo vr headset with a stereo camera running a flight simulator with a real world cockpit.

The cockpit is shown using the the camera. There is a green screen around the cockpit and the flight simulator is rendered in the headset using chroma key. Does any one know if this can be done with the zed mini? And not too much programming. I am not a programmer but might be able to figure it out if it doesnt require any coding. How to insert flight sim video into the chroma key space?



You should take a look at our Unity Green screen capture documentation (Unity - Green Screen VR Capture | Stereolabs).
That’s look very similar to what you are trying to do.

Benjamin Vallon

I just got a Varjo XR3 2 weeks ago, come PM if you have some question and if I can help. ChromaKey is native in Varjo base, just one click and everything green around you will be used.

Fast explanation :

Create :

  • new Unity project
  • New scene
  • Click on camera → Transform to XRRig
  • drop Zed_RIG_Stereo prefab into your main camera (XRRIG now)
  • set your ZED setting (USB, svo or stream)
  • click run and your done.

Its work with all app, you play game? Run your game normally and all green around you will be your game.

Just be careful, i have ZED2 and the resolution is very low compared to XR-3 HMD, the result is not really good.

Here a fast video i send to my boss for showing how chromakey work :

PS: Its not a VR HMD on your video, but an XR HMD.

No need to use “complex” green scene with Varjo system. Chromakey is all one inside Varjo Base.

Thanks for the information. I have read through the unity with green screen documentation. What I am trying to do might work but like I said I am not a programmer. The last programming I did was when I was getting my Electronics Technician training and did some machine language programming on a 8080 CPU and some Microsoft BASIC programming back in the mid to late 80’s.
The Varjo looks nice but the price is way too high for the average user. I don’t know if the Zed mini would have good enough resolution to see the cockpit gauges. I would be running this whit the Pimax 8K X headset 3840x2160 resolution per eye. The camera might be the limiting factor.

Well the XR-3 is only for professional user, they do not sell for personal use, its weird I know…