Construct 3D World outside of AR Frame | Unity, ZED mini

Hey all!

I have a question regarding the AR Passthrough with ZED mini and Rift in unity. I would like to construct a 3D world around the passthrough frame in unity and be able to move objects in and out of the augmented reality frame. One application would be for example to have a mesh recorded and see it beyond the borders of the passthrough frame.

As I see now, objects in the scene get projected onto the frames for both eyes. How would I go about having the objects in “both worlds”, the augmented projection and the scene around the frame?

Thank you for any help on this!