Zed for Unity (v4.07-4.08): Only Right_eye prefab working

Hi,

I’m trying to get the Zed plugin for Unity cameras working (but I can only get right eye to work). Before I press play all the 3 prefab camera display previews are working (stereo left, stereo right, mono). But when I press play all of the displays show a black loading screen.

If I remove the mono camera prefab, then the stereo-right camera starts working fine, even after I press play. But the stereo-left still displays the black loading screen upon “play”.

Maybe this is a bug and some older versions might work for me? If this can’t be solved I could try to revert back to some older versions, but then I would be open to suggestions which old version could be worth trying. Trying SDK 4.07 could be an option too, to perfectly match the plugin? (Update: I can’t seem to find where to download SDK 4.07, I find SDK 3.8.2, but that only supports up to Cuda 11.7.)

I’m using Unity 2021.3.36f1, Zed_Unity_Plugin_v4.0.7 (supposed to work for 4.08), Zed SDK v4.08, Cuda 12.1.68.

I’m not using HDRP or URP in my Unity project.

Hello @Mattttiass, welcome to the forums :slight_smile:

Can you walk me through how to reproduce your issue?
I don’t quite get the process here.

  • Please attach a ZED Diagnostic result.
  • Does ZED Depth Viewer work correctly?
  • How do you set up your scene? Is it a sample scene?
  • How many ZED cameras are you using?
  • How many ZED Rig prefabs are you using?
  • What kind of sources are you using? (Live, SVO, Stream)
  • Do the samples included in the plugin work correctly?

Hi, thanks!

To reproduce it you just need to create an empty Unity project (3d core), import the Zed unity package and add one Zed_Rig_Mono prefab and then the Zed_Rig_Stereo prefab to your scene.

-At this point none of the cameras should work.
-If you now remove the Zed_Rig_Mono prefab from your scene the Right_eye camera should start to work, but not the left.

Below is my Diagnostic result, I dont have a camera connected, but I shouldnt need to for this?

{
“AI Models”: {
“info”: [
“MULTI CLASS DETECTION is not optimized”,
“MULTI CLASS MEDIUM DETECTION is not optimized”,
“MULTI CLASS ACCURATE DETECTION is not optimized”,
“HUMAN BODY FAST DETECTION is not optimized”,
“HUMAN BODY MEDIUM DETECTION is not optimized”,
“HUMAN BODY ACCURATE DETECTION is not optimized”,
“HUMAN BODY 38 FAST DETECTION is not optimized”,
“HUMAN BODY 38 MEDIUM DETECTION is not optimized”,
“HUMAN BODY 38 ACCURATE DETECTION is not optimized”,
“PERSON HEAD DETECTION is not optimized”,
“PERSON HEAD ACCURATE DETECTION is not optimized”,
“REID ASSOCIATION is not optimized”,
“NEURAL DEPTH is not optimized”
]
},
“Camera Test”: {
“InternalDevicesCount”: 0,
“ZEDCount”: 0,
“error”: [
Camera not detected
Make sure the camera is plugged in or try another USB 3.0 port.”
]
},
“Devices”: {
“ASMediaFound”: false,
“CorruptedFirmware”: false,
“OC_Cam_Detected”: false,
“OtherUSBDevices”: [
{
“USBMode”: 3,
“USB_path”: “/3/2”,
“bDescriptorType”: 1,
“bDeviceProtocol”: 0,
“bLength”: 18,
“bMaxPacketSize0”: 9,
“bNumConfigurations”: 2,
“bcdDevice”: “49.0”,
“bcdUSB”: “3.0”,
“bcdUSBClass”: 0,
“bcdUSBSubClass”: 0,
“busNumber”: 3,
“iManufacturer”: 1,
“iProduct”: 2,
“iSerial”: 6,
“idProduct”: 33107,
“idVendor”: 3034
}
],
“USBList”: [
{
“USB_path”: “/3”,
“idProduct”: “0x0820”,
“idVendor”: “0x2109”
},
{
“USB_path”: “/1/1”,
“idProduct”: “0xc412”,
“idVendor”: “0x06c4”
},
{
“USB_path”: “/10”,
“idProduct”: “0x1866”,
“idVendor”: “0x0b05”
},
{
“USB_path”: “/10”,
“idProduct”: “0x1866”,
“idVendor”: “0x0b05”
},
{
“USB_path”: “/3/2”,
“idProduct”: “0x8153”,
“idVendor”: “0x0bda”
},
{
“USB_path”: “/4”,
“idProduct”: “0xc52b”,
“idVendor”: “0x046d”
},
{
“USB_path”: “/3”,
“idProduct”: “0xc05b”,
“idVendor”: “0x046d”
},
{
“USB_path”: “/1”,
“idProduct”: “0x2820”,
“idVendor”: “0x2109”
},
{
“USB_path”: “/14”,
“idProduct”: “0x0026”,
“idVendor”: “0x8087”
},
{
“USB_path”: “/5”,
“idProduct”: “0xc215”,
“idVendor”: “0x046d”
}
],
“ZEDDetected”: null,
“captureDevices”: [
],
“controllers”: [
{
“CompatibleIDs”: “”,
“Description”: “USB Root Hub (USB 3.0)”,
“DeviceName”: “”,
“Driver”: “{36fc9e60-c465-11cf-8056-444553540000}\0005”,
“Location”: “”,
“Manufacturer”: “(Standard USB HUBs)”,
“Name”: “USB”,
“PhysicalName”: “\Device\USBPDO-2”
},
{
“CompatibleIDs”: “PCI\VEN_8086&DEV_15E9&REV_06”,
“Description”: “USB xHCI Compliant Host Controller”,
“DeviceName”: “Intel(R) USB 3.1 eXtensible Host Controller - 1.10 (Microsoft)”,
“Driver”: “{36fc9e60-c465-11cf-8056-444553540000}\0002”,
“Location”: “PCI bus 6, device 0, function 0”,
“Manufacturer”: “Generic USB xHCI Host Controller”,
“Name”: “PCI”,
“PhysicalName”: “\Device\NTPNP_PCI0029”
},
{
“CompatibleIDs”: “USB\USB30_HUB”,
“Description”: “Generic SuperSpeed USB Hub”,
“DeviceName”: “”,
“Driver”: “{36fc9e60-c465-11cf-8056-444553540000}\0031”,
“Location”: “Port_#0003.Hub_#0003”,
“Manufacturer”: “(Standard USB HUBs)”,
“Name”: “USB”,
“PhysicalName”: “\Device\USBPDO-3”
},
{
“CompatibleIDs”: “PCI\VEN_8086&DEV_06ED&REV_00”,
“Description”: “USB xHCI Compliant Host Controller”,
“DeviceName”: “Intel(R) USB 3.1 eXtensible Host Controller - 1.10 (Microsoft)”,
“Driver”: “{36fc9e60-c465-11cf-8056-444553540000}\0000”,
“Location”: “PCI bus 0, device 20, function 0”,
“Manufacturer”: “Generic USB xHCI Host Controller”,
“Name”: “PCI”,
“PhysicalName”: “\Device\NTPNP_PCI0006”
},
{
“CompatibleIDs”: “USB\USB20_HUB”,
“Description”: “Generic USB Hub”,
“DeviceName”: “”,
“Driver”: “{36fc9e60-c465-11cf-8056-444553540000}\0011”,
“Location”: “Port_#0001.Hub_#0001”,
“Manufacturer”: “(Standard USB HUBs)”,
“Name”: “USB”,
“PhysicalName”: “\Device\USBPDO-5”
},
{
“CompatibleIDs”: “PCI\VEN_10DE&DEV_1ADA&REV_A1”,
“Description”: “USB xHCI Compliant Host Controller”,
“DeviceName”: “NVIDIA USB 3.10 eXtensible Host Controller - 1.10 (Microsoft)”,
“Driver”: “{36fc9e60-c465-11cf-8056-444553540000}\0001”,
“Location”: “PCI bus 1, device 0, function 2”,
“Manufacturer”: “Generic USB xHCI Host Controller”,
“Name”: “PCI”,
“PhysicalName”: “\Device\NTPNP_PCI0021”
},
{
“CompatibleIDs”: “”,
“Description”: “USB Root Hub (USB 3.0)”,
“DeviceName”: “”,
“Driver”: “{36fc9e60-c465-11cf-8056-444553540000}\0003”,
“Location”: “”,
“Manufacturer”: “(Standard USB HUBs)”,
“Name”: “USB”,
“PhysicalName”: “\Device\USBPDO-1”
},
{
“CompatibleIDs”: “”,
“Description”: “USB Root Hub (USB 3.0)”,
“DeviceName”: “”,
“Driver”: “{36fc9e60-c465-11cf-8056-444553540000}\0004”,
“Location”: “”,
“Manufacturer”: “(Standard USB HUBs)”,
“Name”: “USB”,
“PhysicalName”: “\Device\USBPDO-0”
}
],
“notTested”: [
“Please connect a camera to test the USB connection.”
]
},
“Graphics Card”: {
“deviceCount”: 1,
“deviceDriverVersion”: 12010,
“devices”: [
{
“arch”: “Turing”,
“computeCapability”: “7.5”,
“cores”: 4608,
“name”: “NVIDIA GeForce RTX 2070 with Max-Q Design”,
“totalMemoryBytes”: “8589606912”,
“totalMemoryMB”: 8191.6875
}
],
“initResult”: 0,
“valid”: [
Graphics card: NVIDIA GeForce RTX 2070 with Max-Q Design”
]
},
“Processor”: {
“OS”: “Windows 10 FallCreators”,
“avxSupported”: true,
“coreCount”: “6”,
“cpu”: “Intel(R) Core™ i7-10750H CPU @ 2.60GHz”,
“datetime”: “2024-04-04 11:17:19”,
“hyperThreading”: “1”,
“motherboard”: “GU502LW, ASUSTeK COMPUTER INC.”,
“ramCapacity”: “15.7875”,
“ramUsage”: “0”,
“ramUsed”: “8.91037”,
“threadCount”: “12”,
“valid”: [
Processor: Intel(R) Core™ i7-10750H CPU @ 2.60GHz”,
Motherboard: GU502LW, ASUSTeK COMPUTER INC.”
]
},
“ZED SDK”: {
“CUDA Toolkit version”: “V12.1.105 -->located in C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.1/bin/nvcc.exe”,
“CUDA loading”: false,
“ZED SDK Version (Diag)”: “4.0.8”,
“ZED SDK Version (RT)”: “4.0.8”,
“ai”: {
“checkAI”: true,
“cudaVersion”: 12010,
“cudnnCudartVersion”: 12000,
“cudnnVersion”: 8801,
“cudnnVersionExpected”: 8801,
“tensorVersion”: 8600
},
“binFiles”: [
“cudnn64_8.dll”,
“cudnn_adv_infer64_8.dll”,
“cudnn_adv_train64_8.dll”,
“cudnn_cnn_infer64_8.dll”,
“cudnn_cnn_train64_8.dll”,
“cudnn_ops_infer64_8.dll”,
“cudnn_ops_train64_8.dll”,
“nvinfer.dll”,
“nvinfer_builder_resource.dll”,
“nvinfer_dispatch.dll”,
“nvinfer_lean.dll”,
“nvinfer_plugin.dll”,
“nvinfer_vc_plugin.dll”,
“nvonnxparser.dll”,
“nvparsers.dll”,
“sl_ai64.dll”,
“sl_zed64.dll”,
“zlibwapi.dll”
],
“driverVersion”: “531.14”,
“drivervDllVersion”: “31.0.15.3114”,
“resourcesFiles”: [
],
“resourcesFilesExpected”: [
“objects_performance_3.2”
],
“runtimeDllVersion”: 12010,
“valid”: [
ZED SDK version: 4.0.8”,
CUDA version: V12.1.105”
]
}
}

So upon reading up on this further, it seems that the Mono_Rig and Left_eye maybe wants to connect to a physical camera? So maybe those don’t work because I don’t provide the physical camera?

Our use case is to emulate our Zed camera in Unity, so we will not be using the physical camera. I guess this is not what the plugin was designed for? It might still be useful to us (although maybe not the most computatuionally efficient) since we will be using a mono camera image at this point (so the Right_eye might be enough). But if we want to output stereo camera data from Unity this plugin might not work well for that?

@Mattttiass

That does clarify things, and as you say, the plugin is not made to emulate the ZED cameras, but to allow using ZED Cameras and the ZED SDK with Unity.
The plugin is not made to output stereo camera data, but rather to take is as input, and specifically ZED camera data. It’s not compatible with other cameras.