ZED SDK 3.3/ CUDA 11.1/Unity/Ubuntu 18.04

IshanBhatnagar14 · February 2, 2021, 10:56am

Hello ,

I have downloaded the ZED Unity Plugin , I am running Ubuntu 18.04 , ZED SDK 3.3 . I can successfully import the plugin , but i am unable to add camera object in my Hierarchy. I am getting " Unable to instantiate prefab, Prefab may be broken. I have ZED 2 Camera !!

I have browsed the Unity forum and have tried the following.

Reloaded the prefab.

BenjaminV · January 25, 2021, 4:41pm

Hi,

Thanks for reaching us out.

It might come from a missing script.
Did you try to play one of the example scene to see if it is working ?
If not, what version of Unity are you using ? (I’m personally using Unity 2019.4.10)

You can also try to reimport the prefab. For that, right click on the prefab, then click on Reimport.

Best regards,
BV

IshanBhatnagar14 · January 28, 2021, 1:30am

Hello , i am using Unity 2019.4.18f1 on Ubuntu 18.04 , i have reinstalled the 3.3 SDK , but NOW Unity is saying ZED SDK is not installed, when i am trying to use Unity.

BenjaminV · January 28, 2021, 8:21am

Hi,

This error appears when you are not using the same plugin version than the SDK version.
In the mean time, we have released the SDK 3.4 (and the Unity plugin 3.4), so make sure you have installed the 3.3 (or 3.4) version for both of them.

Best regards,
BV

IshanBhatnagar14 · January 28, 2021, 12:55pm

Please check 3.4 Unity plugin in not like 3.3 , i cannot import it as a Unity Package , any clues what is wrong ? Its not a unity package extension , if i rename the file to .Unitypackage , even than cannot import !!

BenjaminV · January 28, 2021, 1:31pm

Hi,

Only the source code on the plugin was available for the 3.4 (a compressed folder).
I added the unitypackage for 3.4.

Best regards,
BV

IshanBhatnagar14 · January 28, 2021, 4:08pm

I have upgraded to ZED SDK 3.4 and running Unity 2019.4.18f1 Personal . My Zed SDK runs fine from and all other things are working .

Still Unable to use Unity , I have attached some errors,which i am getting !!

DllNotFoundException: sl_unitywrapper
NullReferenceException: Object reference not set to an instance of an object
ZEDRenderingPlane.OnRenderImage (UnityEngine.RenderTexture source,
UnityEngine.RenderTexture destination) (at
Assets/ZED/SDK/Helpers/Scripts/Display/ZEDRenderingPlane.cs:1327)
UnityEngine.GUIUtility:ProcessEvent(Int32, IntPtr) (at
/home/bokken/buildslave/unity/build/Modules/IMGUI/GUIUtility.cs:197)

X server took longer than 2005 milliseconds to respond to SetGtkWindowSizeAndPosition.
UnityEditor.TooltipView:Show(String, Rect, GUIView)

IshanBhatnagar14 · February 1, 2021, 11:36am

Benjamin, Any idea, how to get this working?

BenjaminV · February 1, 2021, 11:56am

Hi,

You can first verify that the “libslunity_wrapper.so” lib is present in the plugin (at this location : https://github.com/stereolabs/zed-unity/tree/master/ZEDCamera/Assets/ZED/SDK/Plugins/linux).
Run ldd libsl_unitywrapper.so in a terminal at the location of the lib and send me the output.
Then, can you confirm that you installed the 3.4 version for the ZED SDK and the ZED Unity plugin ?
And, if you can run the ZED Diagnostic tool (within the ZED SDK folder, in Tools), and send me the result (the JSON file), it would be nice.

Best regards,
BV

IshanBhatnagar14 · February 1, 2021, 1:50pm

Yes shared Library is in under the Linux plugin

attaching Screenshot of result, I got with ldd libsl_unitywrapper.so

I confirm that Zed sdk 3.4 is installed.

Attaching Json file for Zed diagnostic ( Sorry UPLOAD OF JSON FILE IS NOT ALLOWED ON YOUR WEBSITE)

Also attaching Screenshots of error , I am getting.

BenjaminV · February 1, 2021, 5:21pm

Hi,

Thanks for your help, the issue is solved now.
The lib was built on Ubuntu20 which prevented the plugin to work on previous versions.

You can now download the 3.4 unity package again and your issue will be fixed.
Do not forget to delete your old package before importing the new one.

Best regards,
BV

IshanBhatnagar14 · February 2, 2021, 10:39am

Hi Benjamin,

There are certain issues:

Only the following Examples are working

Dark Room
Planetarium
SIMPLE MR
Object Placement

Rest All Not working !!

Spatial Mapping Crashes the Unity App and app closes. (Error png enclosed)
spatial_Mapping_Error1332×87 13.2 KB

objdetect1307×341 45.9 KB

simple_planetection1340×98 7.65 KB
Object detection not working, error png enclosed !!

IshanBhatnagar14 · February 3, 2021, 10:03am

Benjamin,

I had a look at issues in Github, copied the Cuda bin and library files into unity Editor. Now only Spatial Mapping is not working.

Object detection demo works now, please let me know, can this spatial mapping issue be resolved?

Unity is crashing as soon as i start Spatial Mapping !!

obraun-sl · February 3, 2021, 10:35am

Hi,
Try to change this line here

github.com

stereolabs/zed-unity/blob/master/ZEDCamera/Assets/ZED/SDK/NativeInterface/ZEDCamera.cs#L2139


/// </summary>
/// <param name="resolution_meter">Spatial mapping resolution in meters.</param>
/// <param name="max_range_meter">Maximum scanning range in meters.</param>
/// <param name="saveTexture">True to scan surface textures in addition to geometry.</param>
/// <returns></returns>
public sl.ERROR_CODE EnableSpatialMapping(SPATIAL_MAP_TYPE type, float resolution_meter, float max_range_meter, bool saveTexture = false)
{
    sl.ERROR_CODE spatialMappingStatus = ERROR_CODE.FAILURE;
    //lock (grabLock)
    {
        spatialMappingStatus = (sl.ERROR_CODE)dllz_enable_spatial_mapping(CameraID, (int)type,resolution_meter, max_range_meter, System.Convert.ToInt32(saveTexture), 4096);
    }
    return spatialMappingStatus;
}
/// <summary>
/// Disables the Spatial Mapping process.
/// </summary>
public void DisableSpatialMapping()
{
    lock (grabLock)

Set the last parameter to 2048 instead of 4096.
For some reason, Unity on ubuntu does not support memory higher than 4Gb. You can set any number below 4096 (4095 works for example).

IshanBhatnagar14 · February 3, 2021, 10:57am

Hi Obraun ,

I have 64Gb of Ram and 10gb+ of Vram , will i not be able to take advantage of this on Unity in Ubuntu 18.04 for Spatial memory , by restricting the size , will this affect performance of spatial mapping ?

IshanBhatnagar14 · February 3, 2021, 12:02pm

Thank You so much Zed Team , especially Obraun and Benjamin . Finally Spatial Mapping is Working.

I replaced the value with 4095 !! in line as specified by Obraun.

But How to optimize operation of Zed Camera on Unity SDk , Camera fails to load certain times, it continues to run after the app has stopped.

I have to disconnect it and connect it again for it to work.

How to make the operation of ZED 2 camera smoother in Unity Sdk.

Like after i tired Spatial Imaging, stopped the camera and try to restart, it fails to load.

I have to disconnect and reconnect ZED 2 Camera.

obraun-sl · February 3, 2021, 1:19pm

We will check that. I think that’s only on Linux since I did not encounter this issue on windows.

Regarding spatial mapping, setting to a lower value does not impact the performance but you won’t be able to scan very large areas.
However 4095 should be enough.

IshanBhatnagar14 · February 3, 2021, 1:24pm

What to do , if i want to scan large areas, i should switch to build on Windows ( Unity ) ?

IshanBhatnagar14 · February 4, 2021, 2:37pm

Hi, My plane detection sample in Unity is showing, a dark black mapped surface instead of mesh in your website, do I need to add some component to Zed Plane Detection Manager?

BenjaminV · February 5, 2021, 8:45am

Hi,

Commenting this line https://github.com/stereolabs/zed-unity/blob/master/ZEDCamera/Assets/ZED/SDK/Helpers/Shaders/PlaneDetection/GeometryWirePlane.shader#L30 should fix the issue.

Best regards,
BV