I am using Camera::setRegionOfInterest in Unity, which was added in ZED SDK 3.8.0.
One black and white image is attached.
There is a mesh-like frame on the screen.
In order to get the frame to be removed from this screen, I have created a B&W image in a way that omits within a specified distance from the depth information, and after outputting the file, I use the zedMat.Read function to create a zedMat and pass it to the setRegionOfInterest function.
As a result, I could not get a good self-location estimation with this method.
With the above in mind, I have four questions.
ă»Is the setRegionOfInterest function reflected in Unity?
ă»Is there any way to verify that the coverage is correct after executing the setRegionOfInterest function? (sl.ERROR_CODE was SUCCESS)
ă»Does the white part passed to the setRegionOfInterest function have to be a square?
ă»Is it OK to execute the setRegionOfInterest function every second?
I would also like to add a note about a problem I encountered when executing ZEDMatâs SetValue function in Unity.
ă»The SetValue function is not functioning properly.
The function can be executed (sl.ERROR_CODE is SUCCESS). I can also check the change of pixel values in the image, but no matter what value is entered in the ref byte, it only results in the same color.
Therefore, I could not create a black and white image without using the ZEDMat.Read function.
This is causing the cost of outputting the file and then reloading the file.
var zedCam = ZEDManager.GetInstance(sl.ZED_CAMERA_ID.CAMERA_ID_01);
var pngMat = new ZEDMat();
pngMat.Create(new sl.Resolution((uint)zedCam.zedCamera.ImageWidth, (uint)zedCam.zedCamera.ImageHeight), ZEDMat.MAT_TYPE.MAT_8U_C1,ZEDMat.MEM.MEM_CPU);
pngMat.Read(pngPath);
ERROR_CODE SOIERR = zedCam.zedCamera.SetRegionOfInterest(pngMat);
Hi,
The White part does not have to be a square, there is no limitation on the shape of the mask used.
The setRegionOfInterest() function has not been made to use called every frame, it will impact the performance a bit but it is completely possible. However, in the plugin you can choose the minimum depth range used by the SDK for the Positional tracking.
This parameter is available here: https://github.com/stereolabs/zed-unity/blob/master/ZEDCamera/Assets/ZED/SDK/Helpers/Scripts/ZEDManager.cs#L920
Can you show me how did you use the SetValue, and why do you think itâs not working properly? A small piece of code that reproduces the issue would be appreciated.
Best,
Benjamin Vallon
Stereolabs Support
@BenjaminV Thank you for your reply.
I understand your point that it is better to use depthMinRange.
One point is that you did not answer how to check after the setRegionOfInterest function is executed, so let me explain the intent of this question.
I was thinking about the DepthMap in the URL above.
If the same thing happens in Unity, I would expect the Depth Texture of the Material in the Frame gameObject to change as well, but in my environment I donât see any change before and after the setRegionOfInterest function is executed. I thought that maybe nothing is specified outside of the region.
Can you show me how did you use the SetValue, and why do you think itâs not working properly? A small piece of code that reproduces the issue would be appreciated.
I created the sample program.
Development
ă»Unity 2020.3.42f1
ă»Windows10
ă»ZED SDK 3.8.2
ă»ZED Unity 3.8.0
ZEDManager zedMngr = ZEDManager.GetInstance(sl.ZED_CAMERA_ID.CAMERA_ID_01);
ZEDMat pngMat = new ZEDMat();
pngMat.Create(new sl.Resolution((uint)zedMngr.zedCamera.ImageWidth, (uint)zedMngr.zedCamera.ImageHeight), ZEDMat.MAT_TYPE.MAT_8U_C1,ZEDMat.MEM.MEM_CPU);
byte matBlack = (byte)0x0;
byte matWhite = (byte)0xFF;
ERROR_CODE err_cd = pngMat.SetTo(ref matBlack,ZEDMat.MEM.MEM_CPU);
for(int x = 0;x < zedCam.zedCamera.ImageWidth ; x++)
{
for(int y = 0; y< zedCam.zedCamera.ImageHeight; y++)
{
if(x < zedCam.zedCamera.ImageWidth / 3)
{
err_cd = pngMat.SetValue(x,y,ref matWhite,ZEDMat.MEM.MEM_CPU);
}
}
}
err_cd = pngMat.Write(âC:\ZEDMat.pngâ));
This program produced a single gray png image.
Also, all sl.ERROR_CODE were SUCCESS.
Hi,
I should have fixed it with the last commit. Please pull the master branch (https://github.com/stereolabs/zed-unity/tree/master) and try again, it should work now.
Do not forget to set the Sensing mode to âSTANDARDâ as well. This parameter is available in the ZED Manager inspector window, in the âAdvanced Settingsâ section.
Best,
Benjamin Vallon
Stereolabs Support
1 Like
I implemented it in the latest master branch and it now works as expected.
It is nice to know that when I change the Confidence Threshold in ZED Manager and play the svo file, the position in ZED Manager also changes, making it easier to test.
It already fulfills the function I want to perform, but there is one point I am wondering if it is performing correctly.
I am aware that using the depthMinRange variable changes the position of the GameObject in the ZED Manager in the same way as the SetRegionOfInterest function. However, when I actually changed the value of depthMinRange from -1.0f to 2.0f in line 920 of the ZEDManager.cs, the Position only moved in the same way as -1.0f. This was confirmed by playing back the same svo file.
Does this mean that the depthMinRange variable does not meet our purpose?
