setRegionOfInterest() not working with object or body tracking? SDK 4.0

I have been following this example which shows how to generate a region of interest mask: zed-sdk/main.cpp at master · stereolabs/zed-sdk · GitHub

The mask appears properly in the output (essentially just combining the mask pixels with the cvImage), but this line does nothing: zed.setRegionOfInterest(slROI);

I assumed the camera input would limit the tracking to only whats visible in the mask. But after adding in either object or body tracking, it’s still detecting objects outside the mask?

Including a screenshot which demonstrates the issue- the yellow boxes in the top left image show tracked people, but they are outside the masked area (blue outline):

Minimal code sample:

cv::Mat ROI;
ROI = cv::Mat(720, 1280, CV_8UC1);
ROI.setTo(0);
cv::Rect selection_rect;
selection_rect.x = 900;
selection_rect.y = 0;
selection_rect.width = 600;
selection_rect.height = 600;
cv::rectangle(ROI, selection_rect, cv::Scalar(250), -1);
auto resolution = camera.getCameraInformation().camera_configuration.resolution;
sl::Mat slROI(resolution, MAT_TYPE::U8_C1, ROI.data, ROI.step); 	
camera.setRegionOfInterest(slROI);

Hi,

It seems like a bug. Currently the mask is only applied to the depth information and not on image. The Object Detection module only outputs 2D detection, and it’s only merged with the depth later.

Thanks for the response @alassagne. If the mask is only applied to the depth- the code comments are misleading: ‘Defines a region of interest to focus on for all the SDK, discarding other parts’.

Is there any workaround for object or body tracking to limit the tracking area, or do I need to wait until the bug is fixed in a new release?

You can manually remove the object / bodies outside of the zone, but indeed a fix is required to do this properly.