I just want to piggyback on here with more information because I noticed very similar behavior in ZEDfu and also the python API, and was actually planning to create a forum post about this.
@punnlert When you run ZEDfu and try to create a textured mesh, could you try switching the tracking algorithm to GEN_1 instead of GEN_3? I’m not sure why this happens, but when I use GEN_1 the texturing seems to work correctly. The tracking in GEN_1 doesn’t work as well for certain types of environments, but it may be a good enough stop gap measure for you until a bug fix is in place.
I’ve spent a couple days so far troubleshooting this problem so hopefully some of this extra information will be helpful to the stereolabs team when trying to figure out the issue. Since I’ve been able to replicate the issue in both ZEDfu and the python API and I think it may be the same underlying issue affecting both, I’m just going to focus on how this issue arises in ZEDfu so that we can eliminate the variation of different people’s code. Here is what I discovered during troubleshooting:
Initially when I was looking at the textured mesh, it appeared that the mesh was only appearing in areas which were “seen” in the initial pose of the camera at the first frame. The obvious “cone” shape on the right of punnlert’s second image is what I was seeing as well and that cone usually looked somewhat similar to the FOV of the camera. But when I tried more scenes from other SVO files, it seems more complex than just only things in the first frame having texture. I did a test with two rooms in a building that share a common wall. In one room the wall is painted gray, and in the other room the other side of that same wall is painted light green and has a TV mounted on it. The SVO starts with the camera in the first room with the gray wall. The second room is not visible in the SVO until about halfway through the recording. But when a textured mesh is created in ZEDfu, both sides of the wall are present, the gray side and the green side with the TV visible. Obviously the green side and the TV weren’t “seen” in the first frame of the SVO, but they still appeared in the textured mesh. Note that this behavior of the back side of objects being present in the textured mesh seems to be some what random and difficult to consistently reproduce.
I thought this might be a case where the mapping module is just running out of memory and failing to provide a warning or error. I’ve recorded some extremely short SVO files and set the allowed memory at different steps between default and 8GB, but that seemed to make no difference. My system has 24GB of VRAM and 32GB of system RAM so there should be plenty.
Lastly, I have always had this issue with ZEDfu where the texture doesn’t apply properly, but the behavior when using the python API has changed slightly. I started using the ZED2i earlier this year and at the time the latest SDK was 5.2.3. I followed the spatial mapping tutorial here, and when I added the save_texture and .apply_texture() everything was working as intended. When I updated to 5.4.0 and ran the same code again, the textured mesh was only appearing in the places seen in the first pose and cutting out most of the scene.
The only thing that found was actually fixing the problem was switching the tracking from GEN_3 to GEN_1.
I tried to do my best and include all the information I have so hopefully this provides enough background for the team to be able to replicate this issue and get a bug fix in place.
