Hi,
I would like to make a scanner box which can scan a person and for that, it seems I need at least 12 cameras, 3 cameras in one column and 4 columns . I saw the code example on GitHub with the multi camera but the readme says it was tested with 4 cameras only. Is it possible to create a system with 12 camera or more and create the pointclouds within 3 seconds?
Thanks,
Szabina
Hi @zekanysz
Welcome to the Stereolabs community.
There’s no limitation to the number of cameras that you can use for a multi-camera system from a software point of view.
The only limitation can be from the hardware point of view. A system that must handle 12 cameras simultaneously is highly demanding, so you need a powerful CPU, a big amount of RAM (at least 64 GB DDR5), a powerful GPU with a big amount of VRAM (NVIDIA RTX with at least 24 GB of VRAM).
Hi @Myzhar
Thank you for your reply.
Could you please help me with two other questions?
I uploaded an image about a pointcloud here to show you my results (I tried to scan a doll, in the surrounding there is a wardrobe). This is also the code example from the GitHub repo. The pointcloud result is a little wavy and the 3D preview showen by the open3D is also very wavy when I start the program and try to save the pointcloud. Can you tell me why is the result is so wavy, or can I filter or correct it somehow? It seems of course that the best result is in the center, and as we go to the sides it is most noisy and have more errors. In this pointcloud the leg of the doll is useless, can I solve this to use and scan the whole body?
I don’t know if this information is important or not but I use a ProArt laptop with:
Also I would like to create the best result so I tried to create the pointcloud in HD2K resolution but I cannot scan the doll from about 1.3 meter only from further away. Is there a solution to scan a doll/person from 1.3 meter but in HD2K.
Thanks,
Szabina
Hi Szabina,
regarding the point cloud, I advise you to use the following settings to improve the quality:
Hi @Myzhar
Unfortunatelly, any setup I try to change in init function effects an empty pointcloud, like in the image below. I have no idea why. I tried a lot of settings, like min and max distance, Depth mode etc. any changes I try based on the documentation it effects pointcloud with 0 point. Do you have any idea why?
Thanks,
Szabina
Hi @Myzhar
I think the empty pointcloud is because of the NEURAL mode, the documentation says NEURAL mode Requires AI module. I don’t exactly know what it means.
I think I have a bigger problem. I set the depth_minimum_distance to 1.0 meter but it does not work only from 1.5 meter. The documentation says it should work from 0.3 meter if the focal length is 2.1mm and regarding to this Zed 2 camera this is the case so it should work from 1.0 meter but it is not, even if I give it in the constructor.
@Myzhar I downloaded the newest SDK 4.0 for windows 11 cuda 12
In the Zed Depth Viewer I got an almost perfect result with the NEURAL setting. I would like to achieve the same result from the python program.
Hi @zekanysz
If you have it working in Depth Viewer, it should work without issue in your Python app, so it’s probably an initialization issue.
Are you using the Depth sensing sample? Just adding the line:
init.depth_minimum_distance = 0.3
should do the trick (0.3 or any distance you need like 1.0). You can double check there is not too much clamping by also setting the maximum range:
init.depth_maximum_distance = 2.0
I would put these lines around here in the depth sensing sample:
[...]
def main():
print("Running Depth Sensing sample ... Press 'Esc' to quit\nPress 's' to save the point cloud")
# set NEURAL depth mode and METER unit
# ------------------------
init = sl.InitParameters(depth_mode=sl.DEPTH_MODE.NEURAL,
coordinate_units=sl.UNIT.METER,
coordinate_system=sl.COORDINATE_SYSTEM.RIGHT_HANDED_Y_UP)
# ------------------------
parse_args(init)
# depth clamping
# ------------------------
init.depth_maximum_distance = 2.0
init.depth_minimum_distance = 0.3
# ------------------------
zed = sl.Camera()
status = zed.open(init)
if status != sl.ERROR_CODE.SUCCESS:
print(repr(status))
exit()
[...]
Hi @JPlou
First of all thank you for your answer.
Yes I use the Depth sensing sample, and I tried the init parameters the same way as you suggested. I also tried to give them in the constructor, I tried with MILLIMETER, CENTIMETER but not working only from 1.5 meter. I cannot see anything about the doll from 1 meter. I attached a photo. The doll is there exactly 1 meter from the camera and in the pointcloud there is nothing.
Also in NEURAL mode the ply file is 1KB with 0 points in it. I have no idea why.
I uploaded a screen video about it on oneDrive if you willing to watch.
https://1drv.ms/u/s!AuNJWx53maPWjTPUvb7lP888wEJl?e=Y7lx2o
I would like to ask another question regarding to the multicamera setup. As I mentioned before, the final goal is a system that can scan an entire human body. To do this, we need a minimum of 12 cameras, which we would place by mounting cameras on 4 columns, 3 cameras on each column. We would like to do it in the fastest scanning speed and with the best quality of course 
If we manage to make a good quality with the NEURAL Depth mode, and from about 1.3 meter, we can move on to the next point, which is the speed. Is it possible to connect 3 cameras to some type of Jetson for the fastest scanning speed. It is not a problem if the whole body and ICP reconstruction is not that fast, but we would like to achive a 1-2 sec scanning from all of the 12 cameras. We are afraid that if we use one computer and USH Hubs per columns to connect all of the cameras to the PC, it would be very slow.
Thanks,
Szabina
Hi @zekanysz,
First for the setup, yeah, going for a distributed workflow is probably what you want here. Technically, you can run 3 ZED2i cameras on an Orin NX 16Go (our ZED Boxes are shipped ready-to-use with the SDK), with Neural depth mode, but the performance will be low (I’m calling a 5~10fps at most). However, that might be good enough for your application, I leave that call to you of course.
If I understand your project correctly, you want to use point clouds and not meshes to scan a person. I advise considering using fewer cameras, and trying to use them vertically. Not only you may be able to run them all on one high-end computer (we’ve tested up to 4) with reasonable (albeit low) performance in Neural, but the workflow is simplified and you won’t have to manage the synchronization between the 12 cameras. I also want to point out that for now, we don’t support point cloud fusion, only spatial mapping (meshes) fusion.
All that being said, it’s possible to run 3 cameras on 1 ZED Box 16Go. If you want to do anything else than retrieve the depth of each camera, however, please count 2 cameras per ZED Box or the performances will be too low.
For the Neural issue, I’d need ideally 3 things:
Sorry for the inconvenience and thanks for your understanding
Hi @JPlou,
We basically would like to use the cameras vertically. We should scan athletes too, who can be taller than 2 meter and we would like to scan from the top of the head to the toe, this is why we decided to use 3 cameras vertically in one column to make sure the whole body is covered. We only have one Zed camera right now, so we couldn’t test it with 2 or 3 cameras. Maybe we can move the one camera we have in different positions for now to make some tests, but first, the most important would be to make the python code work with the stable NEURAL Depth Mode and from about 1.3 meter far because the scanning room is already done and we don’t have more space, the 4 columns are about 1.3 meters around the doll/person and we cannot change it, because it is fixated.
Thanks,
Szabina
I need to make another post to attach an image from the scanning room. Like this maybe it will make more sense. So the 4 column is fix we would like to change the previously used intel Real Sense cameras to ZED 2 to make a better resolution and build a scanner with great quality and speed.
Oh … another quick question. Should we use the ZED i cameras or is the ZED 2 cameras perfect too? I can see we can decide the parameters for the ZED i but the ZED 2 can be enough and it is cheaper.
Szabina
Hi @JPlou
I tried to test the whole thing in another computer in a very strong PC. This PC has some other purposes but anyway I wanted to try, but there I have other issues. I created the whole environment and everything it needs to run but the code example says “No module named ‘pyzed.sl’; pyzed is not a package”. Doesn’t matter what I try reinstall everything I cannot run the code at all, I do everything the same way as I did it on my laptop.
Szabina
Hi @zekanysz, sorry for the delay.
- I can make an SVO recording for sure. What should I do only start the github example and save it in an svo file? Or should I stand in front of the camera within 1.3 meter or position the doll or box or something in front of the camera?
Yes, that’s it. Show the problematic situation, so start from the range you need, then the range where the detection starts. We’ll concentrate on why is doesn’t work under this. You can record it using ZED Explorer, it’s easier than the sample, just enable the “H265 LOSSY” or “H264 LOSSY” if 265 is not available in the “General” tab of the settings.
Should we use the ZED i cameras or is the ZED 2 cameras perfect too?
I advise going for the ZED 2i, they are much more reliable (better enclosure) and use an external USB connection, avoiding rare but very difficult-to-fix USB issues.
“No module named ‘pyzed.sl’
