Spatial mapping from streaming depth data on jetson

mememe · April 5, 2022, 8:53pm

Hi - Im using Jetson Nano as well as ZED box … ive done my best to insatll everything correctly … and have had good results … however.

the sample ‘streaming’ actually states that it “Camera Streaming Shows how to stream the ZED stereo video on IP network, decode the video and display its live 3D point cloud.” … it does NOT make a pointcloud from the streamed data … infact you need to change the code to get anything other than a left rgb camera view.

i have a zed2i and a zed mini … so i can try combinations of either camera streaming to eaither computer .
;
if i update the spatial mapping sample to take it’s feed from the streamed camera ip using:

init.set_from_stream(“10.0.0.130”, 30000)

i get an error about freeglut (open gl i think) … i have tried every which way to solve and search results aren’t super helpful … would really love some help please.

im getting :

Running Spatial Mapping sample … Press ‘q’ to quit
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
freeglut (foo): ERROR: Internal error in function fgOpenWindow
nvbuf_utils: dmabuf_fd 1128 mapped entry NOT found
nvbuf_utils: Can not get HW buffer from FD… Exiting…
nvbuf_utils: dmabuf_fd 1128 mapped entry NOT found
nvbuf_utils: Can not get HW buffer from FD… Exiting…
nvbuf_utils: dmabuf_fd 1128 mapped entry NOT found
nvbuf_utils: Can not get HW buffer from FD… Exiting…

ive tried updating opengl (as best as i can figure) and installing freeglut seperately … but its already there.

im using latest sdk - both machines running ubuntu

nano is 18.04 lts - with a TegraX1 nvgpu
ZEDBox is 18.04 lts with TegraX2 nvgpu

both are 64bit

heeeeeeeeeeelp

Myzhar · April 6, 2022, 8:08am

Hi @mememe
can you please describe your settings in detail?

What device are you using as a streamer?
What device are you using as a receiver?
What version of the ZED SDK are you using with both of them?

mememe · April 6, 2022, 9:53am

I have tried both the zed2i and zed mini on both the zedbox and the nano … both running the latest sdk ( downloaded and installed two days ago) …

I have just set the stream sample to stream ( on some occasions I have set the depth mode to performance rather than none just in case) and I have tested that the receiver sample works … and that I can change what is being displayed by altering the program to display either left or right view or indeed the depth image… however there is no code in the receiver program to actually build a pointcloud/ mesh as it states on the Web page… so I get the sample of spatial mapping and change its input to be from stream with the ip address … and then it errors… I’ve tried it both ways round… so either there is something wrong with my install or something wrong with the way I’ve changed the code… I’m now wresting to get pyzed onto my Windows machine so I can see if that can do it… but as ever none of it is going well lol…

mememe · April 6, 2022, 12:07pm

so in the streaming example … i change nothing … like i say … ive tried changing DEPTH.MODE.NONE to DEPTH.MODE.PERFORMANCE but it made no change to the errors

in the receiver program … i did nothing and it worked fine … so i used that to test connections accross local network .

then i edited spatial mapping :

(BTW i also changed init_params to just innt as thats whats used in the sender/reciever files and i was getting confused with the non uniformity of it.)

in the first try i editied the spatial mapping sample by removing its SVO argument and a;dding the argument to add an ip address - and then obviously tried starting it with the ip argument

def main():
print(“Running Spatial Mapping sample … Press ‘q’ to quit”)

# Create a Camera object



zed = sl.Camera()

# Create a InitParameters object and set configuration parameters
init = sl.InitParameters()
init.camera_resolution = sl.RESOLUTION.HD720  # Use HD720 video mode
init.depth_mode = sl.DEPTH_MODE.PERFORMANCE
init.coordinate_units = sl.UNIT.METER         # Set coordinate units
init.coordinate_system = sl.COORDINATE_SYSTEM.RIGHT_HANDED_Y_UP  # OpenGL coordinates

if (len(sys.argv) > 1) :
 ip = sys.argv[1]
 init.set_from_stream(ip)
else :
 print('Usage : python3 streaming_receiver.py ip')
 exit(1)


# If applicable, use the SVO given as parameter
# Otherwise use ZED live stream
###if len(sys.argv) == 2:
   ### filepath = sys.argv[1]
   ### print("Using SVO file: {0}".format(filepath))
   ### init.set_from_svo_file(filepath)

# Open the camera
status = zed.open(init)
if status != sl.ERROR_CODE.SUCCESS:
    print(repr(status))
    exit()


runtime = sl.RuntimeParameters()
mat = sl.Mat()

in the second try i just added the ip address to the program:

def main():
print(“Running Spatial Mapping sample … Press ‘q’ to quit”)

# Create a Camera object
zed = sl.Camera()

# Create a InitParameters object and set configuration parameters
init = sl.InitParameters()
init.set_from_stream("10.0.0.130", 30000) # Specify the IP and port of the sender
#init.camera_resolution = sl.RESOLUTION.HD720  # Use HD720 video mode
init.coordinate_units = sl.UNIT.METER         # Set coordinate units
init.coordinate_system = sl.COORDINATE_SYSTEM.RIGHT_HANDED_Y_UP  # OpenGL coordinates

# If applicable, use the SVO given as parameter
# Otherwise use ZED live stream
if len(sys.argv) == 2:
    filepath = sys.argv[1]
    print("Using SVO file: {0}".format(filepath))
    init.set_from_svo_file(filepath)

obviously these are just snippets of the programs … but these were the only changes i made apart from commenting out any refernces to the SVO etc …

seemed to ALMOST work … but just not able to load the camera feed into a freeglut viewer … or somesuch … its starting to hurt my head

mememe · April 6, 2022, 2:09pm

YIKES - - so i cant get python api working on my w10 desktop so im trying my w10 laptop … and its JUST as bad … its running int certificate errors just trying to run the get_python_api.py file … for GOODNESS SAKE why ???

C:\Program Files (x86)\ZED SDK>python get_python_api.py
→ Downloading to ‘C:\Program Files (x86)\ZED SDK’
Detected platform:
win
Python 3.10
ZED SDK 3.7
→ Checking if https://download.stereolabs.com/zedsdk/3.7/win/py310 exists and is available
Traceback (most recent call last):
File “C:\Users\USER\AppData\Local\Programs\Python\Python310\lib\urllib\request.py”, line 1348, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File “C:\Users\USER\AppData\Local\Programs\Python\Python310\lib\http\client.py”, line 1282, in request
self._send_request(method, url, body, headers, encode_chunked)
File “C:\Users\USER\AppData\Local\Programs\Python\Python310\lib\http\client.py”, line 1328, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File “C:\Users\USER\AppData\Local\Programs\Python\Python310\lib\http\client.py”, line 1277, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File “C:\Users\USER\AppData\Local\Programs\Python\Python310\lib\http\client.py”, line 1037, in _send_output
self.send(msg)
File “C:\Users\USER\AppData\Local\Programs\Python\Python310\lib\http\client.py”, line 975, in send
self.connect()
File “C:\Users\USER\AppData\Local\Programs\Python\Python310\lib\http\client.py”, line 1454, in connect
self.sock = self._context.wrap_socket(self.sock,
File “C:\Users\USER\AppData\Local\Programs\Python\Python310\lib\ssl.py”, line 512, in wrap_socket
return self.sslsocket_class._create(
File “C:\Users\USER\AppData\Local\Programs\Python\Python310\lib\ssl.py”, line 1070, in _create
self.do_handshake()
File “C:\Users\USER\AppData\Local\Programs\Python\Python310\lib\ssl.py”, line 1341, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:997)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “C:\Program Files (x86)\ZED SDK\get_python_api.py”, line 301, in
urllib.request.urlretrieve(whl_file_URL, whl_file)
File “C:\Users\USER\AppData\Local\Programs\Python\Python310\lib\urllib\request.py”, line 241, in urlretrieve
with contextlib.closing(urlopen(url, data)) as fp:
File “C:\Users\USER\AppData\Local\Programs\Python\Python310\lib\urllib\request.py”, line 216, in urlopen
return opener.open(url, data, timeout)
File “C:\Users\USER\AppData\Local\Programs\Python\Python310\lib\urllib\request.py”, line 519, in open
response = self._open(req, data)
File “C:\Users\USER\AppData\Local\Programs\Python\Python310\lib\urllib\request.py”, line 536, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File “C:\Users\USER\AppData\Local\Programs\Python\Python310\lib\urllib\request.py”, line 496, in _call_chain
result = func(*args)
File “C:\Users\USER\AppData\Local\Programs\Python\Python310\lib\urllib\request.py”, line 1391, in https_open
return self.do_open(http.client.HTTPSConnection, req,
File “C:\Users\USER\AppData\Local\Programs\Python\Python310\lib\urllib\request.py”, line 1351, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:997)>

C:\Program Files (x86)\ZED SDK>

obraun-sl · April 7, 2022, 3:04pm

Did you try to use the zed spatial mapping sample with SVO or Live camera as input? Since the error seems to come from freeglut, I don’t think it’s related to streaming input
For the python issue on Windows, try pip install certifi or launch with administration rights