Retrieving point clouds in GPU memory with the Python SDK

Software Python
1

Hi there,

I’m working on a multi ZED X system with an Orin NX Jetson.

I’m retrieving point clouds from the ZED X’s using the retrieve_measure() function, but I’m getting this error when trying to call `retrieve_measure(…, mem_type=sl.MEM.GPU, …):

File "/usr/lib/python3.10/enum.py", line 437, in __getattr__
    raise AttributeError(name) from None
AttributeError: GPU. Did you mean: 'CPU'?

Then I noticed read in the docs that the Python SDK doesn’t support GPU access.

Question: Is there a way to access point clouds from ZED X cameras on the GPU?

I’m trying to do point-cloud processing on the GPU using CuPy for performance, and it’d be great to avoid having to retrieve the point-clouds on the CPU then load the onto the GPU again as this takes time.

Appreciate any advice - thanks.

2

Hi @andrew.stringfield,

Thank you for reaching out to us!

GPU data retrieval using the Python SDK is still experimental as of the 5.0 release but should still be working fine (see this post for some small CuPy talk - Python GPU Retrieval Examples Request)

If you have the following:

I suspect that you are using the 4.2 version of the PyZED package since there, only the sl.MEM.CPU was available. Can you upgrade to the 5.0 instead?

Please note that as of the 5.1 release (which will be released somewhere in September), we’ll introduce more example on how to use CuPy along with the ZED SDK and how to benefit from its acceleration. We’ll also update the docs to remove the line you mentioned (and others).

1 Like
3

@LuckyJ Thanks for the update!

I’ve upgraded to ZED SDK 5.0 and can access sl.MEM.GPU now.

However, when trying to retrieve a point cloud I get this error:

File "pyzed/sl.pyx", line 6807, in pyzed.sl.Mat.get_data
ValueError: Provided MEM type doesn't match Mat's memory_type.

Here’s my code snippet, with verification that the sl.Mat() I’m using is also sl.MEM.GPU:

# Initialise the sl.Mat for storing point clouds
point_cloud = sl.Mat(memory_type=sl.MEM.GPU)

...

# Select memory type to use
sl_memory_type = sl.MEM.GPU
mat_mem_type = point_cloud.get_memory_type()

# Log some info to check types
print(f"mat_mem_type: {mat_mem_type}")
print(f"type(mat_mem_type): {type(mat_mem_type)}")
print("sl_memory_type:", sl_memory_type)
print(f"type(sl_memory_type): {type(sl_memory_type)}")
assert mat_mem_type == sl_memory_type, f"Expected point cloud memory type to be {sl_memory_type}, but got {mat_mem_type}"

# Retrieve the point-cloud on the GPU
camera.retrieve_measure(
                point_cloud, 
                measure=sl.MEASURE.XYZ,
                mem_type=sl_memory_type,
            )

# Try to get the array
result = point_cloud.get_data()  # <-- Fails here

Am I using the API correctly to try and get the GPU array? Do I still need to pass in a mem_type to retrieve_measure() given that the point_cloud: sl.Mat() is already allocated on the GPU?

4

Hi @andrew.stringfield,

Happy to read that!

The issue is that the get_data method accepts an argument to specifiy the memory_type you want to retrieve the data from. By default, it is sl.MEM.CPU, see sl.Mat.get_data() doc for more info.

# Try to get the array
result = point_cloud.get_data() # <-- Fails here
result = point_cloud.get_data(sl.MEM.GPU)  # <-- Should work fine

:warning: Note that the comment Default: MEM.CPU (you cannot change the default value) is wrong there and will be removed in the 5.1 release. MEM.GPU is possible as of the 5.0 release

Yes, because by default, the mem_type argument of retrieve_measure is sl.MEM.CPU (see sl.Camera.retrieve_measure() doc for more info)

5

Thanks @LuckyJ, I didn’t check the documentation for get_data() - that’s fixed the error!

1 Like