Need help in DEM Creation with ZED2i

Software Python
1

Hello,

I have a Jetson Nano Developer Kit paired with my ZED2i camera. My goal is to generate a digital elevation model of individuals standing in front of the camera using pyzed. Could someone assist me with the code for this task? I have all the latest SDK and libraries installed.

Thanks in advance

2

Hi @Shobhit,

For dynamic objects such as people, I would suggest using our object detection module and retrieving the 3D bounding box information to create the elevation map.

You can find samples which retrieve bounding boxes on our GitHub here: https://github.com/stereolabs/zed-sdk/tree/master/object%20detection

3

Hi @mattrouss .

Thank you for your response. I tried the approach which you told but I still not able to achieve the DEM. This is the code which I tried

import pyzed.sl as sl
import cv2
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
from scipy.ndimage import median_filter
from scipy.interpolate import griddata
import os
from datetime import datetime

# Initialize the ZED Camera
zed = sl.Camera()

# Set configuration parameters for the ZED
init_params = sl.InitParameters()
init_params.camera_resolution = sl.RESOLUTION.HD720  
init_params.camera_fps = 60  

# Open the camera
err = zed.open(init_params)
if err != sl.ERROR_CODE.SUCCESS:
    exit(1)

# Enable object detection with body tracking
obj_param = sl.ObjectDetectionParameters()
obj_param.enable_tracking = True

# Set runtime parameters after opening the camera
runtime_parameters = sl.RuntimeParameters()

# Prepare new image size to retrieve half-resolution images
image_size = zed.get_camera_information().camera_configuration.resolution
image_size.width = image_size.width / 2
image_size.height = image_size.height / 2

# Create and set RuntimeParameters after opening the camera
runtime_parameters = sl.RuntimeParameters()

# Capture 50 frames and stop
i = 0
image = sl.Mat(image_size.width, image_size.height, sl.MAT_TYPE.U8_C4)
runtime_parameters.measure3D_reference_frame = sl.REFERENCE_FRAME.CAMERA

while i < 50:
    # A new image is available if grab() returns SUCCESS
    if zed.grab(runtime_parameters) == sl.ERROR_CODE.SUCCESS:
        # Retrieve left image
        zed.retrieve_image(image, sl.VIEW.LEFT)
        
        # Object detection
        objects = sl.Objects()
        zed.retrieve_objects(objects, obj_param)
        
        # For each object detected, retrieve the bounding box information
        for obj in objects.object_list:
            bounding_box = obj.bounding_box_3d
            
            # Use bounding box information to generate DEM
            # Apply median filtering
            filtered_box = median_filter(bounding_box, size=5)
            
            # Apply interpolation
            grid_x, grid_y = np.mgrid[0:1:100j, 0:1:200j]
            points = np.random.rand(1000, 2)
            values = np.random.rand(1000)
            grid_z = griddata(points, values, (grid_x, grid_y), method='cubic')
            
            # 3D visualization
            fig = plt.figure()
            ax = fig.add_subplot(111, projection='3d')
            ax.plot_surface(grid_x, grid_y, grid_z)
        
        # Free up memory
        del objects
        del bounding_box
        del filtered_box
        del grid_x
        del grid_y
        del grid_z
        del fig
        del ax
        
        i = i + 1

# Save the 3D visualization as a PNG image
timestamp = datetime.now().strftime('%Y-%m-%d_%H-%M-%S')
plt.savefig(f'dem_{timestamp}.png')
plt.close(fig)

# Close the camera
zed.close()

I am very new to ZED SDK. Could you please help me with the code snippet to generate the DEM?

Thanks in Advance

4

Hi @Shobhit,

As this feature is currently not supported by the ZED SDK, I will not be able to support you fully in producing this DEM.

I can suggest that you use the 3D bounding box information (the filtered_box variable is unused in your sample) and set the grid_map value to the height of the bounding box.

I’d be happy to answer any other questions about issues or errors that you have with the ZED SDK.

Best,
Matthew